Method, system, device and medium for providing tutoring strategy based on topic

Abstract

The present invention relates to the field of image processing technology, and is particularly suitable for searching for questions by taking photos. In view of the current situation where the auxiliary guidance teaching strategy may not correspond to the question content in the input question image, the present invention proposes a method, system, device and medium for providing a tutoring strategy based on a question, aiming to solve the technical problem of how to more accurately provide corresponding tutoring strategies by searching for questions through computer processing images. To this end, the method of the present invention searches through the text information of the question image in combination with the feature image of the image and predicts the degree of matching between the searched text title and the question image through similarity calculation, machine learning model, image information matching, etc., and obtains the highest degree of the question as the detection result to provide various corresponding tutoring strategies. Thereby improving the efficiency of specific image search and accurately providing tutoring strategies.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and is particularly suitable for image and text matching processing. More specifically, it relates to a method, system, device and medium for providing a question-based tutoring strategy.

[0002] The topic referred to in the present invention refers to the content of various related test questions or problems including but not limited to online or offline user exercise books, answer sheets, answer papers, exercise books, etc. Background Art

[0003] In the existing technology, during online or offline teaching or auxiliary teaching / tutoring, image capture of corresponding question images can be performed and provided to other people (tutors / teachers / experts / classmates, etc.) or machines, collectively referred to as online or offline tutors, for guidance, scoring or teaching.

[0004] However, in the existing technology, the tutor only provides teaching, guidance or grading based on the search results, and cannot determine whether the searched questions in the image actually match the correct text questions actually stored in the question bank, and thus cannot correctly guide the user to obtain the correctly matched tutor for tutoring. In other words, it is impossible to accurately and correctly determine whether the found questions actually correspond to the content of the image, and it is impossible to accurately provide tutors or learning tutoring strategies that match the questions, resulting in a poor user experience.

[0005] Therefore, it is necessary to improve the existing scheme of providing teaching and tutoring learning strategies based on question images to ensure the correctness and accuracy of the strategies, improve efficiency, and enhance the user experience. Summary of the Invention

[0006] (1) Technical issues to be solved

[0007] The present invention aims to solve the technical problem of how to determine a matching tutoring strategy for a question through computer processing; further, the technical problem of how to more accurately locate the question and the corresponding tutoring strategy based on the retrieval processing of the question image and the degree of matching through a computer, so as to provide the corresponding user with a correct and more matching learning tutoring strategy based on the degree of matching after searching the question image to obtain more appropriate tutoring. Furthermore, the efficiency of learning tutoring is improved without the user having to repeatedly determine whether the tutoring meets their needs, effectively improving the accuracy of question searching and the accuracy of obtaining tutoring strategies, and enhancing the user experience.

[0008] (2) Technical solution

[0009] To solve the above technical problems, the first aspect of the present invention proposes a method for providing a tutoring strategy based on a question, comprising: obtaining search results corresponding to the question and the matching degree between the question and each search result; and providing a corresponding tutoring strategy based on the matching degree.

[0010] According to a preferred embodiment of the present invention, obtaining the degree of matching between the title and each search result includes: obtaining the recognition text of the title, the recognition text being an image of the title obtained through text recognition; and performing a matching test between the recognition text and the text title of each search result to obtain the matching degree of each search result.

[0011] According to a preferred embodiment of the present invention, obtaining the matching degree between the title and each search result includes: calculating the matching degree between the title and each search result through one or more of three methods: a similarity-based prediction model, an end-to-end neural network model, and image matching detection; wherein, the image matching detection refers to the matching degree detection between the image picture of the title and the resource image of each search result.

[0012] According to a preferred embodiment of the present invention, the tutoring strategy includes one or more of the following: text answers, video answers, and online answers.

[0013] According to a preferred embodiment of the present invention, a corresponding tutoring strategy is provided based on the matching degree, including: displaying the corresponding text answer content of each search result of the question in order from high to low according to the matching degree score; and also providing video answers and / or online answers for the search result with the highest matching degree score.

[0014] Optionally, the search result with the highest matching score also provides video answers and / or online answers, including: directly displaying a video of the process of answering the question, or first displaying a video answer button and then displaying the video of the process of answering the question after the user selects it; and / or directly jumping to an online answer page to provide one-on-one tutoring, or displaying a link to an online answer page that can provide one-on-one tutoring and then jumping after the user confirms it; wherein, one-on-one tutoring at least includes: providing a tutor for the user who enters the question.

[0015] Optionally, the coaching strategy further includes coaching guidance words corresponding to the matching degree score.

[0016] Optionally, the tutoring strategy is provided based on the title of the search result with the highest matching degree.

[0017] According to a preferred embodiment of the present invention, obtaining a search result corresponding to a title includes: obtaining an image picture containing the title; determining the relative position of the title in the image picture, and removing areas outside the area corresponding to the title in the image picture based on the relative position to clear interference information; performing text conversion on the image picture after clearing the interference information to obtain recognition text corresponding to the title in the title image; and performing a search based on the recognition text in combination with the image picture of the title to obtain the search result.

[0018] According to a preferred embodiment of the present invention, retrieval is performed based on the recognized text in combination with the image picture of the title, including: counting the number of text contents of the recognized text, and determining whether the number is greater than or equal to a preset threshold; if so, analyzing the recognized text to obtain keywords, and performing text retrieval based on the keywords to obtain each resource text title information having the same keywords as the retrieval result; if not, analyzing the image picture of the title corresponding to the recognized text to obtain a feature picture of the title, performing image retrieval based on the feature picture to obtain resource images having the same feature picture, and extracting the text title information of each resource image as the retrieval result.

[0019] According to a preferred embodiment of the present invention, the calculation of the matching degree between the title and each retrieval result includes at least one of the following: performing feature extraction on the recognition text and the retrieval result respectively and inputting them into a trained similarity-based prediction model to predict the matching score; inputting the recognition text and the retrieval result into a trained end-to-end neural network model to predict the matching score; calculating the matching score based on the feature image and the resource image of the retrieval result through a template matching algorithm or a feature matching algorithm.

[0020] In order to solve the above technical problems, the second aspect of the present invention proposes a question-based tutoring strategy providing system, including: a retrieval matching acquisition module, used to obtain retrieval results corresponding to the question and the matching degree between the question and each retrieval result; a tutoring strategy providing module, used to provide corresponding tutoring strategies based on the matching degree.

[0021] In order to solve the above technical problems, the third aspect of the present invention proposes an electronic device, including a processor and a memory, wherein the memory is used to store a computer executable program. When the computer program is executed by the processor, the processor executes the question-based tutoring strategy providing method as described in any one of the first aspects.

[0022] To solve the above technical problems, the fourth aspect of the present invention proposes a computer-readable medium storing a computer-executable program. When the computer-executable program is executed, it implements the method for providing a question-based tutoring strategy as described in any one of the first aspects.

[0023] In order to solve the above technical problems, the fifth aspect of the present invention proposes a computer executable program, which, when executed, implements the method for providing a question-based tutoring strategy as described in any one of the first aspects.

[0024] (3) Beneficial effects

[0025] The present invention detects and calculates the matching degree of the question and the search results based on the question to accurately determine the result with the highest matching degree, and provides the user with a corresponding question tutoring strategy based on the matching degree of each search result. This effectively improves the accuracy and correctness of question searches, and in particular, provides users with the most correct and accurate question tutoring methods and strategies that are most needed by users, which greatly helps to enhance the user experience.

[0026] Furthermore, the question is searched to obtain search results, and a matching degree is calculated between the recognized text of the question and each of the search results. A tutoring strategy is then provided based on the question in the search result with the highest matching degree. By determining the corresponding tutoring strategy through matching degree detection, the present invention can more accurately determine the actual question target. Furthermore, through a multi-faceted and multi-level tutoring approach, users are guided to match the corresponding tutoring strategy and find the most appropriate tutor, effectively improving the user experience.

[0027] Furthermore, for photo-search-type question images / videos, eliminating interference and coordinating the retrieval of image information, such as pictures of questions containing more graphics, with the retrieval of keywords after text recognition and error correction can help improve retrieval efficiency and further improve retrieval accuracy.

[0028] Furthermore, the results of the actual storage (including online and / or offline resource libraries and databases) retrieved or indexed can be adapted to different retrieval situations or retrieval results in a variety of ways, and the degree of matching with the input question can be calculated (matching degree: probability of matching / similarity), and the retrieval results are output according to the degree of matching. Finally, the text title with the highest matching degree is selected, the content of the text title is output, and the teaching and tutoring strategies corresponding to the text title are provided simultaneously, such as video answering process, online one-on-one tutoring, etc. This helps to further improve the accuracy of the provided inventory questions and their tutoring strategies on the basis of improving the detection efficiency of the matching degree, avoid search errors and the resulting erroneous tutoring strategies, avoid wasting resources and the time of tutors / users, and enhance the user's good usage experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is a main flow chart of an embodiment of a method for providing a topic-based tutoring strategy according to the present invention;

[0030] Figure 2 is a main structural block diagram of an embodiment of a system for providing a topic-based tutoring strategy according to the present invention;

[0031] Figure 3 is a main structural block diagram of an embodiment of an electronic device according to the present invention;

[0032] Figure 4 is a schematic diagram of the main structure of an embodiment of a more specific electronic device according to the present invention;

[0033] Figure 5 is a schematic diagram of the main structure of an embodiment of a computer-readable medium according to the present invention;

[0034] Figure 6 、 Figure 7 and Figure 8 This is an example of the process of changing the program interface displayed by an application program APP when a specific application is implemented in an application scenario of providing a tutoring strategy according to the technical solution of the present invention;

[0035] Figure 9 This is a main flow chart of an embodiment of the method for obtaining search results and matching degrees according to the present invention. DETAILED DESCRIPTION

[0036] In the introduction of specific embodiments, the detailed description of the structure, performance, effect or other features is intended to enable those skilled in the art to fully understand the embodiments. However, this does not preclude those skilled in the art from implementing the present invention with a technical solution that does not include the aforementioned structure, performance, effect or other features under specific circumstances.

[0037] The flowcharts in the accompanying drawings are merely illustrative of the process flow and do not necessarily include all of the content, operations, and steps in the flowcharts, nor do they necessarily imply that all of the steps in the flowcharts must be executed in the order shown. For example, some of the steps in the flowcharts may be separated, some may be combined or partially combined, and so on. The execution order shown in the flowcharts may be changed according to actual circumstances without departing from the spirit of the present invention.

[0038] Frames in the accompanying drawings Figure 1The term "functional entity" generally refers to a functional entity and does not necessarily correspond to a physically independent entity. That is, these functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processing unit devices and / or microcontroller devices.

[0039] The same reference numerals in the accompanying drawings represent the same or similar elements, components or parts, and thus repeated descriptions of the same or similar elements, components or parts may be omitted below. It should also be understood that although the first, second, third and other numbered adjectives may be used herein to describe various devices, elements, components or parts, these devices, elements, components or parts should not be limited by these adjectives. In other words, these adjectives are only used to distinguish one from another. For example, the first device may also be called the second device, but this does not deviate from the essential technical solution of the present invention. In addition, the terms "and / or" and "and / or" refer to all combinations including any one or more of the listed items. In order to solve the above technical problems, the present invention proposes a method for detecting the matching degree of a question, one embodiment of which mainly includes: obtaining the question recognition text from the question that needs to be tutored or answered input by the user, such as the image of the question taken, and after correcting the error, performing a search in combination with the image of the question, performing a matching degree detection / calculation between the obtained retrieval results and the question, and providing various tutoring strategies (one-on-one tutoring / teaching, video answer process, text answer content and answer results, etc.) corresponding to the stock text question with the highest matching degree (i.e., the highest matching degree score) for the question.

[0040] The following provides some descriptions of terms that may be involved in the description of various embodiments of the present invention:

[0041] Tutoring strategy: for the questions provided / input by the user, especially the images of the questions, find the corresponding questions and the tutoring / teaching strategies for solving the questions from the questions actually stored in the resource library.

[0042] One embodiment of the present invention provides one-on-one tutoring / teaching for users. This one-on-one tutoring / teaching strategy includes, but is not limited to, providing tutoring / teaching for a specific user or question. Tutors include, but are not limited to, local or remote computers / AI / robots, real-life experts / teachers, and the like. In one embodiment of the present invention, the tutoring strategies provided include, but are not limited to, one or more of the following: providing textual answers, video answers, online answers, and other tutoring / teaching methods for specific questions. The tutoring strategies may also include guidance for tutoring.

[0043] [Example 1]

[0044] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the method implementation of the present invention is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0045] Here we will combine Figure 1 The main steps of an embodiment of the method of the present invention are described in the flowchart shown in FIG. The method for providing a topic-based tutoring strategy includes:

[0046] Step S110: Obtain search results corresponding to the title and the matching degree between the title and each search result.

[0047] Step S120: providing a corresponding coaching strategy based on the matching degree.

[0048] For step S110, this embodiment does not limit the implementation method of obtaining the search results of the title and the matching degree between each search result and the title. For example, in some embodiments, the image of the title can be converted into text to obtain the recognized text, and the search is performed to obtain the search results and then obtain the matching degree between the title and each search result. A specific example of obtaining, such as Figure 9 The main flow chart is shown.

[0049] Step S1101: Obtain an image containing the title. The input title can be, for example, a title image or picture obtained from a camera or other device in a photo search app.

[0050] Step S1102: Obtaining recognition text from the image. In this step, the recognition text corresponding to the title image or title picture is obtained through image recognition technology.

[0051] In one embodiment, the relative position of the title in the image can be determined first, and based on the relative position, the area outside the area corresponding to the title can be removed from the image to remove interference information; then, the image after removing the interference information is subjected to text recognition conversion to obtain the corresponding recognized text.

[0052] In one embodiment, for a question input by a user that needs to be identified (question to be searched), in particular, by identifying a question image to obtain a question image corresponding to the question content, a recognition text can be obtained therefrom, and the recognition text is the text information of the question.

[0053] Wherein, obtaining text information from the input image of the question can specifically include: obtaining the question image input by the user; performing OCR conversion on the question image to obtain the text information of the question image. The conversion process can specifically include, for example: locating each text line in the question image based on a convolutional neural network and identifying the content of each text line; concatenating the identified text content of each line in accordance with the text line typesetting order to obtain OCR-converted text information; wherein the content of the text information includes at least one or more of the following: text, characters, background information, and other feature information of the corresponding text or characters. Wherein, before obtaining text information from the input question image, the rough position of the question text in the question image can also be determined in advance to remove interference information in the question image that is not the question. For example, removing interference from the handwritten answer part.

[0054] Furthermore, the topic image may include a video or image / picture / graphic type, and in this embodiment, may be a single image / frame. "Obtaining" refers to, for example, a topic image provided or input by the user for detection / identification or for which a tutoring strategy is to be provided. Specifically, the method and source for obtaining the topic image include, but are not limited to, various known methods and sources for obtaining the topic image from servers or terminals, as well as various known methods and sources for obtaining the topic image from remote networks and / or local sources. These can all serve as the source of the user input image. For example, the topic image may be captured in real time (including photographing and recording), stored (such as extracted from a gallery), transmitted online (e.g., downloaded) from a remote content server, uploaded offline from a local terminal or remote terminal, or captured by taking a screenshot of a linked page from various terminals (mobile phones, mobile computers, desktop computers, pads, etc.). The user inputs the topic image for which they need to obtain a tutoring strategy through a client application (APP) installed on these terminals (mobile phones, tablets, mobile computers, desktop computers, other handheld devices, etc.), a website page they visit, etc. For example, when the APP is opened, they request tutoring, take a photo of the paper topic, or take a screenshot of the webpage topic according to the APP's instructions or guidance.

[0055] The title image can be a picture with blurred graphics but clear text.

[0056] Furthermore, text / character recognition can be performed on the title image, for example, each part of the title image is processed to convert the image into computer text. The title image is subjected to character recognition conversion to obtain the text information of the title image, wherein the conversion method can adopt a method such as optical character recognition (OCR) conversion to analyze and process the image file to obtain text and layout information. Specifically, examples of image conversion include positioning, that is, finding the area where the title text exists, and title text recognition, that is, performing character recognition on the title text part to obtain the title text information, including but not limited to end-to-end text recognition and scene text positioning algorithms.

[0057] An example of conversion is as follows: First, the position of each text line can be located based on a convolutional neural network (CNN). Then, the content of each line of text can be identified based on a recurrent neural network (CNN). The contents of each identified line of text are then concatenated according to the layout order of the text lines, thereby completing the process of converting the title image into text / characters, i.e., OCR conversion. The resulting text information is the text / character corresponding to the title image. Among them, convolutional neural networks and recurrent neural networks can adopt several existing neural network models for image / picture conversion, or models such as SVM. The specific application and conversion process will not be repeated here.

[0058] Among them, the content of the text information includes at least one or more of the following: text, characters, background information (including but not limited to background, pixels, symbols, color, size, font, thickness, graphics and other image-related information other than text or characters), and other feature information corresponding to the text or characters.

[0059] Furthermore, in the processing of the converted text information, the optimization method can include a step of first eliminating interference or cleaning the interference information, for example, pre-determining the rough location of the title text in the title image to remove interference information in the title image that is not the title (such as some marks made by the user, such as handwriting, etc.). An example of cleaning can be mainly performed by processing the content of a whole page image (or a frame image) to remove interference information. Specifically, in a whole page image, the general location of the title text in the image is first roughly / basically determined (i.e., the image area corresponding to the converted text information), and the information not in this general location is not specifically analyzed and identified. That is, through methods such as image scanning, character features (background, size, thickness, font, color, etc.), or even pre-set conditions (the condition that the upper middle part of the acquired image is the position range), the approximate area (i.e., the location) where these converted title text / characters are located in the image is pre-found for conversion and recognition, thereby further achieving improved processing efficiency and accuracy of the object to be identified.

[0060] Furthermore, the text information obtained may mainly include but is not limited to the text information obtained by identifying the text part of the title image that is identified as the title part, etc. The text information of the title includes text, characters, etc. The text information converted by OCR includes but is not limited to the text in the positioning area, and various information related to the text (pixels, characters, background, color, size, font, thickness and other image information).

[0061] Furthermore, the image corresponding to the text portion of the question can be saved for subsequent use.

[0062] An example application scenario involves locating, recognizing, and converting text into OCR from a user-entered image of a single question. This also allows for the extraction of the corresponding image portion of the text. The text information might include content related to the question, such as: Beef prices rose 10% in early August and fell 15% in early November compared to early October. Did beef prices rise or fall in early November compared to early August? What was the magnitude of the increase or decrease? For example,

[0063] Step S1103: Obtain search results corresponding to the title.

[0064] Specifically, the search can be performed based on the recognized text in combination with the image of the title to obtain the search result.

[0065] In some embodiments, the number of textual contents of the text information can be counted, and it can be determined whether the number is greater than or equal to a preset threshold; if so, the text information is analyzed to obtain keywords of the text information, and a text search is performed based on the keywords to obtain resource text title information with the same keywords as the search results; if not, the image of the title is analyzed to obtain a feature image of the title, an image search is performed based on the feature image to obtain resource images with the same feature image, and the text title information of each resource image is extracted as the search result. In some implementations, a search can be performed based on the text information combined with the image in the image of the title to obtain the search results for the corresponding title.

[0066] When the text information has little textual content, the title image corresponding to the text information is analyzed to obtain a feature image, and an image search is performed based on the feature image to obtain each resource image having the same feature image, and the text title information of each resource image is extracted as the search result. The feature image includes an image and / or corresponding features cut from the title image, such as background information such as texture, brightness, and color. Specifically, the feature image can be a graphic included in the title, such as a geometric figure or function relationship diagram in a math problem, or a map included in a geography problem.

[0067] In one embodiment, when the text information contains a large amount of text content, the text information is analyzed to obtain keywords, and a text search is performed based on the keywords to obtain each resource text title information having the same keywords as the search result. Specifically, for example, the text information is corrected; based on the corrected text information, the keywords are obtained through frequency statistics of the text, whitelist matching, and / or prediction using an entity naming model; and, based on the keywords, all resource text title information having the same keywords is searched and sorted (for example, by resource titles containing the largest number of one or more keywords); and a predetermined number of resource text title information with the highest ranking is selected.

[0068] In one embodiment, when the text information contains relatively little text, the image at the location of the image of the question corresponding to the text information is analyzed. A graphic corresponding to the question is often present near the question. Image recognition is then used to obtain the image corresponding to the question and its corresponding feature information as a feature image for the question. In one example, during image recognition to obtain text information, the image is segmented corresponding to the region where the text information is obtained, and information corresponding to the segmented image in the background information corresponding to the region when the text information is obtained is used as feature information. The segmented image with the feature information is then used as a feature image. Furthermore, image retrieval is performed based on the feature image to obtain each resource image with the same feature image, and the text title information corresponding to each resource image is extracted as the search result. Specifically, a fusion search method can be used: for questions with a large amount of text, a text search is used. For questions with a small amount of text but the corresponding image contains an image of the question, an image search method can be used for questions containing images of shapes such as circles and triangles (geometry problems, etc.). This can effectively increase search efficiency and improve the accuracy of search results.

[0069] The question information actually stored in the resource library / database mainly includes text questions, and of course also includes various graphic symbols of the questions (such as various geometry questions).

[0070] The above-mentioned "more" and "less" of text content are relative, and here it can refer to the text information or the number of words corresponding to the text information exceeding a predetermined word count threshold. Specifically, the number of text contents of the text information (corresponding to the text part of the question) can be counted to determine whether the number is greater than or equal to a preset threshold to determine whether the text content is more or less. If so, it indicates that the text content is more, and if not, it indicates that the text content is less. Based on this, the above-mentioned retrieval process is processed.

[0071] Furthermore, an implementation method of analyzing the text information to obtain keywords for retrieval may also include:

[0072] First, the text information is corrected. Since the characters obtained through OCR conversion may contain errors or deviations, they can be corrected first and then the keywords can be determined. Error correction can be performed based on contextual information, text character sequences, language grammatical rules, etc., using various corresponding text correction algorithms / models (for example, a type of RNN sequence model of a neural network model, or a rule-based traversal algorithm, etc.), to correct the text information. This will not be described in detail here.

[0073] Secondly, according to the text information after error correction, the keywords are selected from the results of the above steps according to certain rules through frequency statistics of text, whitelist matching and / or prediction through entity naming model. The above methods of selecting keywords are only examples, and can be specifically: for example, the frequency of words is determined through frequency statistics, where "words" are the results of the above steps, and can be determined by but not limited to the grammatical rules and word formation rules of the language. Find out which words are in the whitelist, such as "linear equation of two variables" is in the whitelist; model prediction, such as the neural network model NER (namely named entity recognition NER, to identify and extract entities with specific meanings in text information), etc. to select available keywords. These methods can be used alone or in combination of two or three, or used one by one in sequence according to the characteristics of the method.

[0074] Third, all resource text title information with the same keyword is searched based on the keyword and all the resource text title information is sorted, such as which one contains more keywords or contains a certain keyword more frequently, etc. The more keywords, the higher the ranking, and then a predetermined number of resource text title information with the highest ranking is selected. For example, a batch of topics with the same keyword are indexed based on the keyword, and then these topics are sorted using various ranking models or algorithms (such as logistic regression, SVM model, etc.). Finally, the top ranked results, such as the preset top 10, are selected as the resource text title information to be tested for matching with the topic in the topic image input by the user.

[0075] An embodiment of obtaining a feature image of a question for image retrieval includes extracting images / pictures with content near a corresponding position in the question image corresponding to the text information (i.e., images near the text portion of the question image; by defining "nearby," the extraction range can be limited to the same question), extracting these images (including triangles and circles in geometry questions, for example), i.e., feature images, and performing image search. Image search can use various image indexing algorithms, such as various functions (SIFT descriptors, fingerprint algorithms, bundling features algorithms, hash functions, local N-th order moments of images, etc.) to obtain feature images and then encode the feature information. Using the features and their encoding, stored images with the encoded information in a resource library are searched to obtain the corresponding text information. The searched stored text information results are sorted according to the corresponding similarity during the search, and the top 10 resource text title information is selected as the search result, and the matching degree is tested with the recognized text of the question (e.g., the question in the question image input by the user).

[0076] An application scenario, for example: based on a keyword text search of the text information A of the question image (in a situation with a large number of words) or an image search based on the feature picture of the corresponding part of A (a situation with fewer words but with graphics, such as a geometry problem), a batch of text question information A1, A2, ...An stored in a resource library (database) with the same keyword or the same image feature encoding is obtained, that is, n search results of A, and sorted or sorted based on the similarity during the search to find the top 10 A1, A2, ...A10, and these 10 are used as the final search results to enter the subsequent matching degree detection.

[0077] Step S1104: Obtain the matching degree between the title and each search result.

[0078] For example, after the image picture of the aforementioned title is subjected to text recognition to obtain the recognized text, the recognition text can be tested for matching with the text titles of the search results obtained by searching through the recognition text, thereby obtaining the matching degree of each search result.

[0079] Among them, the degree of matching (i.e., the degree of matching or similarity / similarity) between the title and the retrieval result can be detected / calculated to determine the matching probability or the degree of matching. For example, the degree of matching can be represented by a matching probability score. Specifically, the degree of matching between the title and each retrieval result can be calculated by one or more of the three methods: a similarity-based prediction model, an end-to-end neural network model, and image matching detection; wherein, the image matching detection refers to the detection of the degree of matching between the image of the title and the resource image of each retrieval result. Further, for example, the recognition text and the retrieval result can be respectively subjected to feature extraction and input into a trained similarity-based prediction model to predict the matching score; the recognition text and the retrieval result can be input into a trained end-to-end neural network model to predict the matching score; the matching score can be calculated based on the feature image and the resource image of the retrieval result through a template matching algorithm or a feature matching algorithm, and so on.

[0080] In one embodiment, obtaining the degree of match between the title and the search result includes, but is not limited to, calculating the degree of match between the title and the search result by at least one or more of three methods: a similarity-based prediction model (such as a predicted probability), an end-to-end neural network model, and image matching detection between a feature image in the title image and a resource image of the search result. The three methods are specifically as follows: predicting the probability based on the features of the text information and the search result using a trained similarity-based prediction model, and using the probability as the match score; inputting the text information and the search result into a trained end-to-end neural network model in character sequence to predict the probability, and using the probability as the match score; calculating the match score based on the feature image and the resource image of the search result using a template matching algorithm or a feature matching algorithm. In addition, a numerical threshold condition or numerical range can also be preset, and when the predicted similarity / match value meets the threshold condition or falls within a certain numerical range, a predetermined score is given as the match score.

[0081] In one embodiment, a match is predicted using a trained similarity-based prediction model based on the features of the text information and the results. Specifically, this may include: calculating key similarities between two text segments using simhash, sentence mutual information, or the longest common substring length; training a logistic regression model or a tree model using the similarities and features of the two text segments (e.g., feature vectors and distances obtained through word segmentation) to obtain a trained similarity-based prediction model; and inputting the features of the text information and the results into the similarity-based prediction model to predict the probability of a match. A high probability may indicate a high match score and a high degree of match.

[0082] In one embodiment, the degree of match is predicted based on the input of the text information and the result into a trained end-to-end neural network model. Specifically, this may include: inputting two text segments into an end-to-end neural network model as character sequences to train the neural network model, thereby obtaining a trained end-to-end neural network model; and inputting the text information and the result into the neural network model as character sequences to predict the probability of a match. A high probability may indicate a high match score, ie, a high degree of match.

[0083] In one embodiment, a match degree is detected based on the feature image and the resource image corresponding to the result. This may specifically include calculating the match degree based on the feature image and the resource image using a template matching algorithm or a feature matching algorithm. The match degree can be designed as a probability or represented by a simple numerical value, such as 0 for no match, 1 for a perfect match, and -1 for complete irrelevance. Similarly, a high match degree corresponds to a high match score. In this embodiment, matching detection can be divided into three categories: 1. By performing some classic similarity calculations on the two texts (such as simhash, sentence mutual information, longest common substring length, etc.), some key similarities are obtained, and then these similarities and the question text's own information (length, subject, etc.) are used as features to perform traditional machine learning model training, such as logistic regression model, tree model, etc., to determine the model parameters. After training, these models or algorithms are used to predict / calculate the matching degree between the text information of the question image to be tested for matching and the retrieved text question (for example, high probability indicates high matching, etc.); 2. Directly use a neural network model (such as seq2seq, various prediction models based on CNN, RNN, etc.), take the two texts directly as input, perform end-to-end model training, determine the model parameters, and then predict / calculate the matching degree; 3. For some graphic questions, the matching degree of the question will also be detected through image information, such as template matching, feature matching, etc., to determine the matching degree.

[0084] The neural network model / algorithm, prediction model / algorithm, similarity calculation method, matching probability prediction or matching degree calculation used in the above three implementations may also use several commonly used major machine learning models / algorithms, which will not be described in detail here.

[0085] It can be seen that the present invention will perform error correction before using text information retrieval to avoid errors after OCR text conversion. The search results with errors and defects in the title itself can be screened out / avoided in advance and corrected in advance, thereby improving the accuracy and efficiency of the retrieval; and the use of text information retrieval and image retrieval are combined, and they cooperate with each other under certain conditions to adapt to the retrieval and recognition of more different types of title images, especially for titles with more graphics and less text, the retrieval efficiency and accuracy are further improved. Furthermore, the matching degree of the retrieval results obtained by the search is calculated to determine whether these retrieval results are similar to or the degree of matching with the title that the user needs to answer. The matching degree calculation is combined with multiple methods to determine the degree of matching between these retrieval results and the title. The higher the value, the closer it is to the title that the user wants to answer, corresponding to the user's real needs.

[0086] In step S120, in which a corresponding tutoring strategy is provided based on the degree of match, the provided tutoring strategy may be correlated with the degree of match of each search result. Tutoring strategies include learning guidance based on the search results, including but not limited to strategic designs that guide users to review, learn, and master results and processes related to the search question, including related knowledge points and similar questions. Tutoring strategies may include one or more of the following: text answers, video answers, and online answers.

[0087] In some embodiments, this step displays and provides a tutoring strategy for the question with the highest matching degree based on the question with the highest matching degree. That is, at least the tutoring strategy for the question with the highest matching degree is displayed or displayed preferentially. Other questions with relatively low matching degrees and their tutoring strategies may not be displayed or displayed as the main content. This can help the user focus on the question with the highest matching degree and its corresponding tutoring strategy. Optionally, the tutoring strategy also includes tutoring guidance language corresponding to the matching degree score.

[0088] Specifically, for example, the corresponding text answers of the search results of the question can be displayed in order from high to low according to the matching score; video answers and / or online answers are also provided for the search result with the highest matching score; optionally, video answers and / or online answers are also provided for the search result with the highest matching score, including at least one of the following: directly displaying a video of the process of answering the question, or first displaying a video answer button and then displaying the video of the process of answering the question after the user selects it; directly jumping to the online answer page to provide one-on-one tutoring, or displaying a link to the online answer page that can provide one-on-one tutoring and jumping after the user confirms it; wherein, one-on-one tutoring includes: assigning a tutor to the user who enters the question for tutoring; optionally, providing the tutoring strategy based on the question of the search result with the highest matching score; optionally, the tutoring strategy also includes tutoring guidance terms corresponding to the matching score.

[0089] In one embodiment, the tutoring strategies provided to the user include, but are not limited to, one or more of the following: text answers (also known as text questions or text / text answer content), video answers, online answers, and the like.

[0090] In one embodiment, the search results are displayed in order of the matching scores from high to low, and the corresponding text answers are displayed in sequence; the search results with the highest matching scores are provided with video answers and / or online answers simultaneously. For example: the search results with the highest matching scores, such as the probability of the matching degree between the evaluation question and the search results, are output. Following the above example, the top 10 results are output, and the one with the highest matching score is placed at the front, and the default display is displayed directly. Figure 6 Result 1 is shown, and the text title content of the result is found and displayed, and the corresponding tutoring strategy is provided. For example, when the text title content is displayed, it is accompanied by the answer result, especially it can guide the user to provide the corresponding other tutoring teaching strategies, such as synchronously displaying video answers, or even online answers (one-on-one tutoring, etc.). An example of synchronously providing video answers and / or online answers to the search result with the highest matching score: directly displaying the video of the process of answering the question, or first displaying the video answer button and then displaying the video of the process of answering the question after the user selects it; and / or, directly jumping to the online answer page to provide one-on-one tutoring, or displaying a link to the online answer page that can provide one-on-one tutoring and then jumping to a more detailed answer after the user confirms it; wherein, the one-on-one tutoring at least includes: providing a tutor for the user who inputs the question, and / or, providing a tutor for the input question.

[0091] In addition, you can set up a function to provide tutoring strategies for similar questions, similar questions, and related questions based on the degree of match for other results with lower scores, such as the last 9 results of the top 10. Text answers are provided for these results by default. Moreover, after selection, in addition to providing text answers, you can also obtain corresponding video answers, online answers, etc. by selecting functions such as explanations of similar questions.

[0092] An application scenario is as follows: ten search results A1, A2, ... A10 are matched with the text information of the input question image (model prediction or template matching, etc.) to obtain a matching score. Among them, A3 has the highest score, so the text title corresponding to A3 and its corresponding tutoring strategy are found. For example: the search text information of the user input image is "Xiao Ming likes to eat watermelon", and the obtained alternative title 1 is "Xiao Ming likes to eat big watermelon", and the alternative title 2 is "Xiao Ming likes to eat winter melon". Through the previous calculation, it can be determined that the matching degree between alternative title 1 and the user search is higher. According to the matching situation, alternative title 1 and its tutoring strategy are provided to the user first. Alternatively, the top-ranked search results, such as alternative title 1 and the corresponding tutoring strategy, alternative title 2 and the corresponding tutoring strategy, are provided to the user together, and the alternative title 1 and the corresponding tutoring strategy with a high matching degree are displayed first. For example, a stacked display can be used, with the highly matched alternative question 1 and the corresponding counseling strategy placed in the first layer / first level that the user can directly view; the less matched alternative question 2 and the corresponding counseling strategy are placed in a lower layer or most of their content is hidden. Only when the user selects the top name of alternative question 2, such as result 2, will the entire content, that is, all alternative questions 2 and the corresponding counseling strategies, be expanded to display.

[0093] Tutoring strategies may include, but are not limited to, one or more of the following: text-based solutions, video-based solutions, and online solutions. For example, content related to Alternative Question 1, video, image, or text-based solutions, and online or video-based tutoring / teaching methods may be provided. Text-based solutions may or may not include the answer; video solutions can be opened by the user by clicking on them, or a video pop-up can be displayed to demonstrate the solution process; online solutions may directly redirect the user to a tutor's page or provide a link for the user to select and then redirect to the tutor's page, thereby obtaining individual tutoring for the user or for that question or question type, etc.

[0094] It can be seen that the present invention combines the text of the question (the text converted by OCR) with the image search process of the question and performs a matching test with the actual text correct and accurate text question that is searched, and determines the correct question that matches the question, that is, the resource question with the highest matching degree, which helps to improve the accuracy and correctness of the image recognition processing, thereby helping to accurately locate the correct tutor and provide the correct tutoring strategy. The present invention uses computer processing matching degree detection to ultimately determine which question should be given the corresponding tutoring strategy based on the level of matching. It can more accurately determine the actual question object and provide the correct tutoring strategy. The algorithm has low complexity, is simple and convenient, easy to implement, and has high efficiency. Furthermore, according to the multi-faceted and multi-level tutoring method, the user is guided to match the corresponding tutoring strategy and find the most appropriate corresponding tutor, avoiding the waste of various resources and time, and effectively improving the user experience.

[0095] It can be seen that the technical solution of the present invention does not simply provide the image to the requesting party arbitrarily, but performs OCR text conversion and then performs a matching test with the corresponding actual text title searched based on the image. The search results with errors and defects in the title itself can be screened out in time and then corrected.

[0096] It can be seen that the technical solution of the present invention determines the correct question that the OCR text of the question matches by detecting the matching degree between the OCR text of the question and the correct and accurate text question of the actual text searched, which helps to improve the accuracy of image recognition processing and further helps to accurately locate the correct tutoring strategy in the future.

[0097] [Example 2]

[0098] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the system implementation of the present invention is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0099] Here we will combine Figure 2 The main structural block diagram of an embodiment of the method of the present invention is shown as follows. In this embodiment, the system at least includes a search and match acquisition module 110 and a coaching strategy providing module 120.

[0100] The search matching acquisition module 110 is used to obtain the search results corresponding to the topic and the matching degree between the topic and each search result.

[0101] The coaching strategy providing module 120 is configured to provide a corresponding coaching strategy based on the matching degree.

[0102] In this embodiment, the search matching acquisition module 110 does not limit the implementation method for obtaining the search results of the question and the matching degree between each search result and the question. For example, in some embodiments, the recognized text can be obtained by converting the image of the question into text, and then searching to obtain the search results and then determining the matching degree between the question and each search result.

[0103] In one embodiment, the search and matching acquisition module 110 specifically further includes an input module 1101 , a conversion module 1102 , a search module 1103 , and a matching module 1104 .

[0104] The input module 1101 is configured to obtain an image or picture containing the title. The input title may be, for example, a title image or picture obtained from a camera or other device in a photo search app.

[0105] The conversion module 1102 is configured to obtain the recognition text from the image picture. Specifically, the recognition text corresponding to the title image or title picture is obtained through image recognition technology.

[0106] In one embodiment, the relative position of the title in the image can be determined first, and based on the relative position, the area outside the area corresponding to the title can be removed from the image to remove interference information; then, the image after removing the interference information is subjected to text recognition conversion to obtain the corresponding recognized text.

[0107] In one embodiment, for a question input by a user that needs to be identified (question to be searched), in particular, by identifying a question image to obtain a question image corresponding to the question content, a recognition text can be obtained therefrom, and the recognition text is the text information of the question.

[0108] Wherein, obtaining text information from the input image of the question can specifically include: obtaining the question image input by the user; performing OCR conversion on the question image to obtain the text information of the question image. The conversion process can specifically include, for example: locating each text line in the question image based on a convolutional neural network and identifying the content of each text line; concatenating the identified text content of each line in accordance with the text line typesetting order to obtain OCR-converted text information; wherein the content of the text information includes at least one or more of the following: text, characters, background information, and other feature information of the corresponding text or characters. Wherein, before obtaining text information from the input question image, the rough position of the question text in the question image can also be determined in advance to remove interference information in the question image that is not the question. For example, removing interference from the handwritten answer part.

[0109] Furthermore, the topic image may include a video or image / picture / graphic type, and in this embodiment, may be a single image / frame. "Obtaining" refers to, for example, a topic image provided or input by the user for detection / identification or for which a tutoring strategy is to be provided. Specifically, the method and source for obtaining the topic image include, but are not limited to, various known methods and sources for obtaining the topic image from servers or terminals, as well as various known methods and sources for obtaining the topic image from remote networks and / or local sources. These can all serve as the source of the user input image. For example, the topic image may be captured in real time (including photographing and recording), stored (such as extracted from a gallery), transmitted online (e.g., downloaded) from a remote content server, uploaded offline from a local terminal or remote terminal, or captured by taking a screenshot of a linked page from various terminals (mobile phones, mobile computers, desktop computers, pads, etc.). The user inputs the topic image for which they need to obtain a tutoring strategy through a client application (APP) installed on these terminals (mobile phones, tablets, mobile computers, desktop computers, other handheld devices, etc.), a website page they visit, etc. For example, when the APP is opened, they request tutoring, take a photo of the paper topic, or take a screenshot of the webpage topic according to the APP's instructions or guidance.

[0110] The title image can be a picture with blurred graphics but clear text.

[0111] Furthermore, text / character recognition can be performed on the title image, for example, each part of the title image is processed to convert the image into computer text. The title image is subjected to character recognition conversion to obtain the text information of the title image, wherein the conversion method can adopt a method such as optical character recognition (OCR) conversion to analyze and process the image file to obtain text and layout information. Specifically, examples of image conversion include positioning, that is, finding the area where the title text exists, and title text recognition, that is, performing character recognition on the title text part to obtain the title text information, including but not limited to end-to-end text recognition and scene text positioning algorithms.

[0112] An example of conversion is as follows: First, the position of each text line can be located based on a convolutional neural network (CNN). Then, the content of each line of text can be identified based on a recurrent neural network (CNN). The contents of each identified line of text are then concatenated according to the layout order of the text lines, thereby completing the process of converting the title image into text / characters, i.e., OCR conversion. The resulting text information is the text / character corresponding to the title image. Among them, convolutional neural networks and recurrent neural networks can adopt several existing neural network models for image / picture conversion, or models such as SVM. The specific application and conversion process will not be repeated here.

[0113] Among them, the content of the text information includes at least one or more of the following: text, characters, background information (including but not limited to background, pixels, symbols, color, size, font, thickness, graphics and other image-related information other than text or characters), and other feature information corresponding to the text or characters.

[0114] Furthermore, in the processing of the converted text information, the optimization method can include first eliminating interference or cleaning the interference information, for example, pre-determining the rough position of the title text in the title image to remove interference information in the title image that is not the title (such as some marks made by the user, such as handwriting, etc.). An example of cleaning can be mainly performed by processing the content of a whole page image (or a frame image) to remove interference information. Specifically, in a whole page image, first roughly / basically determine the general position of the title text in the image (that is, the image area corresponding to the converted text information), and do not specifically analyze and identify the information that is not in this general position. That is, through methods such as image scanning, character features (background, size, thickness, font, color, etc.), or even pre-set conditions (the condition that the upper and middle part of the acquired image is the position range), the approximate area (that is, the location) where these converted title text / characters are located in the image is found in advance for conversion and recognition, thereby further achieving improved processing efficiency and the accuracy of the object to be identified.

[0115] Furthermore, the text information obtained may mainly include but is not limited to the text information obtained by identifying the text part of the title image that is identified as the title part, etc. The text information of the title includes text, characters, etc. The text information converted by OCR includes but is not limited to the text in the positioning area, and various information related to the text (pixels, characters, background, color, size, font, thickness and other image information).

[0116] Furthermore, the image corresponding to the text portion of the question can be saved for subsequent use.

[0117] An example application scenario involves locating, recognizing, and converting text into OCR from a user-entered image of a single question. This also allows for the extraction of the corresponding image portion of the text. The text information might include content related to the question, such as: Beef prices rose 10% in early August and fell 15% in early November compared to early October. Did beef prices rise or fall in early November compared to early August? What was the magnitude of the increase or decrease? For example,

[0118] The retrieval module 1103 is configured to perform retrieval based on the recognized text in combination with the image of the title to obtain the retrieval result.

[0119] Specifically, the search can be performed based on the recognized text in combination with the image of the title to obtain the search result.

[0120] In some embodiments, the number of textual contents of the text information can be counted, and it can be determined whether the number is greater than or equal to a preset threshold; if so, the text information is analyzed to obtain keywords of the text information, and a text search is performed based on the keywords to obtain resource text title information with the same keywords as the search results; if not, the image of the title is analyzed to obtain a feature image of the title, an image search is performed based on the feature image to obtain resource images with the same feature image, and the text title information of each resource image is extracted as the search result. In some implementations, a search can be performed based on the text information combined with the image in the image of the title to obtain the search results for the corresponding title.

[0121] When the text information has little textual content, the title image corresponding to the text information is analyzed to obtain a feature image, and an image search is performed based on the feature image to obtain each resource image having the same feature image, and the text title information of each resource image is extracted as the search result. The feature image includes an image and / or corresponding features cut from the title image, such as background information such as texture, brightness, and color. Specifically, the feature image can be a graphic included in the title, such as a geometric figure or function relationship diagram in a math problem, or a map included in a geography problem.

[0122] In one embodiment, when the text information contains a large amount of text content, the text information is analyzed to obtain keywords, and a text search is performed based on the keywords to obtain each resource text title information having the same keywords as the search result. Specifically, for example, the text information is corrected; based on the corrected text information, the keywords are obtained through frequency statistics of the text, whitelist matching, and / or prediction using an entity naming model; and, based on the keywords, all resource text title information having the same keywords is searched and sorted (for example, by resource titles containing the largest number of one or more keywords); and a predetermined number of resource text title information with the highest ranking is selected.

[0123] In one embodiment, when the text information contains relatively little text, the image at the location of the image of the question corresponding to the text information is analyzed. A graphic corresponding to the question is often present near the question. Image recognition is then used to obtain the image corresponding to the question and its corresponding feature information as a feature image for the question. In one example, during image recognition to obtain text information, the image is segmented corresponding to the region where the text information is obtained, and information corresponding to the segmented image in the background information corresponding to the region when the text information is obtained is used as feature information. The segmented image with the feature information is then used as a feature image. Furthermore, image retrieval is performed based on the feature image to obtain each resource image with the same feature image, and the text title information corresponding to each resource image is extracted as the search result. Specifically, a fusion search method can be used: for questions with a large amount of text, a text search is used. For questions with a small amount of text but the corresponding image contains an image of the question, an image search method can be used for questions containing images of shapes such as circles and triangles (geometry problems, etc.). This can effectively increase search efficiency and improve the accuracy of search results.

[0124] The question information actually stored in the resource library / database mainly includes text questions, and of course also includes various graphic symbols of the questions (such as various geometry questions).

[0125] The above-mentioned "more" and "less" of text content are relative, and here it can refer to the text information or the number of words corresponding to the text information exceeding a predetermined word count threshold. Specifically, the number of text contents of the text information (corresponding to the text part of the question) can be counted to determine whether the number is greater than or equal to a preset threshold to determine whether the text content is more or less. If so, it indicates that the text content is more, and if not, it indicates that the text content is less. Based on this, the above-mentioned retrieval process is processed.

[0126] Furthermore, an implementation method of analyzing the text information to obtain keywords for retrieval may also include:

[0127] First, the text information is corrected. Since the characters obtained through OCR conversion may contain errors or deviations, they can be corrected first and then the keywords can be determined. Error correction can be performed based on contextual information, text character sequences, language grammatical rules, etc., using various corresponding text correction algorithms / models (for example, a type of RNN sequence model of a neural network model, or a rule-based traversal algorithm, etc.), to correct the text information. This will not be described in detail here.

[0128] Secondly, according to the text information after error correction, the keywords are selected from the results of the above steps according to certain rules through frequency statistics of text, whitelist matching and / or prediction through entity naming model. The above methods of selecting keywords are only examples, and can be specifically: for example, the frequency of words is determined through frequency statistics, where "words" are the results of the above steps, and can be determined by but not limited to the grammatical rules and word formation rules of the language. Find out which words are in the whitelist, such as "linear equation of two variables" is in the whitelist; model prediction, such as the neural network model NER (namely named entity recognition NER, to identify and extract entities with specific meanings in text information), etc. to select available keywords. These methods can be used alone or in combination of two or three, or used one by one in sequence according to the characteristics of the method.

[0129] Third, all resource text title information with the same keyword is searched based on the keyword and all the resource text title information is sorted, such as which one contains more keywords or contains a certain keyword more frequently, etc. The more keywords, the higher the ranking, and then a predetermined number of resource text title information with the highest ranking is selected. For example, a batch of topics with the same keyword are indexed based on the keyword, and then these topics are sorted using various ranking models or algorithms (such as logistic regression, SVM model, etc.). Finally, the top ranked results, such as the preset top 10, are selected as the resource text title information to be tested for matching with the topic in the topic image input by the user.

[0130] An embodiment of obtaining a feature image of a question for image retrieval includes extracting images / pictures with content near a corresponding position in the question image corresponding to the text information (i.e., images near the text portion of the question image; by defining "nearby," the extraction range can be limited to the same question), extracting these images (including triangles and circles in geometry questions, for example), i.e., feature images, and performing image search. Image search can use various image indexing algorithms, such as various functions (SIFT descriptors, fingerprint algorithms, bundling features algorithms, hash functions, local N-th order moments of images, etc.) to obtain feature images and then encode the feature information. Using the features and their encoding, stored images with the encoded information in a resource library are searched to obtain the corresponding text information. The searched stored text information results are sorted according to the corresponding similarity during the search, and the top 10 resource text title information is selected as the search result, and the matching degree is tested with the recognized text of the question (e.g., the question in the question image input by the user).

[0131] An application scenario, for example: based on a keyword text search of the text information A of the question image (in a situation with a large number of words) or an image search based on the feature picture of the corresponding part of A (a situation with fewer words but with graphics, such as a geometry problem), a batch of text question information A1, A2, ...An stored in a resource library (database) with the same keyword or the same image feature encoding is obtained, that is, n search results of A, and sorted or sorted based on the similarity during the search to find the top 10 A1, A2, ...A10, and these 10 are used as the final search results to enter the subsequent matching degree detection.

[0132] The matching module 1104 is used to obtain the matching degree between the title and each search result.

[0133] For example, after the image picture of the aforementioned title is subjected to text recognition to obtain the recognized text, the recognition text can be tested for matching with the text titles of the search results obtained by searching through the recognition text, thereby obtaining the matching degree of each search result.

[0134] Among them, the degree of matching (i.e., the degree of matching or similarity / similarity) between the title and the retrieval result can be detected / calculated to determine the matching probability or the degree of matching. For example, the degree of matching can be represented by a matching probability score. Specifically, the degree of matching between the title and each retrieval result can be calculated by one or more of the three methods: a similarity-based prediction model, an end-to-end neural network model, and image matching detection; wherein, the image matching detection refers to the detection of the degree of matching between the image of the title and the resource image of each retrieval result. Further, for example, the recognition text and the retrieval result can be respectively subjected to feature extraction and input into a trained similarity-based prediction model to predict the matching score; the recognition text and the retrieval result can be input into a trained end-to-end neural network model to predict the matching score; the matching score can be calculated based on the feature image and the resource image of the retrieval result through a template matching algorithm or a feature matching algorithm, and so on.

[0135] In one embodiment, obtaining the degree of match between the title and the search result includes, but is not limited to, calculating the degree of match between the title and the search result by at least one or more of three methods: a similarity-based prediction model (such as a predicted probability), an end-to-end neural network model, and image matching detection between a feature image in the title image and a resource image of the search result. The three methods are specifically as follows: predicting the probability based on the features of the text information and the search result using a trained similarity-based prediction model, and using the probability as the match score; inputting the text information and the search result into a trained end-to-end neural network model in character sequence to predict the probability, and using the probability as the match score; calculating the match score based on the feature image and the resource image of the search result using a template matching algorithm or a feature matching algorithm. In addition, a numerical threshold condition or numerical range can also be preset, and when the predicted similarity / match value meets the threshold condition or falls within a certain numerical range, a predetermined score is given as the match score.

[0136] In one embodiment, a match is predicted using a trained similarity-based prediction model based on the features of the text information and the results. Specifically, this may include: calculating key similarities between two text segments using simhash, sentence mutual information, or the longest common substring length; training a logistic regression model or a tree model using the similarities and features of the two text segments (e.g., feature vectors and distances obtained through word segmentation) to obtain a trained similarity-based prediction model; and inputting the features of the text information and the results into the similarity-based prediction model to predict the probability of a match. A high probability may indicate a high match score and a high degree of match.

[0137] In one embodiment, the degree of match is predicted based on the input of the text information and the result into a trained end-to-end neural network model. Specifically, this may include: inputting two text segments into an end-to-end neural network model as character sequences to train the neural network model, thereby obtaining a trained end-to-end neural network model; and inputting the text information and the result into the neural network model as character sequences to predict the probability of a match. A high probability may indicate a high match score, ie, a high degree of match.

[0138] In one embodiment, a match degree is detected based on the feature image and the resource image corresponding to the result. This may specifically include calculating the match degree based on the feature image and the resource image using a template matching algorithm or a feature matching algorithm. The match degree can be designed as a probability or represented by a simple numerical value, such as 0 for no match, 1 for a perfect match, and -1 for complete irrelevance. Similarly, a high match degree corresponds to a high match score. In this embodiment, matching detection can be divided into three categories: 1. By performing some classic similarity calculations on the two texts (such as simhash, sentence mutual information, longest common substring length, etc.), some key similarities are obtained, and then these similarities and the question text's own information (length, subject, etc.) are used as features to perform traditional machine learning model training, such as logistic regression model, tree model, etc., to determine the model parameters. After training, these models or algorithms are used to predict / calculate the matching degree between the text information of the question image to be tested for matching and the retrieved text question (for example, high probability indicates high matching, etc.); 2. Directly use a neural network model (such as seq2seq, various prediction models based on CNN, RNN, etc.), take the two texts directly as input, perform end-to-end model training, determine the model parameters, and then predict / calculate the matching degree; 3. For some graphic questions, the matching degree of the question will also be detected through image information, such as template matching, feature matching, etc., to determine the matching degree.

[0139] The neural network model / algorithm, prediction model / algorithm, similarity calculation method, matching probability prediction or matching degree calculation used in the above three implementations may also use several commonly used major machine learning models / algorithms, which will not be described in detail here.

[0140] It can be seen that the present invention will perform error correction before using text information retrieval to avoid errors after OCR text conversion. The search results with errors and defects in the title itself can be screened out / avoided in advance and corrected in advance, thereby improving the accuracy and efficiency of the retrieval; and the use of text information retrieval and image retrieval are combined, and they cooperate with each other under certain conditions to adapt to the retrieval and recognition of more different types of title images, especially for titles with more graphics and less text, the retrieval efficiency and accuracy are further improved. Furthermore, the matching degree of the retrieval results obtained by the search is calculated to determine whether these retrieval results are similar to or the degree of matching with the title that the user needs to answer. The matching degree calculation is combined with multiple methods to determine the degree of matching between these retrieval results and the title. The higher the value, the closer it is to the title that the user wants to answer, corresponding to the user's real needs.

[0141] The tutoring strategy providing module 120 specifically provides tutoring strategies that are, for example, related to the matching degree of each search result. Tutoring strategies include learning guidance based on the search results, including but not limited to strategic designs that guide users to review, learn, and master results, processes, and even related knowledge points and similar questions related to the search question. These tutoring strategies may include one or more of the following: text answers, video answers, and online answers.

[0142] In some embodiments, this module displays and provides the tutoring strategy for the question with the highest matching degree based on the question with the highest matching degree. That is, at least the tutoring strategy for the question with the highest matching degree is displayed or displayed preferentially. Other questions with relatively low matching degrees and their tutoring strategies may not be displayed or displayed as the main content. This can help the user focus on the question with the highest matching degree and its corresponding tutoring strategy. Optionally, the tutoring strategy also includes tutoring guidance language corresponding to the matching degree score. Specifically, for example, the corresponding text answers of the search results of the question can be displayed in order from high to low according to the matching score; video answers and / or online answers are also provided for the search result with the highest matching score; optionally, video answers and / or online answers are also provided for the search result with the highest matching score, including at least one of the following: directly displaying a video of the process of answering the question, or first displaying a video answer button and then displaying the video of the process of answering the question after the user selects it; directly jumping to the online answer page to provide one-on-one tutoring, or displaying a link to the online answer page that can provide one-on-one tutoring and jumping after the user confirms it; wherein, one-on-one tutoring includes: assigning a tutor to the user who enters the question for tutoring; optionally, providing the tutoring strategy based on the question of the search result with the highest matching score; optionally, the tutoring strategy also includes tutoring guidance terms corresponding to the matching score.

[0143] In one embodiment, the tutoring strategies provided to the user include, but are not limited to, one or more of the following: text answers (also known as text questions or text / text answer content), video answers, online answers, and the like.

[0144] In one embodiment, the search results are displayed in order of the matching scores from high to low, and the corresponding text answers are displayed in sequence; the search results with the highest matching scores are provided with video answers and / or online answers simultaneously. For example: the search results with the highest matching scores, such as the probability of the matching degree between the evaluation question and the search results, are output. Following the above example, the top 10 results are output, and the one with the highest matching score is placed at the front, and the default display is displayed directly. Figure 6Result 1 is shown, and the text title content of the result is found and displayed, and the corresponding tutoring strategy is provided. For example, when the text title content is displayed, it is accompanied by the answer result, especially it can guide the user to provide the corresponding other tutoring teaching strategies, such as synchronously displaying video answers, or even online answers (one-on-one tutoring, etc.). An example of synchronously providing video answers and / or online answers to the search result with the highest matching score: directly displaying the video of the process of answering the question, or first displaying the video answer button and then displaying the video of the process of answering the question after the user selects it; and / or, directly jumping to the online answer page to provide one-on-one tutoring, or displaying a link to the online answer page that can provide one-on-one tutoring and then jumping to a more detailed answer after the user confirms it; wherein, the one-on-one tutoring at least includes: providing a tutor for the user who inputs the question, and / or, providing a tutor for the input question.

[0145] In addition, you can set up a function to provide tutoring strategies for similar questions, similar questions, and related questions based on the degree of match for other results with lower scores, such as the last 9 results of the top 10. Text answers are provided for these results by default. Moreover, after selection, in addition to providing text answers, you can also obtain corresponding video answers, online answers, etc. by selecting functions such as explanations of similar questions.

[0146] An application scenario is as follows: ten search results A1, A2, ... A10 are matched with the text information of the input question image (model prediction or template matching, etc.) to obtain a matching score. Among them, A3 has the highest score, so the text title corresponding to A3 and its corresponding tutoring strategy are found. For example: the search text information of the user input image is "Xiao Ming likes to eat watermelon", and the obtained alternative title 1 is "Xiao Ming likes to eat big watermelon", and the alternative title 2 is "Xiao Ming likes to eat winter melon". Through the previous calculation, it can be determined that the matching degree between alternative title 1 and the user search is higher. According to the matching situation, alternative title 1 and its tutoring strategy are provided to the user first. Alternatively, the top-ranked search results, such as alternative title 1 and the corresponding tutoring strategy, alternative title 2 and the corresponding tutoring strategy, are provided to the user together, and the alternative title 1 and the corresponding tutoring strategy with a high matching degree are displayed first. For example, a stacked display can be used, with the highly matched alternative question 1 and the corresponding counseling strategy placed in the first layer / first level that the user can directly view; the less matched alternative question 2 and the corresponding counseling strategy are placed in a lower layer or most of their content is hidden. Only when the user selects the top name of alternative question 2, such as result 2, will the entire content, that is, all alternative questions 2 and the corresponding counseling strategies, be expanded to display.

[0147] Tutoring strategies may include, but are not limited to, one or more of the following: text-based solutions, video-based solutions, and online solutions. For example, content related to Alternative Question 1, video, image, or text-based solutions, and online or video-based tutoring / teaching methods may be provided. Text-based solutions may or may not include the answer; video solutions can be opened by the user by clicking on them, or a video pop-up can be displayed to demonstrate the solution process; online solutions may directly redirect the user to a tutor's page or provide a link for the user to select and then redirect to the tutor's page, thereby obtaining individual tutoring for the user or for that question or question type, etc.

[0148] It can be seen that the present invention combines the text of the question (the text converted by OCR) with the image search process of the question and performs a matching test with the actual text correct and accurate text question that is searched, and determines the correct question that matches the question, that is, the resource question with the highest matching degree, which helps to improve the accuracy and correctness of the image recognition processing, thereby helping to accurately locate the correct tutor and provide the correct tutoring strategy. The present invention uses computer processing matching degree detection to ultimately determine which question should be given the corresponding tutoring strategy based on the level of matching. It can more accurately determine the actual question object and provide the correct tutoring strategy. The algorithm has low complexity, is simple and convenient, easy to implement, and has high efficiency. Furthermore, according to the multi-faceted and multi-level tutoring method, the user is guided to match the corresponding tutoring strategy and find the most appropriate corresponding tutor, avoiding the waste of various resources and time, and effectively improving the user experience.

[0149] In addition, the module structure is not a limitation to the system of the present invention. For example, the input module and the conversion module may not be included in the search and match acquisition module 110. Modules can be combined arbitrarily by splitting functions into modules.

[0150] [Example 3]

[0151] The following describes an overall application scenario and further illustrates the implementation process of the present invention in combination with Examples 1 and 2:

[0152] A database is used as a storage resource, containing many exercise books (e.g., books) and exercises. Each exercise has corresponding strategies, such as tutoring, solutions, and teaching. Users can capture any exercise page (image or video) they want. For example, a terminal can open a learning client app and capture an image of the exercise being worked on in the capture interface, i.e., input the image. The exercise image is converted by optical character recognition (OCR) to obtain text information or a feature image of the exercise content. Keyword analysis of the text information or feature extraction of the image is used to retrieve the corresponding text exercise information from the database. All retrieved text exercise information is ranked as search results, such as those containing the most keywords or key feature images. The top 10 results are then ranked, and a match is calculated between these 10 results and the text information in the user-entered exercise image, or the resource images of these 10 results and the feature image of the exercise content in the user-entered exercise image. The result with the highest match score is then output as the corresponding text exercise. Furthermore, corresponding supplementary learning strategies, such as video solutions and online tutoring / teaching, are provided. That is, the efficiency and accuracy of image / video question searching are improved by using computers, and based on the improved accurate question searching, accurate and correct corresponding tutoring strategies are provided to users or such questions.

[0153] More specifically, combined Figures 6 to 8 The process of providing coaching strategies displayed in the APP interface shown is a more specific description of an example of an online application scenario of the technical solution according to the present invention, so as to further understand the implementation and application of the present invention from the user's perspective.

[0154] Figure 6 In the case of a student who needs guidance on a topic, they can open an app that provides guidance strategies. This app is based on the topic-based guidance strategy providing method described in the present invention. The app provides a function for taking pictures or extracting existing pictures from the gallery and displaying them on the interface. If a topic search result has already been obtained, a "Take another picture" function (button) is also provided on the display interface, which allows you to start searching for and providing guidance on a new topic.

[0155] Students choose or take pictures of the topic, e.g. Figure 6 The top picture contains "3. Fill in the form" and a table. The table contains the graphics and fill-in-the-blank content (math problem) in the question. The picture of the question already has content filled in by students, etc.

[0156] According to the technical solution of the present invention, a search is performed to obtain multiple results (i.e., preliminary search results), and after providing a matching degree calculation, the results are output in the order of matching degree scores. Figure 6In the example, the five results sorted by matching scores are not all displayed. Instead, result 1 (or the result with the highest score) is displayed by default. The text "result 1" can be highlighted or marked as displayed or selected. Figure 6 The picture corresponding to the "question" in the middle is the resource picture of result 1. The picture contains "2. Fill in the table" and a table. The table also contains the graphics and sky content in the question and even content that someone has filled in before (math problem). In the "answer" part, there can be a text / text answer, such as Figure 6 If the question has the correct answer, you can directly prompt "see the answer above" or Figure 8 Here, a link button "One-click Help" for "Question Answers" is displayed synchronously on the APP interface, providing a tutoring strategy for online teacher answers and even video answers for the result 1 with the highest score.

[0157] Can be designed to Figure 6 In this case, you click on the link for online tutoring and choose the teacher's answer. If the tutoring strategy believes that the matching result 1 is already very accurate and the text answer is clear and correct, a dialog box such as Figure 7 As shown, prompts and interactions such as "The match is very accurate, the answer is correct, and you don't need a teacher to explain it" are given (for example, the prompt sentence "Your question is correct, you can view it on the current page" and corresponding interactive buttons, such as "Continue asking questions" and "View" buttons). Some questions require a dynamic solution process to explain. In this way, when displaying result 1, in addition to the text solution, a video solution is also required to demonstrate the problem-solving process.

[0158] In addition, the interface can also provide various other extended functions such as obtaining answers or consulting teachers as quickly as possible for free or for a fee when the question cannot be found. The interface can even provide printing functions, adding a wrong question book, and interactive feedback functions if the resource picture under the provided "question" is not the question you want to search for, etc.

[0159] In addition, Figure 8 In the example shown, a similar question explanation function can also be provided. Select other similar results from the 5 results. For example, click on result 4 that was not directly displayed on the interface before and provide an "answer". Figure 8 Further, after selecting result 4, you can also provide "answers to similar questions" and expand the video of similar questions (that is, other students with lower scores can also obtain other tutoring strategies based on the provided selections). You can also provide similarity and other explanations. In most cases, corresponding text / text "answers" are also provided, such as Figure 8As shown in the figure, for the same type of questions, the corresponding video, remote online and other tutoring strategies are also provided according to the student's choice of interaction. In other words, various one-on-one tutoring is provided after selection.

[0160] As can be seen, embodiments of the present invention can provide a corresponding coaching strategy display interface for each search result based on the matching degree of each search result. The coaching strategy also includes coaching guidance phrases corresponding to the search result based on the matching degree, as well as interactive phrases with the user. The entire result display interface varies accordingly based on the matching degree, providing a better user experience.

[0161] Figures 6 to 8 This is merely an example of an interactive interface for an application program (APP) in the solution of the present invention and is not intended to limit the interface display methods of the present invention. Other display methods may also be used, such as displaying all search results in order, providing corresponding text answers, and simultaneously providing a video answer for the result with the highest matching score, with a link to the video and online answer.

[0162] [Example 4]

[0163] Figure 3 It is a structural schematic block diagram of an electronic device according to an embodiment of the present invention, which includes a processor and a memory, wherein the memory is used to store a computer executable program. When the computer program is executed by the processor, the processor executes the method for providing a question-based tutoring strategy as in the aforementioned embodiment 1.

[0164] like Figure 3 As shown, the electronic device is implemented as a general-purpose computing device. The processor may be one or multiple processors working in concert. The present invention also does not exclude distributed processing, meaning that the processors may be dispersed across different physical devices. The electronic device of the present invention is not limited to a single entity but may also be the sum of multiple physical devices.

[0165] The memory stores a computer executable program, generally a machine-readable code, which can be executed by the processor to enable the electronic device to perform the method of the present invention, or at least some steps in the method.

[0166] The memory includes a volatile memory, such as a random access memory unit (RAM) and / or a cache memory unit, and may also be a non-volatile memory, such as a read-only memory unit (ROM).

[0167] Optionally, in this embodiment, the electronic device further includes an I / O interface for exchanging data between the electronic device and an external device. The I / O interface can represent one or more of several types of bus structures, including a storage unit bus or storage unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus structures.

[0168] More specifically, see Figure 4 1 is a block diagram of a more specific example of an electronic device according to this embodiment. The electronic device 200 of this exemplary embodiment is implemented as a general-purpose data processing device. The components of the electronic device 200 may include, but are not limited to, at least one processing unit 210, at least one storage unit 220, a bus 230 connecting various system components (including the storage unit 220 and the processing unit 210), a display unit 240, and the like.

[0169] The storage unit 220 stores a computer-readable program, which may be a source program or a read-only program. The program can be executed by the processing unit 210, causing the processing unit 210 to perform the steps of various embodiments of the present invention. For example, the processing unit 210 can perform the steps of the methods of the aforementioned embodiments 2 to 5.

[0170] The storage unit 220 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 2201 and / or a cache memory unit 2202, and may further include a read-only memory unit (ROM) 2203. The storage unit 220 may also include a program / utility 2204 having a set (at least one) of program modules 2205, such program modules 2205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination thereof may include an implementation of a network environment.

[0171] Bus 230 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0172] The electronic device 200 can also communicate with one or more external devices 300 (e.g., a keyboard, a display, a network device, a Bluetooth device, etc.), allowing a user to interact with the electronic device 200 via these external devices 300, and / or allowing the electronic device 200 to communicate with one or more other data processing devices (e.g., a router, a modem, etc.). Such communication can be performed via an input / output (I / O) interface 250, or with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via a network adapter 260. The network adapter 260 can communicate with other modules of the electronic device 200 via the bus 230. It should be understood that, although not shown in the figures, other hardware and / or software modules may be used in the electronic device 200, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0173] It should be understood that Figure 3 、 4 The electronic device shown is merely an example of the present invention. The electronic device of the present invention may also include elements or components not shown in the above examples. For example, some electronic devices also include display units such as screens, and some electronic devices also include human-computer interaction elements such as buttons and keyboards. As long as the electronic device can execute a computer-readable program stored in its memory to implement the method of the present invention or at least some of the steps of the method, it is considered an electronic device covered by the present invention.

[0174] [Example 5]

[0175] Figure 5 FIG is a schematic diagram of a computer readable recording medium according to an embodiment of the present invention. Figure 8 As shown, a computer-readable recording medium stores a computer-executable program, and when the computer-executable program is executed, the method for providing a question-based tutoring strategy of the present invention is implemented. The computer-readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0176] The program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, and the like, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0177] Through the above description of the embodiments, it is easy for those skilled in the art to understand that the present invention can be implemented by hardware capable of executing specific computer programs, such as the system of the present invention, and the electronic processing unit, server, client, mobile phone, control unit, processor, etc. contained in the system. The present invention can also be implemented by a vehicle containing at least a portion of the above-mentioned system or components. The present invention can also be implemented by computer software that executes the method of the present invention, such as control software executed by a microprocessor, electronic control unit, client, server, etc. on the locomotive side. However, it should be noted that the computer software that executes the method of the present invention is not limited to being executed by one or a specific hardware entity. It can also be implemented in a distributed manner by unspecified hardware. For example, some method steps executed by the computer program can be executed on the locomotive side, and another part can be executed in a mobile terminal or smart helmet, etc. For computer software, the software product can be stored in a computer-readable storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.), or it can be distributed and stored on a network, as long as it can enable an electronic device to execute the method according to the present invention.

[0178] Through the above description of the implementation modes, those skilled in the art will easily understand that the exemplary embodiments described in the present invention can be implemented by software, or by combining software with necessary hardware.

[0179] The specific embodiments described above further illustrate the objectives, technical solutions, and beneficial effects of the present invention. It should be understood that the present invention is not inherently related to any specific computer, virtual device, or electronic device, and various general-purpose devices can also implement the present invention. The above description is merely a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the scope of protection of the present invention.

Claims

1. A method for providing a tutoring strategy based on a topic, characterized in that: include: Obtaining search results corresponding to a topic, including: obtaining an image containing the topic, determining the relative position of the topic in the image, removing areas outside the area corresponding to the topic in the image according to the relative position to remove interference information, performing text recognition conversion on the image after removing the interference information to obtain corresponding recognition text, and performing a search based on the recognition text in combination with the image of the topic to obtain the search results; wherein, performing a search based on the recognition text in combination with the image of the topic includes: counting the number of text contents of the recognition text and determining whether the number is greater than or equal to a preset threshold; if so, analyzing the recognition text to obtain the recognition text. Keywords of the text and performing text retrieval based on the keywords to obtain resource text title information with the same keywords as the retrieval result, wherein analyzing the recognized text to obtain the keywords of the recognized text includes obtaining the keywords by statistically analyzing the frequency of words, whitelist matching and / or entity naming model prediction after correcting the recognized text; if not, analyzing the image of the title corresponding to the recognized text to obtain a feature image of the title, performing image retrieval based on the feature image to obtain resource images with the same feature image, and extracting the text title information of each resource image as the retrieval result, wherein the feature image includes images and features cut out from the image image of the title; and, Obtaining the matching degree between the title and each search result; According to the matching degree, a corresponding tutoring strategy is provided, and the tutoring strategy includes one or more of the following: text answer, video answer, and online answer.

2. The method according to claim 1, characterized in that Obtaining the matching degree between the title and each search result, including: Obtaining a recognition text of the question, wherein the recognition text is obtained by text recognition of an image of the question; The recognition text is tested for matching with the text title of each search result to obtain the matching degree of each search result.

3. The method according to claim 1, characterized in that Obtaining the matching degree between the title and each search result, including: The degree of matching between the title and each search result is calculated by one or more of the three methods: a similarity-based prediction model, an end-to-end neural network model, and an image matching detection; wherein the image matching detection refers to the matching detection between the image picture of the title and the resource image of each search result.

4. The method according to claim 1, characterized in that: Provide corresponding counseling strategies based on the matching degree, including: Displaying the corresponding text answers of the search results of the question in order from high to low according to the matching scores; A video answer and / or an online answer is also provided for the search result with the highest matching score.

5. The method according to claim 4, characterized in that A video answer and / or an online answer is also provided for the search result with the highest matching score, including at least one of the following: Directly display the video of the problem-solving process, or first display a video answer button and then display the video of the problem-solving process after the user selects it; Directly jump to the online answer page to provide one-on-one tutoring, or display a link to the online answer page that can provide one-on-one tutoring for the user to confirm and jump to; wherein, the one-on-one tutoring includes: assigning a tutor to the user who inputs the question for tutoring.

6. The method according to claim 5, characterized in that Also includes: The coaching strategy is provided based on the title of the search result with the highest matching degree, and / or the coaching strategy further includes coaching guiding words corresponding to the score of the matching degree.

7. The method according to claim 3, characterized in that The calculating the matching degree between the title and each search result includes at least one of the following: Extract features from the recognized text and the search results respectively and input them into a trained similarity-based prediction model to predict the matching score; Inputting the recognized text and the search result into a trained end-to-end neural network model to predict a matching score; According to the feature image and the resource image of the search result, a matching degree score is calculated by a template matching algorithm or a feature matching algorithm.

8. A system for providing a tutoring strategy based on a topic, characterized in that: include: A retrieval matching acquisition module is used to obtain retrieval results corresponding to a topic, including: obtaining an image picture containing the topic, determining the relative position of the topic in the image picture, removing areas outside the area corresponding to the topic in the image picture according to the relative position to remove interference information, performing text recognition conversion on the image picture after removing the interference information to obtain corresponding recognition text, and performing retrieval based on the recognition text in combination with the image picture of the topic to obtain the retrieval result; wherein, performing retrieval based on the recognition text in combination with the image picture of the topic includes: counting the number of text contents of the recognition text and determining whether the number is greater than or equal to a preset threshold; if so, analyzing the recognition text to obtain relevant information of the recognition text. Keywords and perform text retrieval based on the keywords to obtain resource text title information with the same keywords as the retrieval results, wherein analyzing the recognized text to obtain the keywords of the recognized text includes obtaining the keywords by statistically analyzing the frequency of words, whitelist matching and / or entity naming model prediction after correcting the recognized text; if not, analyzing the image of the image position of the title corresponding to the recognized text to obtain the feature image of the title, performing image retrieval based on the feature image to obtain resource images with the same feature image, and extracting the text title information of each resource image as the retrieval result, wherein the feature image includes a -image and features cut out from the image image of the title; and obtaining the matching degree between the title and each retrieval result; The tutoring strategy providing module is used to provide corresponding tutoring strategies according to the matching degree, and the tutoring strategies include one or more of the following: text answers, video answers, and online answers.

9. An electronic device comprising a processor and a memory, wherein the memory is used to store a computer executable program, wherein: When the computer executable program is executed by the processor, the processor performs the method according to any one of claims 1 to 7.

Images