Question searching method and device, terminal and computer readable storage medium

Abstract

The invention belongs to the technical field of intelligent terminals, and provides a question searching method and device, a terminal and a computer readable storage medium. And when the current preview frame image is the initial preview frame image or the questions in the current preview frame image are changed, pre-searching the questions of the whole page of the current page based on the current preview frame image, that is, searching the questions of the whole page in advance. According to the invention, when the terminal receives the selection operation of the user on the target question in the current page, the terminal can directly obtain the target question searching result corresponding to the target question from the question pre-searching result, and does not need to search questions in a local database or a question searching server containing a large number of questions, so that the question searching speed is improved, and the user experience is improved. Second-level loading of the question searching result can be achieved, and the question searching experience of a user is improved.

Description

Technical Field

[0001] The present application belongs to the technical field of intelligent terminals, and in particular, relates to a question searching method, device, terminal and computer-readable storage medium. Background Art

[0002] With the development of terminal technology, the question search function of intelligent terminals such as learning machines and teaching aids can capture images of book pages, perform optical character recognition (OCR) on the images of book pages, search for content related to the text recognized by OCR in the database and display it, for example, searching for answers to questions, pronunciation or translation, etc., which brings great convenience to student users' learning.

[0003] However, the current question search function still has the problem of slow search speed, and it is impossible to load the search results in seconds. Summary of the invention

[0004] The embodiments of the present application provide a question searching method, device, terminal and computer-readable storage medium, which can solve the problem of slow question searching speed in current question searching methods.

[0005] The first aspect of the embodiment of the present application provides a question searching method, which comprises:

[0006] Get the current preview frame image obtained by the camera capturing the image of the current book page;

[0007] When the current preview frame image is an initial preview frame image or the title in the current preview frame image is changed, pre-searching the title of the entire page of the current book page based on the current preview frame image to obtain the pre-search result of the current book page;

[0008] When a selection operation of a user on a target title in the current book page is monitored, a target search result corresponding to the target title is obtained from the pre-search result.

[0009] A second aspect of an embodiment of the present application provides a question searching device, the question searching device comprising:

[0010] An acquisition unit, used for acquiring a current preview frame image obtained by a camera performing image acquisition on a current book page;

[0011] A first question search unit, configured to perform a preliminary question search on the entire page of the current book page based on the current preview frame image when the current preview frame image is an initial preview frame image or the question in the current preview frame image is changed, to obtain a preliminary question search result for the current book page;

[0012] The second question searching unit is configured to obtain a target question searching result corresponding to the target question from the pre-question searching result when monitoring a user's selection operation on the target question in the current book page.

[0013] A third aspect of an embodiment of the present application provides a terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the question searching method as described in the first aspect above is implemented.

[0014] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the question-searching method as described in the first aspect above is implemented.

[0015] A fifth aspect of an embodiment of the present application provides a computer program product. When the computer program product runs on a terminal, the terminal implements the question searching method as described in the first aspect above when executing the computer program product.

[0016] In an embodiment of the present application, a current preview frame image is obtained by acquiring a current page of a book through image acquisition by a camera, and when the current preview frame image is an initial preview frame image or a title in the current preview frame image changes, a pre-search of the entire page of titles of the current page of the book is performed based on the current preview frame image, that is, the entire page of titles is searched in advance, so that when the terminal receives a user's selection operation on a target title in the current page of the book, it can directly obtain a target search result corresponding to the target title from the pre-search result, that is, after the user selects a target title to be searched in the current page, the terminal can directly obtain the target search result corresponding to the target title from the pre-search result, without searching in a local database containing a large number of titles or a search server, thereby improving the search speed, enabling the search results to be loaded in seconds, and improving the user's search experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0018] Figure 1 This is a schematic diagram of a first implementation flow of a question search method provided in an embodiment of the present application;

[0019] Figure 2 This is a schematic diagram of a first implementation flow of detecting whether a title in a current preview frame image has changed, provided by an embodiment of the present application;

[0020] Figure 3 This is a schematic diagram of a second implementation flow of detecting whether a title in a current preview frame image has changed provided by an embodiment of the present application;

[0021] Figure 4 It is a schematic diagram of the implementation process of parallel search provided in the embodiment of the present application;

[0022] Figure 5 It is a schematic diagram of an implementation flow of matching a current preview frame image with a book page stored in a local database provided by an embodiment of the present application;

[0023] Figure 6 This is a schematic diagram of a second implementation flow of a question search method provided in an embodiment of the present application;

[0024] Figure 7 is a schematic diagram of the structure of a question-searching device provided in an embodiment of the present application;

[0025] Figure 8 It is a schematic diagram of the structure of the terminal provided in the embodiment of the present application. DETAILED DESCRIPTION

[0026] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0027] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0028] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0029] "One embodiment" or "some embodiments" described in this specification means that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the sentences "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification are not necessarily the same embodiments, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0030] In the related art, the question search function of intelligent terminals such as learning machines and teaching aids generally searches for the questions to be searched only after the questions to be searched are determined. For example, the question search is performed only after the user clicks or selects the question to be searched (the question to be searched). Therefore, there is a problem of slow question search speed, and the search results cannot be loaded in seconds.

[0031] Based on the above technical problems, the embodiments of the present application provide a question searching method, device, terminal and computer-readable storage medium, which can improve the question searching speed and realize the loading of question searching results in seconds.

[0032] The scheme of the present application is illustrated below by way of examples.

[0033] For example, Figure 1 The present invention provides a flowchart of a method for searching questions provided by an embodiment of the present invention. The method for searching questions is applied to a terminal provided with a camera and can be executed by a searching device on the terminal. The terminal can be a smart terminal such as a learning machine, a teaching aid, a mobile phone, a laptop computer, etc. The present invention does not impose any restrictions on the specific type of the terminal. The method for searching questions can be implemented in the following steps 101 to 103.

[0034] Step 101, obtaining a current preview frame image obtained by capturing an image of a current book page by a camera.

[0035] In the embodiment of the present application, the above-mentioned camera can be a front camera of the terminal or a rear camera of the terminal.

[0036] The above-mentioned current preview frame image refers to the preview frame image generated at the current moment when the camera collects the external light signal. Generally, the collection frequency of the preview frame image is 30 frames per second.

[0037] Step 102, when the current preview frame image is the initial preview frame image or the title in the current preview frame image is changed, pre-search the entire page title of the current book page based on the current preview frame image to obtain the pre-search result of the current book page.

[0038] In the embodiment of the present application, the initial preview frame image refers to the first preview frame image containing the question captured by the camera after the terminal turns on the question search function.

[0039] Specifically, after the terminal turns on the question search function, the first frame captured by the camera contains a preview frame image of the question, and when the user turns pages or changes books, causing the question in the current preview frame image to change, it means that the question in the current preview frame image is a question that has not been searched, and the terminal has not cached the pre-search results of the questions on the current page of the book. Therefore, it is necessary to pre-search the entire page of questions on the current page of the book based on the current preview frame image in order to obtain the pre-search results of the current page of the book.

[0040] It should be noted that the above-mentioned pre-searching of the entire page of titles of the current book page based on the current preview frame image means pre-searching each title in the current book page. Therefore, the pre-search result of the current book page includes the search result of each title in the current book page.

[0041] Step 103, when monitoring the user's selection operation on the target title in the current book page, obtaining the target search result corresponding to the target title from the pre-search result.

[0042] In an embodiment of the present application, the current book page may contain multiple questions, and the user generally only pays attention to one question at a time, that is, only selects one question and views its search results. Therefore, the terminal can monitor the user's selection operation on a target question in the current book page, and because the pre-search results of the current book page include the search results of each question in the current book page, when the terminal monitors the selection operation, it can obtain the target search results corresponding to the target question from the pre-search results.

[0043] It is understandable that the user can also select multiple target topics in the current book page, that is, select multiple target topics in the current book page at one time. When the terminal monitors the user's selection operation on multiple target topics in the current book page, it can obtain the target search results corresponding to these multiple target topics from the pre-search results based on the selection operation.

[0044] For example, when the user uses the reading pen to select a target topic by framing the topic, one or more target topics can be framed at a time. When the user uses a finger or the reading pen to select a target topic by clicking, one target topic can be clicked at a time.

[0045] It should be noted that the "topic" mentioned in the embodiments of the present application can be a word, a phrase, a sentence or a paragraph, and the "topic" can be a math topic, a Chinese topic, an English topic or other subject topic. When the topic is a word, a phrase, a sentence or a paragraph, the above-mentioned pre-search result can be the answer, pronunciation, translation and other types of answer information corresponding to the topic. This application does not limit the type of topic, nor does it limit the type of pre-search result.

[0046] In a specific application, the above-mentioned terminal can be provided with a display screen. When the user turns on the question search function of the terminal, the camera of the terminal can capture images of books, test papers (current book pages) and other objects within its shooting range, obtain the current preview frame image, and display the current preview frame image.

[0047] Next, the terminal can detect whether the current preview frame image is the initial preview frame image or whether the title in the current preview frame image has changed, and when the current preview frame image is the initial preview frame image or the title in the current preview frame image has changed (for example, the user turns pages or changes books), the terminal can pre-search the entire page of titles of the current book page based on the current preview frame image to obtain the pre-search result of the current book page. At the same time, the terminal monitors the user's selection operation of the target title in the current book page displayed on the display screen, and when the user's selection operation of the target title in the current book page displayed on the display screen is monitored, the target search result corresponding to the target title is obtained from the pre-search result for display, so as to achieve the loading of the search result in seconds.

[0048] In an embodiment of the present application, a current preview frame image is obtained by acquiring an image of the current book page through a camera, and when the current preview frame image is the initial preview frame image or the title in the current preview frame image changes, a pre-search of the entire page of titles of the current book page is performed based on the current preview frame image, that is, the entire page of titles is searched in advance, so that when the terminal receives a user's selection operation on a target title in the current book page, it can directly obtain a target search result corresponding to the target title from the pre-search result, that is, after the user selects a target title to be searched in the current page, the terminal can directly obtain the target search result corresponding to the target title from the pre-search result, without searching in a local database containing a large number of titles or a search server, thereby improving the search speed, being able to load the search results in seconds, and improving the user's search experience.

[0049] In one embodiment of the present application, Figure 2 As shown, after the above step 101, it is possible to detect whether the title in the current preview frame image has changed based on the following steps 201 to 202, and if the title in the current preview frame image has changed, continue to execute the above steps 102 to 103.

[0050] Step 201: Calculate a first similarity between a current preview frame image and a previous preview frame image.

[0051] In an embodiment of the present application, in the process of calculating the first similarity between the current preview frame image and the previous preview frame image, the first similarity between the current preview frame image and the previous preview frame image can be determined by calculating the image feature information of the current preview frame image and the feature information of the previous preview frame image, and by comparing the image feature information of the current preview frame image with the feature information of the previous preview frame image. The specific comparison process can be implemented by referring to the similarity calculation formula in the relevant technology.

[0052] Among them, the above-mentioned image feature information may include one or more image feature information of Histogram of Oriented Gradient (HOG) features, Local Binary Pattern (LBP) features, Haar features (Haar-like features), Scale Invariant Feature Transform (SIFT) features, Speeded Up Robust Features (SURF), and ORB features (Oriented FAST and Rotated BRIEF), and this application does not impose any restrictions on this.

[0053] Step 202: If the first similarity is less than a first preset similarity threshold, it is determined that the title in the current preview frame image has changed.

[0054] In the embodiment of the present application, the above-mentioned first preset similarity threshold can be obtained according to practical experience, for example, it can be set to 97% or 98%, and the present application does not impose any limitation on this.

[0055] In addition to using the above Figure 2 In addition to detecting whether the title in the current preview frame image has changed in the manner shown in FIG. Figure 3 As shown, it is also possible to detect whether the title in the current preview frame image is changed based on the following steps 301 to 303.

[0056] Step 301, obtaining multiple preview frame images captured most recently by the camera.

[0057] In the embodiment of the present application, the multiple preview frame images most recently captured by the camera refer to the multiple preview frame images continuously captured by the camera including the current preview frame image.

[0058] Step 302: Detect page turning and book changing based on the multiple preview frame images.

[0059] Step 303: When a page turning action or a book changing action is detected, it is determined that a title in the current preview frame image has changed.

[0060] In the embodiment of the present application, the above-mentioned page turning action and book changing action detection can be implemented using a trained artificial intelligence model.

[0061] The artificial intelligence model may include: a convolutional neural network model CNN, a recurrent neural network model RNN and a deep neural network model DNN. Among them, the convolutional neural network model is a multi-layer neural network that can continuously reduce the dimension of image recognition problems with huge amounts of data, and ultimately enable them to be trained. Therefore, the artificial intelligence model in the embodiment of the present application may be a convolutional neural network model.

[0062] In the embodiment of the present application, before using the convolutional neural network model to detect the page turning action and book changing action of the above-mentioned multiple preview frame images, the untrained convolutional neural network model can be trained using the continuously collected multiple sample frames and the action labels of the multiple sample frames to obtain a trained convolutional neural network model. The action labels of the multiple sample frames are used to indicate whether the multiple sample frames are sample images corresponding to the page turning action or the book changing action.

[0063] Optionally, the training step of the convolutional neural network model may include: obtaining a plurality of continuously collected sample images and action labels of the plurality of sample images; using the convolutional neural network model to detect page turning actions and book changing actions on the plurality of sample images, and adjusting the parameters of the convolutional neural network model according to the detection results, until the adjusted convolutional neural network model can detect whether the plurality of sample images are sample images corresponding to page turning actions or book changing actions, or the accuracy of detecting whether the plurality of sample images are sample images corresponding to page turning actions or book changing actions is greater than a preset threshold, and the adjusted convolutional neural network model is used as a trained convolutional neural network model.

[0064] Among them, the parameters of the above-mentioned convolutional neural network model may include the weights, biases, coefficients of the regression function of each convolutional layer in the convolutional neural network model, and may also include the learning rate, number of iterations, number of neurons in each layer, etc. At present, commonly used convolutional neural network models include RCNN (Regions based Convolutional Neural Network, region-based convolutional neural network) model, Fast-RCNN model and Faster-RCNN model. Among them, the Faster-RCNN model is evolved on the basis of the RCNN model and the fast-RCNN model. Compared with the RCNN model and the fast-RCNN model, the faster-RCNN model still cannot realize real-time detection of images, but compared with the RCNN model and the fast-RCNN model, it has higher detection accuracy and detection speed. Therefore, in some embodiments of the present application, the above-mentioned convolutional neural network model can use the faster-RCNN model.

[0065] It should be noted that the above Figure 2 and Figure 3 The method is merely an example of a method for detecting whether the title in the current preview frame image has changed (i.e., whether there is a page turning action or a book changing action), and is not intended to limit the scope of protection of the present application. Other methods for detecting whether the title in the current preview frame image has changed are also applicable to the present application, and are not listed one by one here.

[0066] Furthermore, it should be noted that in practical applications, the above Figure 2 and Figure 3 The detection methods can be selected to be executed by one of them, or both can be executed at the same time to improve the detection accuracy, and this application does not impose any restrictions on this.

[0067] In order to further improve the speed of searching, in one embodiment of the present application, in the process of executing the above step 102 to obtain the preliminary search results of the current book page, it can be achieved by parallelly searching the entire page of titles of the current book page in the local database and the search server based on the current preview frame image.

[0068] Specifically, since the entire page of titles of the current book page is pre-searched in the local database and the title search server respectively in a parallel execution mode, rather than first searching the local database and then searching the title in the title search server, when the pre-search result of the current book page cannot be found in the local database, the search time will not be extended, and the search time is the same as the search time when the title is searched only in the title search server. When the pre-search result of the current book page is stored in the local database, since the number of titles in the local database is much smaller than the number of titles in the title search server, the pre-search result of the current book page can be quickly searched.

[0069] Therefore, in the embodiment of the present application, a pre-search of the entire page of titles of the current book page is performed in parallel in the local database and the title search server respectively. Compared with the method of first searching the title in the local database and then searching the title in the title search server, the speed of searching the title can be further improved.

[0070] In one embodiment, Figure 4 As shown, the above-mentioned method of concurrently searching the entire page title of the current book page in the local database and the title search server based on the current preview frame image can be specifically implemented based on the following steps 401 to 403.

[0071] Step 401, matching the current preview frame image with the book page stored in the local database, and sending the current preview frame image to the question search server, which performs a preliminary question search on the entire page of the current book page based on the current preview frame image.

[0072] That is, in a parallel execution mode, the entire page of titles of the current book page is pre-searched in the local database and the title search server respectively.

[0073] Step 402: If there is a book page in the local database that successfully matches the current preview frame image, the answer information corresponding to the book page in the local database that successfully matches the current preview frame image is used as the pre-search result of the current book page.

[0074] Step 403: If there is no book page successfully matching the current preview frame image in the local database, the answer information corresponding to the book page matching the current preview frame image searched by the question search server is used as the pre-question search result of the current book page.

[0075] That is to say, in the process of searching the entire page of questions of the current book page in parallel in the local database and the question search server, when there is a book page that successfully matches the current preview frame image in the local database, the preliminary question search result of the current book page searched by the question search server can be discarded, and the answer information corresponding to the book page that successfully matches the current preview frame image in the local database can be directly used as the preliminary question search result of the current book page. When there is no book page that successfully matches the current preview frame image in the local database, the answer information corresponding to the book page that matches the current preview frame image searched by the question search server can be used as the preliminary question search result of the current book page.

[0076] Specifically, since the question search server searches for answer information among a large number of questions, the search time is generally longer than the search time in the local database. Therefore, when there is a book page in the local database that successfully matches the current preview frame image, there is no need to wait for the pre-search result returned by the question search server. The answer information corresponding to the book page in the local database that successfully matches the current preview frame image can be directly used as the pre-search result of the current book page to quickly determine the pre-search result of the current book page.

[0077] It should be noted that, in some embodiments of the present application, the book pages and the answer information corresponding to the book pages stored in the local database may be pre-stored on the terminal by the terminal service provider; or may be pre-entered on the terminal by the user. For example, the user may store the book pages and the answer information corresponding to the book pages in the terminal in advance by scanning the book pages and the answer pages of the exercise book.

[0078] In addition, in some embodiments of the present application, the book pages and the answer information corresponding to the book pages stored in the above-mentioned local database may be obtained by storing each book page in the target book and the answer information corresponding to each book page in the target book in the local database when the question search server searches for the target book corresponding to the book page matching the current preview frame image after the terminal executes the above-mentioned step 403. The present application does not impose any restrictions on this.

[0079] In an embodiment of the present application, when the target book corresponding to the book page matching the current preview frame image is searched on the question search server, each book page in the target book and the answer information corresponding to each book page in the target book are stored in a local database, so that the user can directly search the local database for the answer information of any question in the current book during the subsequent question search process, which is beneficial to improve the question search speed and accuracy.

[0080] In one embodiment of the present application, Figure 5As shown, in the above step 401, matching the current preview frame image with the book page stored in the local database can be implemented based on the following steps 501 to 502.

[0081] Step 501: Calculate a second similarity between the current preview frame image and the book page stored in the local database.

[0082] In the embodiment of the present application, the implementation method of the above step 501 can refer to the description of the above step 201, which will not be repeated here.

[0083] Step 502: If there is a book page in the local database whose second similarity is greater than or equal to the second preset similarity threshold, it is determined that there is a book page in the local database that successfully matches the current preview frame image.

[0084] In the embodiment of the present application, the value of the second preset similarity threshold may be the same as or different from the value of the first preset similarity threshold, and the present application does not impose any limitation on this.

[0085] In an embodiment of the present application, image search is implemented in the local database by calculating the second similarity between the current preview frame image and the book page stored in the local database, so that there is no need to perform OCR text recognition on the current preview frame image during the question search process. Therefore, fast search of questions can be achieved, further improving the question search speed.

[0086] It should be noted that, in some embodiments of the present application, in addition to the above Figure 5 In addition to searching by calculating similarity, you can also search by optical character recognition (OCR).

[0087] Specifically, in the above step 401, when matching the current preview frame image with the book page stored in the local database, the current preview frame image can be firstly subjected to OCR recognition to obtain the OCR text information of the current preview frame image, and then the OCR text information is compared with the OCR text information of the book page stored in the local database to determine whether a book page successfully matching the current preview frame image is stored in the local database.

[0088] Similarly, in the above step 402, when the title search server performs a pre-title search for the entire page title of the current book page based on the current preview frame image, it can also perform a pre-title search based on the above Figure 5 As shown, the question search is performed by calculating the similarity between images, that is, searching by image, or the question search can be performed by obtaining the OCR text information of the current preview frame image, that is, searching by text retrieval. This application does not impose any restrictions on this.

[0089] It should be noted that in order to improve the accuracy of image search, in the above-mentioned embodiments, a preliminary search is performed on the entire page of titles of the current book page based on the current preview frame image. In the process of obtaining the preliminary search results of the current book page, a preliminary search can be performed based on the entire page of titles, rather than searching each title separately.

[0090] For example, after obtaining the OCR text information of the current preview frame image, text retrieval can be performed based on the entire OCR text information of the current preview frame image to obtain a preliminary search result containing the answer information of each question in the current book page, so as to avoid the situation where the search accuracy is low when using the OCR text information of a single question to search in a question bank with a large number of similar questions. This achieves the goal of improving the search speed while ensuring the search accuracy.

[0091] It should also be noted that in the process of pre-searching questions based on the whole page of questions, if a book page matching the whole page of questions cannot be found, the whole page of questions can be divided into multiple single questions by splitting the questions, and then searching for each single question, and finally obtaining the pre-search result containing the answer information of each question in the whole page of questions, that is, the pre-search result of the current book page.

[0092] Specifically, Figure 6 As shown, in an actual application, the specific execution process of the question searching method of the present application may include the following steps 601 to 612.

[0093] Step 601, the terminal starts the question search function;

[0094] Step 602: The camera of the terminal collects an image of the current book page to obtain a current preview frame image, and displays the current preview frame image;

[0095] Step 603, detecting whether the current preview frame image is the initial preview frame image or whether the title in the current preview frame image has changed;

[0096] In the embodiment of the present application, if the current preview frame image is the initial preview frame image or the title in the current preview frame image has changed, the following steps 604 and 607 are executed in parallel; if the current preview frame image is not the initial preview frame image and the title in the current preview frame image has not changed, the above step 603 is returned to be executed.

[0097] Step 604: Calculate a second similarity between the current preview frame image and the book page stored in the local database.

[0098] Step 605 , determining whether there is a book page in the local database whose second similarity is greater than or equal to a second preset similarity threshold.

[0099] Step 606: If there is a book page in the local database whose second similarity is greater than or equal to the second preset similarity threshold, it is determined that there is a book page in the local database that successfully matches the current preview frame image, and the answer information corresponding to the book page in the local database that successfully matches the current preview frame image is cached.

[0100] Step 607: Obtain the entire OCR text information of the current preview frame image.

[0101] In the embodiment of the present application, the above step 607 can be executed by the question search server to reduce the amount of question search calculation of the terminal.

[0102] Step 608, performing text retrieval based on the entire OCR text information of the current preview frame image in the question search server;

[0103] Step 609: If a book page matching the entire page of questions cannot be found, the question search server divides the entire page of questions into multiple single questions by way of question segmentation, and then searches each single question, and finally obtains a preliminary question search result containing answer information for each question in the entire page of questions;

[0104] Step 610, if a book page matching the entire page of questions is found, and there is no book page matching the current preview frame image in the local database, the answer information corresponding to the book page matching the current preview frame image found by the question search server is used as the pre-search result of the current book page and cached.

[0105] Step 611, detecting whether the title search server has searched for a target book corresponding to a book page matching the current preview frame image.

[0106] Step 612: If the question search server searches for a target book corresponding to a book page matching the current preview frame image, each book page in the target book and the answer information corresponding to each book page in the target book are stored in a local database.

[0107] In an embodiment of the present application, after the above step 602, it also includes monitoring the user's selection operation on the target title in the current book page, and when monitoring the user's selection operation on the target title in the current book page, obtaining the target search result corresponding to the target title from the pre-search result.

[0108] In an embodiment of the present application, by turning on the preview function of the camera and turning on page turning and book changing detection when the question search function is turned on, when the current preview frame image is the initial preview frame image or the question in the current preview frame image changes, a pre-search is performed on the entire page of questions of the current book page based on the current preview frame image, that is, the entire page of questions is searched in advance, so that when the terminal receives the user's selection operation on the target question in the current book page, it can directly obtain the target search result corresponding to the target question from the pre-search result, that is, after the user selects the target question to be searched in the current page, the terminal can directly obtain the target search result corresponding to the target question from the pre-search result, without searching in a local database containing a large number of questions or a question search server, thereby improving the speed of searching questions, and can achieve the loading of search results in seconds, thereby improving the user's question search experience.

[0109] It should be noted that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0110] For example, the above step 604 and step 607 are executed in parallel rather than one after another.

[0111] The present application also provides a question search device, such as Figure 7 As shown, the question searching device 700 may include an acquisition unit 701 , a first question searching unit 702 , and a second question searching unit 703 .

[0112] An acquisition unit, used for acquiring a current preview frame image obtained by a camera performing image acquisition on a current book page;

[0113] A first question search unit is used to perform a preliminary question search on the entire page of the current book page based on the current preview frame image when the current preview frame image is the initial preview frame image or the question in the current preview frame image changes, so as to obtain a preliminary question search result for the current book page;

[0114] The second question searching unit is used to obtain a target question searching result corresponding to the target question from the pre-question searching result when monitoring a user's selection operation on the target question in the current book page.

[0115] In one embodiment, the question-searching device further includes a computing unit, which is used to:

[0116] Calculating a first similarity between the current preview frame image and the previous preview frame image;

[0117] If the first similarity is less than a first preset similarity threshold, it is determined that the title in the current preview frame image has changed.

[0118] In one embodiment, the above-mentioned computing unit is further specifically used for:

[0119] After acquiring the current preview frame image obtained by the camera capturing the image of the current book page, acquiring multiple preview frame images captured most recently by the camera;

[0120] Detect page turning and book changing actions based on multiple preview frame images;

[0121] When a page turning action or a book changing action is detected, it is determined that a title in the current preview frame image is changed.

[0122] In one embodiment, the first question searching unit 702 is further used for:

[0123] Based on the current preview frame image, the entire page of the current book page is searched in parallel in the local database and the search server to obtain the preliminary search result of the current book page.

[0124] In one embodiment, the first question searching unit 702 is further specifically used for:

[0125] Matching the current preview frame image with the book page stored in the local database, and sending the current preview frame image to the question search server, which performs a preliminary search for the entire page title of the current book page based on the current preview frame image;

[0126] If there is a book page in the local database that successfully matches the current preview frame image, the answer information corresponding to the book page in the local database that successfully matches the current preview frame image is used as the pre-search result of the current book page;

[0127] If there is no book page successfully matching the current preview frame image in the local database, the answer information corresponding to the book page matching the current preview frame image searched by the question search server is used as the pre-question search result of the current book page.

[0128] In one embodiment, the above-mentioned computing unit is further specifically used for:

[0129] Calculating a second similarity between the current preview frame image and the book page stored in the local database;

[0130] If there exists a book page in the local database whose second similarity is greater than or equal to the second preset similarity threshold, it is determined that there exists a book page in the local database that successfully matches the current preview frame image.

[0131] In one embodiment, the first question searching unit 702 is further specifically used for:

[0132] After the answer information corresponding to the book page matching the current preview frame image searched by the question search server is used as the pre-question search result of the current book page, if the question search server searches for a target book corresponding to the book page matching the current preview frame image, each book page in the target book and the answer information corresponding to each book page in the target book are stored in the local database.

[0133] It should be noted that, for the convenience and simplicity of description, the specific working process of the above-described question-searching device 700 can refer to the above-mentioned Figures 1 to 6 The corresponding process of the question search method described in will not be described in detail here.

[0134] Figure 8 Schematic diagram of the structure of the terminal provided in the embodiment of the present application. Figure 8 As shown, the terminal 8 of this embodiment may include: a processor 80, a memory 81, and a computer program 82 stored in the memory 81 and executable on the processor 80, such as a question search program. When the processor 80 executes the computer program 82, the steps in the above-mentioned various question search method embodiments are implemented, such as Figure 1 Alternatively, when the processor 80 executes the computer program 82, the functions of each unit in the above-mentioned device embodiments are realized, for example Figure 6 The functions of the various units shown.

[0135] Exemplarily, the computer program 82 may be divided into one or more modules / units, one or more modules / units are stored in the memory 81, and are executed by the processor 80 to complete the present application. One or more modules / units may be a series of computer program instruction segments that can complete specific functions, and the instruction segments are used to describe the execution process of the computer program 82 in the terminal 8. For example, the computer program 82 may be divided into an acquisition unit, a first question search unit, and a second question search unit, and the specific functions of each unit are as follows:

[0136] An acquisition unit, used for acquiring a current preview frame image obtained by a camera performing image acquisition on a current book page;

[0137] A first question search unit is used to perform a preliminary question search on the entire page of the current book page based on the current preview frame image when the current preview frame image is the initial preview frame image or the question in the current preview frame image changes, so as to obtain a preliminary question search result for the current book page;

[0138] The second question searching unit is used to obtain a target question searching result corresponding to the target question from the pre-question searching result when monitoring a user's selection operation on the target question in the current book page.

[0139] Those skilled in the art will understand that Figure 8It is only an example of a terminal and does not constitute a limitation on the terminal. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the question-searching device may also include input and output devices, network access devices, buses, etc.

[0140] The processor 80 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0141] The memory 81 may be an internal storage unit of the terminal 8, such as a hard disk or memory of the terminal 8. The memory 81 may also be an external storage device of the terminal 8, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the terminal 8. Further, the terminal 8 may also include both an internal storage unit of the terminal 8 and an external storage device. The memory 81 is used to store computer programs and other programs and data required by the question-searching device. The memory 81 may also be used to temporarily store data that has been output or is to be output.

[0142] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0143] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0144] An embodiment of the present application provides a computer program product. When the computer program product runs on a terminal, the terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0145] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by controlling the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera / search device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0146] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0147] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0148] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0149] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0150] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A question search method, It is characterized in that The question searching method comprises: Get the current preview frame image obtained by the camera capturing the image of the current book page; When the current preview frame image is an initial preview frame image or the title in the current preview frame image is changed, pre-searching the title of the entire page of the current book page based on the current preview frame image to obtain the pre-search result of the current book page; When a selection operation of a user on a target title in the current book page is monitored, a target search result corresponding to the target title is obtained from the pre-search result.

2. The question searching method according to claim 1, It is characterized in that After acquiring the current preview frame image obtained by the camera capturing the image of the current book page, the method includes: Calculating a first similarity between the current preview frame image and the previous preview frame image; If the first similarity is less than a first preset similarity threshold, it is determined that the title in the current preview frame image has changed.

3. The question searching method according to claim 1, It is characterized in that After acquiring the current preview frame image obtained by the camera capturing the image of the current book page, the method includes: Obtain the latest preview frames of the images captured by the camera; Performing page turning action and book changing action detection based on the multiple preview frame images; When a page turning action or a book changing action is detected, it is determined that the title in the current preview frame image is changed.

4. The question searching method according to any one of claims 1 to 3, It is characterized in that The pre-searching the entire page title of the current book page based on the current preview frame image to obtain the pre-search result of the current book page includes: Based on the current preview frame image, the entire page title of the current book page is searched in parallel in the local database and the title search server to obtain the preliminary title search result of the current book page.

5. The question searching method according to claim 4, It is characterized in that The step of searching the entire page title of the current book page in parallel in the local database and the title search server based on the current preview frame image to obtain the preliminary title search result of the current book page includes: Matching the current preview frame image with the book page stored in the local database, and sending the current preview frame image to the question search server, where the question search server pre-searches the entire page title of the current book page based on the current preview frame image; If there is a book page in the local database that successfully matches the current preview frame image, then the answer information corresponding to the book page in the local database that successfully matches the current preview frame image is used as the pre-search result of the current book page; If there is no book page successfully matching the current preview frame image in the local database, the answer information corresponding to the book page matching the current preview frame image searched by the question search server is used as the pre-question search result of the current book page.

6. The question searching method according to claim 5, It is characterized in that The matching of the current preview frame image with the book pages stored in the local database includes: Calculating a second similarity between the current preview frame image and the book page stored in the local database; If a book page with a second similarity greater than or equal to a second preset similarity threshold exists in the local database, it is determined that a book page successfully matched with the current preview frame image exists in the local database.

7. The question searching method according to claim 5, It is characterized in that After the answer information corresponding to the book page matching the current preview frame image searched by the question search server is used as the pre-question search result of the current book page, the method includes: If the question search server searches for a target book corresponding to a book page matching the current preview frame image, each book page in the target book and answer information corresponding to each book page in the target book are stored in the local database.

8. A question searching device, It is characterized in that The question searching device comprises: An acquisition unit, used for acquiring a current preview frame image obtained by a camera performing image acquisition on a current book page; A first question search unit, configured to perform a preliminary question search on the entire page of the current book page based on the current preview frame image when the current preview frame image is an initial preview frame image or the question in the current preview frame image is changed, to obtain a preliminary question search result for the current book page; The second question searching unit is configured to obtain a target question searching result corresponding to the target question from the pre-question searching result when monitoring a user's selection operation on the target question in the current book page.

9. A terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the question searching method as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program. It is characterized in that When the computer program is executed by a processor, the question searching method according to any one of claims 1 to 7 is implemented.

Images