Search methods, devices, electronic equipment, and storage media for illustration titles

Abstract

This disclosure provides a method, apparatus, electronic device, and storage medium for searching illustration titles. The method includes: inputting an image to be identified into a pre-trained lightweight image recognition model to obtain an image recognition result; if the image recognition result indicates that the image to be identified is a title image, inputting the image to be identified into a pre-trained title image feature extraction model to obtain a first title feature vector; performing a title search in a title database based on the title feature vector to obtain a first title search result and returning the first title search result; if the image recognition result indicates that the image to be identified is not a title image, returning a prompt message indicating that the image to be identified is not a title image. This solution enables the title search system to recognize non-title images, and the use of a lightweight image recognition model helps improve recognition efficiency.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to a method, apparatus, electronic device, and storage medium for searching illustration titles. Background Technology

[0002] In the field of intelligent education, scenarios involving the recognition of user-uploaded question images are frequently encountered. For example, in a question-and-answer system, the system recognizes the question image uploaded by the user and returns several questions most similar to the uploaded question, along with their corresponding answers and explanations, for the user's reference.

[0003] In practical applications, some users upload non-topic images, such as sensitive images, to the question-and-answer system for unknown purposes. Since the current question-and-answer system cannot recognize non-topic images, its ability to distinguish images is poor. Summary of the Invention

[0004] In order to solve the above-mentioned technical problems, or at least partially solve the above-mentioned technical problems, the present disclosure provides a method, apparatus, electronic device and storage medium for searching illustration titles.

[0005] According to one aspect of this disclosure, a method for searching illustration titles is provided, comprising:

[0006] The image to be recognized is input into a pre-trained lightweight image recognition model to obtain the image recognition result;

[0007] If the image recognition result indicates that the image to be recognized is a question image, the image to be recognized is input into a pre-trained question image feature extraction model to obtain a first question feature vector;

[0008] Based on the question feature vector, a question search is performed in the question database to obtain the first question search result and return the first question search result.

[0009] If the image recognition result indicates that the image to be recognized is not the title image, a prompt message indicating that the image to be recognized is not the title image is returned.

[0010] According to another aspect of this disclosure, a search device for illustration titles is provided, comprising:

[0011] The first acquisition module is used to input the image to be recognized into a pre-trained lightweight image recognition model to obtain the image recognition result;

[0012] The second acquisition module is used to input the image to be identified into a pre-trained question image feature extraction model when the image recognition result indicates that the image to be identified is a question image, so as to obtain a first question feature vector;

[0013] The question search module is used to search for questions in the question database based on the question feature vector to obtain the first question search result.

[0014] The first return module is used to return the search results for the first question.

[0015] The second return module is used to return a prompt message indicating that the image to be identified is not a title image when the image recognition result indicates that the image to be identified is not a title image.

[0016] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0017] Processor; and

[0018] Stored program memory,

[0019] The program includes instructions that, when executed by the processor, cause the processor to perform the search method for the illustration title according to one of the foregoing aspects.

[0020] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform the search method for the illustrated title according to the foregoing aspect.

[0021] According to another aspect of this disclosure, a computer program product is provided, comprising a computer program, wherein the computer program, when executed by a processor, implements the method for searching illustration titles as described in the foregoing aspect.

[0022] One or more technical solutions provided in this disclosure involve inputting the image to be recognized into a pre-trained lightweight image recognition model to obtain an image recognition result. If the image recognition result indicates that the image to be recognized is a question image, the image to be recognized is input into a pre-trained question image feature extraction model to obtain a first question feature vector. Then, based on the question feature vector, a question search is performed in a question database to obtain a first question search result and return it. If the image recognition result indicates that the image to be recognized is not a question image, a prompt message indicating that the image to be recognized is not a question image is returned. By adopting the solution of this disclosure, question images and non-question images can be identified during question search. Only question images are returned with corresponding question search results, while non-question images are prompted. This enables the question search system to recognize non-question images, and the use of a lightweight image recognition model helps improve recognition efficiency. Attached Figure Description

[0023] Further details, features, and advantages of this disclosure are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0024] Figure 1 A flowchart illustrating a method for searching illustrated titles according to an exemplary embodiment of this disclosure is shown;

[0025] Figure 2 A flowchart illustrating a method for searching illustrated titles according to another exemplary embodiment of this disclosure is shown;

[0026] Figure 3 A schematic block diagram of a search apparatus for illustrated titles is shown according to an exemplary embodiment of the present disclosure;

[0027] Figure 4 A structural block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0028] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0029] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0030] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc., used in this disclosure are only used to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0031] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0032] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0033] The following description, with reference to the accompanying drawings, outlines the search methods, apparatus, electronic devices, and storage media for the illustrated titles provided in this disclosure.

[0034] Figure 1 A flowchart is shown of a method for searching illustrated questions according to an exemplary embodiment of the present disclosure. This method can be executed by an illustrated question search device provided in an embodiment of the present disclosure, wherein the device can be implemented in software and / or hardware, and is generally integrated into an electronic device, including devices that support question search functions such as mobile phones and tablets, as well as intelligent learning devices such as question answering systems.

[0035] like Figure 1 As shown, the search method for this illustration title may include the following steps:

[0036] Step 101: Input the image to be recognized into a pre-trained lightweight image recognition model to obtain the image recognition result.

[0037] The image to be identified can be an image currently uploaded by the user. Users (e.g., students) can upload the image by taking a photo, screenshotting, or other means. The image to be identified may be a title image containing illustrations, or it may be a non-title image such as a sensitive image.

[0038] In this embodiment of the disclosure, for an image to be identified, the image to be identified can be input into a lightweight image recognition model, which will then perform image recognition on the image to be identified and output the image recognition result.

[0039] The lightweight image recognition model is pre-trained. Considering the high response time requirements during question search, this disclosure uses lightweight models such as EfficientNet and MobileNet to train a lightweight image recognition model for image recognition. For example, the EfficientNet model has high running efficiency and good recognition performance. Therefore, the EfficientNet model can be used as the initial model, and iterative training can be performed on the EfficientNet model using a large number of collected training samples to obtain a lightweight image recognition model, which is used to perform image recognition on the input image and output the image recognition result.

[0040] For example, the image recognition result output by the lightweight image recognition model may only include the category number. For example, 0 represents the category number corresponding to the title image, and 1 represents the category number corresponding to the non-title image. If the image recognition result is 0, it means that the image to be recognized is the title image. If the image recognition result is 1, it means that the image to be recognized is the non-title image.

[0041] For example, the image recognition result output by the lightweight image recognition model may include a category number (e.g., 0 represents the title image, 1 represents the non-title image) and the corresponding confidence score. When the category number is that of a non-title image and the corresponding confidence score is greater than a preset confidence threshold, the image to be recognized is determined to be a non-title image; otherwise, the image to be recognized is determined to be a title image. The confidence threshold can be set according to the actual application scenario or the training effect of the lightweight image recognition model. The higher the accuracy requirement of the application scenario, the larger the confidence threshold can be set, but the confidence threshold shall not exceed 1. For example, in a high-accuracy recognition scenario, the confidence threshold can be set to 0.95.

[0042] In one optional embodiment of this disclosure, before inputting the image to be recognized into the lightweight image recognition model, the image to be recognized may be preprocessed. The preprocessing may include, but is not limited to, scaling the image to a fixed size, standardizing the image data, etc.

[0043] Step 102: If the image recognition result indicates that the image to be recognized is a question image, the image to be recognized is input into a pre-trained question image feature extraction model to obtain a first question feature vector.

[0044] The question image feature extraction model is pre-trained. For example, several question images can be collected in advance as training samples. The lightweight model can be iteratively trained using the collected training samples to obtain the question image feature extraction model, which is used to extract features from the image to be recognized.

[0045] In this embodiment of the disclosure, after obtaining the image recognition result corresponding to the image to be recognized, it can be determined whether the image to be recognized is a question image based on the image recognition result. If the image recognition result indicates that the image to be recognized is a question image, the image to be recognized (or the pre-processed image to be recognized) can be input into a pre-trained question image feature extraction model, which will extract features and output the corresponding feature vector, referred to as the first question feature vector.

[0046] Step 103: Based on the question feature vector, perform a question search in the question database to obtain the first question search result and return the first question search result.

[0047] In this embodiment of the disclosure, after obtaining the first question feature vector corresponding to the image to be identified, the first question feature vector can be used to search for questions in the question database to obtain the corresponding question search results, which are called the first question search results. The first question search results may include one question or multiple questions; this disclosure does not limit the number of questions included in the obtained first question search results.

[0048] For example, a query can be performed in a pre-indexed question database based on the obtained first question feature vector. Each question in the question database corresponds to an index, which can be represented as a vector. The feature vectors corresponding to each question in the question database can be used as the question indexes. The question database can be an Elasticsearch library (ES) or other vector search libraries such as Milvus. During the query, the similarity between the first question feature vector and each index in the question database is calculated, and the questions corresponding to each index are sorted in descending order of similarity. The top N questions are returned as the first question search result, where N is a positive integer.

[0049] In this embodiment of the disclosure, after obtaining the first question search result, the first question search result can be returned to the user for reference, thereby realizing the question search corresponding to the image to be identified.

[0050] Step 104: If the image recognition result indicates that the image to be recognized is not the title image, return a prompt message indicating that the image to be recognized is not the title image.

[0051] In this embodiment of the disclosure, after obtaining the image recognition result corresponding to the image to be recognized, if the image recognition result indicates that the image to be recognized is not a title image, the electronic device can return a prompt message that the image to be recognized is not a title image, so as to provide the user with relevant information to alert the user that the image uploaded this time is not a title image and cannot be used for title search.

[0052] For example, a prompt message such as "Image error, please upload a correct image containing the question" can be displayed on the screen of an electronic device in the form of a pop-up window to prompt the user.

[0053] The method for searching illustration titles according to embodiments of this disclosure involves inputting the image to be identified into a pre-trained lightweight image recognition model to obtain an image recognition result. If the image recognition result indicates that the image to be identified is a title image, the image to be identified is input into a pre-trained title image feature extraction model to obtain a first title feature vector. Then, a title search is performed in a title database based on the title feature vector to obtain a first title search result and return it. If the image recognition result indicates that the image to be identified is not a title image, a prompt message indicating that the image to be identified is not a title image is returned. By employing the scheme of this disclosure, title images and non-title images can be identified during title search. Only title images are returned with corresponding title search results, while prompts are provided for non-title images. This enables the title search system to recognize non-title images, and the use of a lightweight image recognition model helps improve recognition efficiency.

[0054] In one optional embodiment of this disclosure, when the image recognition result indicates that the image to be recognized is a non-title image, the image to be recognized (or the image to be recognized after preprocessing) can be input into a pre-trained non-title image feature extraction model to obtain a non-title feature vector. Then, based on the non-title feature vector, an image search is performed in the image library, and the similarity between each image in the image library and the non-title feature vector (called the first similarity) is calculated. If the maximum value of the first similarity is greater than a preset threshold, a prompt message indicating that the image to be recognized is a non-title image is returned.

[0055] The preset threshold can be set according to actual needs, and this disclosure does not limit the specific value of the preset threshold. The non-topic image feature extraction model is pre-trained. For example, several non-topic images can be collected in advance as training samples, and the lightweight model can be iteratively trained using the collected training samples to obtain the non-topic image feature extraction model, which is used to extract features from the image to be recognized. Using a highly efficient lightweight model to train the topic image feature extraction model and the non-topic image feature extraction model can make the feature extraction of the image to be recognized more efficient.

[0056] In this embodiment, when the image recognition result indicates that the image to be recognized is a non-title image, the image to be recognized is further input into a non-title image feature extraction model for feature extraction to obtain a non-title feature vector. This non-title feature vector is then used to search for images in a pre-indexed image library. The image library contains multiple non-title images, each corresponding to an index, which can be represented by the feature vector of the non-title image. During the image search, the similarity between the non-title feature vector and each index in the image library can be calculated as the similarity between the non-title feature vector and each non-title image in the image library (referred to as the first similarity). Next, a maximum value can be determined from the first similarity, and this maximum value is compared with a preset threshold. If the maximum value is greater than the preset threshold, the image to be recognized is determined to be a non-title image, and a prompt message indicating that the image to be recognized is a non-title image is returned. Thus, when the image recognition model identifies an image as a non-title image, a secondary judgment is performed by extracting features and conducting an image search, ensuring the accuracy of non-title image recognition.

[0057] Figure 2 A flowchart illustrating a method for searching illustrated titles according to another exemplary embodiment of this disclosure is shown. Figure 2 As shown, the search method for this illustration title may include the following steps:

[0058] Step 201: Input the image to be recognized into a pre-trained lightweight image recognition model to obtain the image recognition result.

[0059] Step 202: If the image recognition result indicates that the image to be recognized is a question image, the image to be recognized is input into a pre-trained question image feature extraction model to obtain a first question feature vector.

[0060] Step 203: Based on the question feature vector, perform a question search in the question database to obtain the first question search result and return the first question search result.

[0061] Step 204: If the image recognition result indicates that the image to be recognized is not a title image, the image to be recognized is input into a pre-trained non-title image feature extraction model to obtain a non-title feature vector.

[0062] Step 205: Based on the non-topic feature vector, perform an image search in the image library and calculate the first similarity between each image in the image library and the non-topic feature vector.

[0063] Step 206: Using the question image feature extraction model, obtain the second question feature vector of the image to be identified.

[0064] Step 207: Based on the second question feature vector, perform a question search in the question database and calculate the second similarity between each question in the question database and the second question feature vector.

[0065] It is understood that the execution order of steps 204-205 and steps 206-207 is not important; they can be executed simultaneously or sequentially. This embodiment only uses the example of steps 206-207 being executed after step 205 to explain this disclosure, and should not be regarded as a limitation of this disclosure.

[0066] Step 208: If the maximum value in the first similarity is greater than a preset threshold, and the maximum value in the first similarity is greater than the maximum value in the second similarity, return a prompt message indicating that the image to be identified is not the title image.

[0067] The preset threshold can be set according to actual needs, and this disclosure does not limit its specific value.

[0068] It should be noted that for a detailed description of each step in the embodiments of this disclosure, please refer to the relevant descriptions in the foregoing embodiments, which will not be repeated here.

[0069] In this embodiment of the present disclosure, when the image recognition result indicates that the image to be recognized is a non-topic image, the image to be recognized is subjected to both topic image feature extraction and non-topic image feature extraction. Specifically, a second topic feature vector is extracted using a topic image feature extraction model, and a non-topic feature vector is extracted using a non-topic image feature extraction model. Then, the similarity between each topic in the question bank and the second topic feature vector (referred to as the second similarity) is calculated, and the similarity between each image in the image bank and the non-topic feature vector (referred to as the first similarity) is calculated. The maximum value in the first similarity and the maximum value in the second similarity are compared, as well as the relationship between the maximum value in the first similarity and a preset threshold. If the maximum value in the first similarity is greater than the preset threshold and the maximum value in the first similarity is greater than the maximum value in the second similarity, then the image to be recognized is determined to be a non-topic image, and a prompt message indicating that the image to be recognized is a non-topic image is returned.

[0070] In one optional embodiment of this disclosure, if the maximum value in the first similarity score is not greater than a preset threshold, or if the maximum value in the first similarity score is not greater than the maximum value in the second similarity score, then a second question search result is determined based on the second similarity score. The second question search result is then returned to the user for reference, thus realizing question image-based question search. For example, the questions in the question bank can be sorted in descending order of similarity based on the second similarity score, and the top N questions (N is a positive integer) can be returned to the user as the second question search result.

[0071] Considering that in practical applications, the number of non-topic images uploaded by users is far less than the number of topic images, the solution disclosed in this disclosure adopts a design approach of image recognition before feature extraction and search. This ensures that in practical applications, the uploaded topic images undergo a process of one recognition, one feature extraction, and one search, thereby guaranteeing the efficiency of topic search. For the very few images identified as non-topic images by the image recognition model, a second determination of whether the uploaded image is a non-topic image is made by extracting non-topic feature vectors and topic feature vectors and comparing their similarity. This ensures both processing efficiency and maximum accuracy. Compared to using only one feature extraction model and one search library, the accuracy of the solution disclosed in this disclosure is greatly improved. Furthermore, in this embodiment, a topic image feature extraction model is used to extract topic feature vectors, and a non-topic image feature extraction model is used to extract non-topic feature vectors. This method of extracting corresponding feature vectors using specific models, compared to using only one feature extraction model, results in feature vectors with stronger feature representation capabilities, contributing to improved accuracy of search results.

[0072] The method for searching illustration titles in this embodiment further extracts title features and non-title features when the image recognition result indicates that the image to be identified is a non-title image, and performs searches on them separately. The maximum similarity is compared to determine whether the image to be identified is a non-title image. This multi-judgment and confirmation method can ensure the accuracy of the non-title image determination result.

[0073] In one optional embodiment of this disclosure, the non-title image feature extraction model further outputs a classification result of the image to be identified, wherein the classification result indicates whether the image to be identified is a non-title image. That is, the non-title image feature extraction model includes two output heads: a feature output head and a classification output head. The feature output head outputs the non-title feature vector of the image to be identified, and the classification output head outputs the classification result indicating whether the image to be identified is a non-title image. This multi-task trained non-title image feature extraction model, with the added classification output head, enables the model to extract non-title feature vectors with better feature representation capabilities.

[0074] For example, the classification result output by the non-topic image feature extraction model may only include the category number. For example, 0 represents the category number corresponding to the topic image, and 1 represents the category number corresponding to the non-topic image. If the classification result is 0, it means that the image to be identified is a topic image. If the classification result is 1, it means that the image to be identified is a non-topic image.

[0075] For example, the classification result output by the non-topic image feature extraction model may include a category number (e.g., 0 represents a title image, 1 represents a non-topic image) and the corresponding confidence score. When the category number is that of a non-topic image and the corresponding confidence score is greater than a preset confidence score threshold, the image to be identified is determined to be a non-topic image; otherwise, the image to be identified is determined to be a title image. The confidence score threshold can be set according to the actual application scenario or the training effect of the non-topic image feature extraction model. This disclosure does not limit the specific value of the confidence score threshold.

[0076] In this embodiment of the disclosure, the classification result output by the non-topic image feature extraction model can be used to correct the maximum value in the first similarity, so as to correct the calculation error during image search when the image quality of the image to be identified is poor, thereby improving the recall capability of image search results while ensuring high accuracy.

[0077] Therefore, in this embodiment of the disclosure, when the maximum value of the first similarity is greater than a preset threshold and the maximum value of the first similarity is greater than the maximum value of the second similarity, returning a prompt message indicating that the image to be identified is a non-title image may include: responding to the classification result output by the non-title image feature extraction model indicating that the image to be identified is a non-title image, increasing the maximum value of the first similarity by a preset value to obtain a third similarity; and returning a prompt message indicating that the image to be identified is a non-title image when the third similarity is greater than the preset threshold and the third similarity is greater than the maximum value of the second similarity. The preset value can be set according to actual needs and experience; for example, the preset value can be set to 0.1. If the third similarity is not greater than the preset threshold, or the third similarity is not greater than the maximum value of the second similarity, then the image to be identified is determined to be a normal title image, and the title search result is determined based on the second similarity and returned to the user for reference. For example, based on the second similarity, the N most similar questions can be obtained from the question bank as title search results and returned to the user, where N is a positive integer.

[0078] In this embodiment of the disclosure, when the classification result output by the non-title image feature extraction model indicates that the image to be identified is a non-title image, the maximum similarity calculated during image search is increased by a preset value to obtain a third similarity. Then, the third similarity is compared with the maximum value in the second similarity and the preset threshold to determine whether the image to be identified is a non-title image. This helps to correct the problem of large errors in image search results when the image quality is poor. While ensuring high accuracy, it improves the recall capability of electronic devices for non-title images, which is very suitable for illustration title search scenarios.

[0079] In one optional embodiment of this disclosure, both the non-topic image feature extraction model and the topic image feature extraction model are lightweight models, and the number of parameters in the non-topic image feature extraction model is greater than the number of parameters in the topic image feature extraction model.

[0080] For example, the MobileNetV3-Small model can be used for feature extraction of question images, while the MobileNetV3-Large model can be used for feature extraction of non-question images. The MobileNetV3-Large model and the MobileNetV3-Small model have the same network structure, but the number of convolutional kernels is different. The MobileNetV3-Large model has a larger number of parameters than the MobileNetV3-Small model.

[0081] Considering that most of the images uploaded by users are normal title images, and only a very small number of them are non-title images, the feature extraction model for title images in this embodiment adopts a lightweight, small-scale model to ensure the efficiency of feature extraction for title images. For the occasionally uploaded non-title images, a lightweight, large-scale model is used for feature extraction. The non-title image feature extraction model contains more model parameters than the title image feature extraction model. Although the larger number of parameters may lead to a slightly lower feature extraction efficiency, it ensures that the extracted non-title feature vectors have better feature representation capabilities, thereby making the image search results more accurate.

[0082] In one alternative embodiment of this disclosure, the network structure of the lightweight image recognition model differs from that of the non-topic image feature extraction model.

[0083] The difference between the network structure of the lightweight image recognition model and the network structure of the non-topic image feature extraction model means that the lightweight image recognition model and the non-topic image feature extraction model do not use the same model. For example, it is not possible for both to use the EfficientNet model, nor are they models from the same series. For example, MobileNetV1, MobileNetV2, and MobileNetV3 belong to the same series of network models, and their network structures are the same. Therefore, the lightweight image recognition model and the non-topic image feature extraction model cannot both be one of the three models.

[0084] For example, the lightweight image recognition model can use the EfficientNet model, while the non-topic image feature extraction model can use the MobileNetV3 model. Since they use different models, their network structures are different. This ensures the complementarity of the determination results for whether the image to be identified is a non-topic image, thereby further improving the recognition accuracy of non-topic images.

[0085] This exemplary embodiment also provides a search device for illustration titles. Figure 3 A schematic block diagram of a search apparatus for illustrated titles according to exemplary embodiments of the present disclosure is shown, such as Figure 3 As shown, the problem search device 30 includes: a first acquisition module 310, a second acquisition module 320, a problem search module 330, a first return module 340, and a second return module 350.

[0086] The first acquisition module 310 is used to input the image to be recognized into a pre-trained lightweight image recognition model to obtain the image recognition result.

[0087] The second acquisition module 320 is used to input the image to be identified into a pre-trained question image feature extraction model when the image recognition result indicates that the image to be identified is a question image, so as to obtain a first question feature vector;

[0088] The question search module 330 is used to perform a question search in the question database based on the question feature vector to obtain a first question search result.

[0089] The first return module 340 is used to return the search results for the first question.

[0090] The second return module 350 is used to return a prompt message indicating that the image to be recognized is not a title image when the image recognition result indicates that the image to be recognized is not a title image.

[0091] Optionally, the second return module 350 includes:

[0092] The first acquisition unit is used to input the image to be identified into a pre-trained non-topic image feature extraction model to obtain a non-topic feature vector when the image recognition result indicates that the image to be identified is not a title image.

[0093] The second acquisition unit is used to perform image search in the image library based on the non-topic feature vector, and calculate the first similarity between each image in the image library and the non-topic feature vector;

[0094] The return unit is used to return a prompt message that the image to be identified is not the title image if the maximum value in the first similarity is greater than a preset threshold.

[0095] Optionally, the second return module 350 further includes:

[0096] The third acquisition unit is used to acquire the second question feature vector of the image to be identified using the question image feature extraction model;

[0097] The fourth acquisition unit is used to search for questions in the question bank based on the second question feature vector, and to calculate the second similarity between each question in the question bank and the second question feature vector.

[0098] The return unit is further configured to: return a prompt message indicating that the image to be identified is not the title image when the maximum value in the first similarity is greater than a preset threshold and the maximum value in the first similarity is greater than the maximum value in the second similarity.

[0099] Optionally, the non-topic image feature extraction model also outputs the classification result of the image to be identified; the return unit is further used for:

[0100] In response to the classification result indicating that the image to be identified is a non-title image, the maximum value of the first similarity is increased by a preset value to obtain a third similarity;

[0101] If the third similarity is greater than the preset threshold and the third similarity is greater than the maximum value of the second similarity, a prompt message is returned indicating that the image to be identified is not the title image.

[0102] Optionally, the second return module 350 further includes:

[0103] The fifth acquisition unit is used to determine the second question search result based on the second similarity when the maximum value in the first similarity is not greater than a preset threshold, or when the maximum value in the first similarity is not greater than the maximum value in the second similarity.

[0104] The return unit is also used to return the search results for the second question.

[0105] Optionally, both the non-topic image feature extraction model and the topic image feature extraction model are lightweight models, and the number of parameters in the non-topic image feature extraction model is greater than the number of parameters in the topic image feature extraction model.

[0106] Optionally, the network structure of the lightweight image recognition model is different from the network structure of the non-topic image feature extraction model.

[0107] The illustration title search device provided in this disclosure can execute any illustration title search method applicable to electronic devices provided in this disclosure, and has the corresponding functional modules and beneficial effects for executing the method. Content not described in detail in the device embodiments of this disclosure can be referred to the descriptions in any method embodiments of this disclosure.

[0108] Exemplary embodiments of this disclosure also provide an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, the computer program, when executed by the at least one processor, causing the electronic device to perform a search method for the illustrated titles according to embodiments of this disclosure.

[0109] Exemplary embodiments of this disclosure also provide a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform a search method for the illustrated titles according to embodiments of this disclosure.

[0110] Exemplary embodiments of this disclosure also provide a computer program product, including a computer program, wherein, when executed by a computer's processor, the computer program is used to cause the computer to perform a search method for the illustrated titles according to embodiments of this disclosure.

[0111] refer to Figure 4 The present invention describes a structural block diagram of an electronic device 1100 that can serve as a server or client of the present disclosure, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0112] like Figure 4 As shown, the electronic device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1102 or a computer program loaded from a storage unit 1108 into a random access memory (RAM) 1103. The RAM 1103 may also store various programs and data required for the operation of the device 1100. The computing unit 1101, ROM 1102, and RAM 1103 are interconnected via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0113] Multiple components in electronic device 1100 are connected to I / O interface 1105, including: input unit 1106, output unit 1107, storage unit 1108, and communication unit 1109. Input unit 1106 can be any type of device capable of inputting information to electronic device 1100. Input unit 1106 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of electronic device. Output unit 1107 can be any type of device capable of presenting information and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 1108 may include, but is not limited to, disk and optical disk. Communication unit 1109 allows electronic device 1100 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth™ devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0114] The computing unit 1101 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1101 performs the various methods and processes described above. For example, in some embodiments, the method for searching illustration titles can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1108. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 1100 via ROM 1102 and / or communication unit 1109. In some embodiments, the computing unit 1101 can be configured to perform the method for searching illustration titles by any other suitable means (e.g., by means of firmware).

[0115] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0116] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0117] As used in this disclosure, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0118] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0119] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0120] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

Claims

1. A method for searching illustration titles, wherein, The method includes: The image to be recognized is input into a pre-trained lightweight image recognition model to obtain the image recognition result; If the image recognition result indicates that the image to be recognized is a question image, the image to be recognized is input into a pre-trained question image feature extraction model to obtain a first question feature vector; Based on the first question feature vector, a question search is performed in the question database to obtain the first question search result and return the first question search result. If the image recognition result indicates that the image to be recognized is a non-title image, a prompt message indicating that the image to be recognized is a non-title image is returned. Specifically, when the image recognition result indicates that the image to be recognized is a non-title image, the image to be recognized is input into a pre-trained non-title image feature extraction model. This model includes a feature output head and a classification output head. The feature output head outputs the non-title feature vector of the image to be recognized, and the classification output head outputs a classification result indicating whether the image to be recognized is a non-title image. Based on the non-title feature vector corresponding to the image to be recognized, an image search is performed in the image database, and the characteristics of each image in the image database are calculated. The system calculates a first similarity between the image and the non-question feature vector, and obtains a second question feature vector for the image to be identified using the question image feature extraction model. Based on the second question feature vector, it searches for questions in the question database and calculates a second similarity between each question in the question database and the second question feature vector. When the classification result indicates that the image to be identified is a non-question image, it increases the maximum value of the first similarity by a preset value to obtain a third similarity. When the third similarity is greater than a preset threshold and the third similarity is greater than the maximum value of the second similarity, it returns a prompt message that the image to be identified is a non-question image. The image database contains multiple non-question images.

2. The method for searching illustration titles as described in claim 1, wherein, The method further includes: If the maximum value in the first similarity is not greater than a preset threshold, or if the maximum value in the first similarity is not greater than the maximum value in the second similarity, then the second question search result is determined based on the second similarity. Return to the search results for the second question.

3. The method for searching illustration titles as described in claim 1, wherein, Both the non-topic image feature extraction model and the topic image feature extraction model are lightweight models, and the number of parameters in the non-topic image feature extraction model is greater than the number of parameters in the topic image feature extraction model.

4. The method for searching illustration titles as described in any one of claims 1-3, wherein, The network structure of the lightweight image recognition model is different from that of the non-topic image feature extraction model.

5. A search device for illustration titles, wherein, The device includes: The first acquisition module is used to input the image to be recognized into a pre-trained lightweight image recognition model to obtain the image recognition result; The second acquisition module is used to input the image to be identified into a pre-trained question image feature extraction model when the image recognition result indicates that the image to be identified is a question image, so as to obtain a first question feature vector; The question search module is used to search for questions in the question database based on the first question feature vector to obtain the first question search result. The first return module is used to return the search results for the first question. The second return module is used to return a prompt message indicating that the image to be identified is a non-title image when the image recognition result indicates that the image to be identified is a non-title image. Specifically, when the image recognition result indicates that the image to be identified is a non-title image, the image to be identified is input into a pre-trained non-title image feature extraction model. This model includes a feature output head and a classification output head. The feature output head outputs the non-title feature vector of the image to be identified, and the classification output head outputs a classification result indicating whether the image to be identified is a non-title image. Based on the non-title feature vector corresponding to the image to be identified, an image search is performed in the image database, and the value of each feature vector in the image database is calculated. The image is compared with the first similarity between the image and the non-question feature vector, and the second question feature vector of the image to be identified is obtained by using the question image feature extraction model. Based on the second question feature vector, a question search is performed in the question library, and a second similarity is calculated between each question in the question library and the second question feature vector. When the classification result indicates that the image to be identified is a non-question image, the maximum value of the first similarity is increased by a preset value to obtain a third similarity. When the third similarity is greater than a preset threshold and the third similarity is greater than the maximum value of the second similarity, a prompt message indicating that the image to be identified is a non-question image is returned. The image library contains multiple non-question images.

6. An electronic device, comprising: processor; as well as Stored program memory, The program includes instructions that, when executed by the processor, cause the processor to perform the search method for the illustrated title according to any one of claims 1-4.

7. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the search method for the illustration title according to any one of claims 1-4.

Images