Title index acquisition method and apparatus, electronic device, and storage medium

Abstract

The present disclosure provides a question index acquisition method and device, electronic equipment and storage medium. The method comprises: acquiring an answer question image to be acquired; acquiring a target moire background image corresponding to the answer question image from a pre-constructed moire background image library; fusing the answer question image and the target moire background image to generate a simulation question image; performing feature extraction on the simulation question image to obtain a feature vector corresponding to the simulation question image; and determining the feature vector as a question index corresponding to the answer question image. By using the scheme of the present disclosure, the feature difference between the feature of the user-uploaded shooting image and the index of the answer question image is reduced, and the deviation of the feature matching result can be reduced when searching for a question through the feature of the user-uploaded shooting image, thereby improving the accuracy of the question search.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to a method, apparatus, electronic device, and storage medium for obtaining a title index. Background Technology

[0002] With the development of internet technology, image recognition has rapidly emerged and flourished, and its applications have spread to all corners of the national economy and social life, bringing tremendous changes to people's learning, working, and even lifestyles. In many scenarios, image recognition tools process images uploaded by users after taking photos. For example, in educational settings, students need to upload images of questions taken from the screen or paper documents to a Q&A system, which then recognizes the question images and provides corresponding explanations or answers.

[0003] However, since user-uploaded images are usually accompanied by moiré patterns, while the answer images in the question search database are pure white, interference-free images, the background difference between the user-uploaded images and the answer images is large. This results in a large difference in the features extracted from the two images, leading to a large deviation in the feature matching results and a low accuracy rate in question search. Summary of the Invention

[0004] In order to solve the above-mentioned technical problems, or at least partially solve the above-mentioned technical problems, the present disclosure provides a method, apparatus, electronic device and storage medium for obtaining a title index.

[0005] According to one aspect of this disclosure, a method for obtaining a question index is provided, comprising:

[0006] Retrieve the image of the answer question for the index to be retrieved;

[0007] Obtain the target moiré background image corresponding to the answer question image from a pre-built moiré background image library;

[0008] The answer question image is fused with the target moiré background image to generate a simulated question image;

[0009] Feature extraction is performed on the simulated problem image to obtain the feature vector corresponding to the simulated problem image;

[0010] The feature vector is determined as the question index corresponding to the answer question image.

[0011] According to another aspect of this disclosure, a question index acquisition device is provided, comprising:

[0012] The answer / question retrieval module is used to retrieve the image of the answer / question for the index to be retrieved.

[0013] The background image acquisition module is used to acquire the target moiré background image corresponding to the answer question image from a pre-built moiré background image library;

[0014] An image fusion module is used to fuse the answer question image with the target moiré background image to generate a simulated question image;

[0015] The feature extraction module is used to extract features from the simulated problem image to obtain the feature vector corresponding to the simulated problem image.

[0016] The index determination module is used to determine the feature vector as the question index corresponding to the answer question image.

[0017] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0018] Processor; and

[0019] Stored program memory,

[0020] The program includes instructions that, when executed by the processor, cause the processor to perform the question index acquisition method according to one of the foregoing aspects.

[0021] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform the question index acquisition method according to the foregoing aspect.

[0022] According to another aspect of this disclosure, a computer program product is provided, comprising a computer program, wherein the computer program, when executed by a processor, implements the question index acquisition method described in the foregoing aspect.

[0023] One or more technical solutions provided in this disclosure acquire an answer question image to be indexed, and obtain a target moiré background image corresponding to the answer question image from a pre-built moiré background image library. Then, the answer question image and the target moiré background image are fused to generate a simulated question image. Features are then extracted from the simulated question image to obtain a feature vector corresponding to the simulated question image, and this feature vector is determined as the question index corresponding to the answer question image. Using this disclosure, since the simulated question image generated by image fusion carries moiré patterns, the feature vector obtained from feature extraction of the simulated question image is used as the index of the answer question image for question search. This reduces the feature difference between the features of the user-uploaded image and the index of the answer question image, thereby reducing the deviation of feature matching results when searching for questions based on the features of the user-uploaded image, and thus improving the accuracy of question search. Attached Figure Description

[0024] Further details, features, and advantages of this disclosure are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0025] Figure 1 A flowchart of a question index acquisition method according to an exemplary embodiment of the present disclosure is shown;

[0026] Figure 2 A flowchart illustrating a method for obtaining a title index according to another exemplary embodiment of this disclosure is shown;

[0027] Figure 3 A flowchart illustrating a method for obtaining a question index according to yet another exemplary embodiment of this disclosure is shown;

[0028] Figure 4 A schematic diagram showing the boundaries of non-title regions in a user-uploaded image according to an exemplary embodiment of the present disclosure is provided.

[0029] Figure 5 A schematic block diagram of a title index acquisition apparatus according to an exemplary embodiment of the present disclosure is shown;

[0030] Figure 6 A structural block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0031] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0032] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0033] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc., used in this disclosure are only used to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0034] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0035] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0036] The following description, with reference to the accompanying drawings, details the method, apparatus, electronic device, and storage medium for obtaining the title index provided in this disclosure.

[0037] Figure 1 A flowchart of a question index acquisition method according to an exemplary embodiment of the present disclosure is shown. The method can be executed by a question index acquisition device provided in the embodiments of the present disclosure, wherein the device can be implemented in software and / or hardware, and can generally be integrated in an electronic device, including a computer, tablet computer, server and other devices.

[0038] like Figure 1 As shown, the method for obtaining the question index may include the following steps:

[0039] Step 101: Obtain the image of the answer question with the index to be obtained.

[0040] The answer question images are standard question images from the question bank, and their backgrounds are usually pure white, without any moiré patterns or other noise interference.

[0041] In this embodiment of the disclosure, the answer question image to be indexed can be any image in the question bank.

[0042] Step 102: Obtain the target moiré background image corresponding to the answer question image from a pre-built moiré background image library.

[0043] The moiré background image library can be pre-built, which includes multiple moiré background images that do not contain foreground content, such as title-related text, illustrations, etc.

[0044] In this embodiment of the disclosure, when constructing a moiré pattern background image library, moiré pattern background images can be obtained through at least one of the following methods, and the obtained moiré pattern background images can be used to construct the moiré pattern background image library:

[0045] (1) Collect a batch of workbooks and test papers used by students and use various mobile phones on the market to randomly take some pictures of the blank pages of the workbooks as moiré background images. When taking pictures, try to simulate the user's shooting behavior. For example, you can collect some images in indoor and outdoor environments, morning, noon and evening under different lighting conditions.

[0046] (2) Take random photos of some images with moiré patterns on a blank screen and blank pages on the screen as moiré pattern background images. When taking photos, try to simulate the user's shooting behavior. For example, you can collect some images in indoor and outdoor environments, morning, noon and evening under different lighting conditions.

[0047] (3) Obtain several real question images uploaded by online users, and use a pre-trained detection model to detect the text and question illustrations in the real question images to obtain the position information of the detection boxes corresponding to the text and question illustrations. Then, based on the position information of the detection boxes, erase the text and question illustrations in the real question images to obtain background images. The erasure method is to fill the text area and question illustration area in the real question images with white. Then, use image inpainting technology to fill the text area and question illustration area of ​​the background images with background pixels to obtain several moiré background images. Optionally, in order to ensure the quality of the moiré background images, before erasing the text and question illustrations, the detection results output by the detection model can be checked by manual review to ensure the accuracy of the detection results of the text and question illustrations in the real question images. Then, the question information is erased and the background pixels are filled on the checked images. Among them, different filling methods can be used when filling the text area and the illustration area with background pixels. Examples are as follows:

[0048] For example, a background pixel filling model can be pre-trained, such as a U2Net model or a VAE model. During training, several solid-color background images with moiré patterns can be collected, and portions of these images can be filled with white. The white-filled images are used as training samples, while the corresponding original background images are used as supervised training images. In practical use, the background image obtained by erasing the text and illustrations from a real question image is input into the trained background pixel filling model. The model then fills the background image with pixels, resulting in a moiré background image where both the text and illustration areas are filled with background pixels.

[0049] For example, a rectangular area with the largest area can be determined from the background image, excluding the text area and the title illustration area. The top left vertex of this rectangular area can be aligned with the top left vertex of the text area, and the size of the rectangular area can be adjusted to match that of the text area to obtain the target rectangular area. Specifically, during resizing, if the width of the rectangular area is greater than the width of the text area, the width of the rectangular area is cropped to the same size as the text area. If the width of the rectangular area is less than the width of the text area, the width of the rectangular area is increased to the same size as the text area. The pixel values ​​of the increased width portion can be used to fill the right boundary pixel values ​​of the rectangular area. In other words, the width of the rectangular area can be increased by adding pixels to the right side of the rectangular area, making the width of the supplemented rectangular area the same as the width of the text area, and the pixel values ​​of the supplemented pixels are consistent with the right boundary pixel values ​​of the rectangular area. If the height of the rectangular area is greater than the height of the text area, the height of the rectangular area is cropped to the same size as the height of the text area. If the height of the rectangular area is less than the height of the text area, the height of the rectangular area is increased to the same size as the height of the text area, and the pixel values ​​of the increased height portion can be used to fill the bottom boundary pixel values ​​of the rectangular area. After obtaining the target rectangular area, the pixel values ​​of the target rectangular area can be used to fill the text area with pixels. For example, the pixel values ​​of each pixel in the text area of ​​the background image can be replaced with the pixel values ​​of each pixel in the target rectangular area. It is understandable that the above filling process only takes the alignment of the top left corner as an example. The top right corner of the text area and the rectangular area can also be aligned, or the bottom left corner can be aligned, or the bottom right corner can be aligned. In addition, the filling process is only explained using the text area as an example. The filling of the illustration area in the title is similar and will not be elaborated here.

[0050] It should be noted that, in this embodiment of the disclosure, for the moiré background image obtained by the above method (3), the moiré background image can be marked using the title identifier corresponding to the original real title image (i.e., the user-uploaded image), that is, the moiré background image is bound to the title identifier corresponding to the real title image, so as to find the corresponding target moiré background image according to the title identifier. The determination of the title identifier of the user-uploaded image can be referred to the relevant description of step 202, or other methods can be used to determine the title identifier of the user-uploaded image, which is not limited in this disclosure.

[0051] In this embodiment of the disclosure, for the obtained answer question image, the corresponding target moiré background image can be obtained from the moiré background image library.

[0052] It should be noted that the embodiments of this disclosure provide different methods to determine the target moiré background image corresponding to the answer question image. The specific process will be described in detail in subsequent embodiments, and will not be repeated here.

[0053] Step 103: Fuse the answer question image with the target moiré background image to generate a simulated question image.

[0054] In this embodiment of the disclosure, after obtaining the target moiré background image, the answer question image can be fused with the target moiré background image to obtain a simulated question image. The simulated question image contains question information (such as text and question illustrations) and is also accompanied by moiré interference.

[0055] For example, during the fusion process, the size of the answer question image and the target moiré background image can be adjusted to the same size. Then, the pixel values ​​of each pixel in the answer question image and the corresponding pixel values ​​in the target moiré background image are summed, weighted, or averaged to obtain the pixel values ​​of each pixel in the simulated question image.

[0056] For example, a question region image can be cropped from the answer question image, and a background image of the same size can be cropped from the target moiré background image based on the size of the question region image. Then, the background image and the question region image are fused to obtain a simulated question image. Image fusion can be performed by summing or averaging the pixel values ​​of corresponding pixels.

[0057] Step 104: Extract features from the simulated problem image to obtain the feature vector corresponding to the simulated problem image.

[0058] In this embodiment of the disclosure, after obtaining the simulated problem image, feature extraction can be performed on the simulated problem image to obtain the corresponding feature vector.

[0059] For example, a feature extraction model can be pre-trained, and the obtained simulation problem image can be input into the feature extraction model for feature extraction. The feature extraction model outputs the feature vector corresponding to the simulation problem image. For example, a lightweight MobileNetV3 model can be used as the feature extraction model. In actual use, other feature extraction models can also be used. This disclosure does not limit the network structure of the feature extraction model.

[0060] In one optional embodiment of this disclosure, before extracting features from the simulated problem image, the simulated problem image can be preprocessed, such as scaling the simulated problem image to a preset size, standardizing the simulated problem image, etc.

[0061] Step 105: Determine the feature vector as the question index corresponding to the answer question image.

[0062] In this embodiment of the disclosure, after obtaining the feature vector corresponding to the simulated question image, the feature vector can be determined as the question index corresponding to the answer question image. The question index can be used to perform feature matching with the feature vector of the user-uploaded image during question search, so as to determine whether the answer question image corresponding to the question index corresponds to the user-uploaded image based on the feature matching result.

[0063] Furthermore, in an optional embodiment of this disclosure, the question index and the question identifier of the answer question image can be bound and stored in an index library. The index library can be, for example, a Milvus search index library, an ElasticSearch (ES) index library, etc. In a practical question search scenario, the feature vector of the user-uploaded image can be extracted, and this feature vector can be matched with each question index in the index library to find the target question index with the highest matching degree. The target question identifier bound to this target question index can then be obtained. Based on the target question identifier, the answer question image corresponding to the target question identifier and its related question information, such as question analysis, solution steps, etc., can be found from the question library.

[0064] In practical applications, the number of question indices corresponding to each answer question image can be set based on the number and size of the answer question images in the question bank and business requirements. The feature vector corresponding to the simulated question image is one of the question indices. For example, the number of question indices corresponding to each answer question image can be set to 5. Theoretically, the larger the number of question indices, the higher the accuracy of question search. However, as the number of question indices increases, the number of question indices that need to be matched during question search also increases, affecting search efficiency. Moreover, after the number of question indices reaches a certain level, further increasing the number will have limited improvement in accuracy. Therefore, the number of question indices corresponding to an answer question image does not need to be set very large and can be set according to actual needs.

[0065] The question index acquisition method of this disclosure involves acquiring an answer question image for which an index is to be obtained, and retrieving a target moiré background image corresponding to the answer question image from a pre-built moiré background image library. The answer question image and the target moiré background image are then fused to generate a simulated question image. Feature extraction is then performed on the simulated question image to obtain a feature vector corresponding to the simulated question image, and this feature vector is determined as the question index corresponding to the answer question image. By employing this scheme, since the simulated question image generated by image fusion carries moiré patterns, the feature vector obtained from feature extraction of the simulated question image is used as the index of the answer question image for question search. This reduces the feature difference between the features of the user-uploaded image and the index of the answer question image, thereby reducing the deviation of feature matching results when searching for questions based on the features of the user-uploaded image, and thus improving the accuracy of question search.

[0066] Figure 2 A flowchart of a method for obtaining a question index according to another exemplary embodiment of this disclosure is shown, such as... Figure 2 As shown, the method for obtaining the question index may include the following steps:

[0067] Step 200: Obtain the image of the answer question with the index to be obtained.

[0068] Step 201: Obtain the target uploaded image associated with the answer question image.

[0069] In this embodiment of the disclosure, the target uploaded image associated with the answer question image can be a real question image corresponding to the answer question image. The real question image is a question image uploaded by an online user (referred to as a user-uploaded image in this embodiment). Alternatively, it can be a real question image uploaded by a user that is most similar to the answer question image. The similarity between answer question images can be characterized by the closest size of the question area in the image, the closest question content contained in the image, etc.

[0070] The existence of a corresponding user-uploaded image for an answer question image can be determined using a preset identifier carried by the answer question image. During the question search process for all users in the system, answer question images are marked based on the search results. If a matching answer question image is found based on a user-uploaded question image, that answer question image is marked as having a corresponding user-uploaded image. During marking, only answer question images with corresponding user-uploaded images can be marked, or different identifiers can be used for answer question images with and without corresponding user-uploaded images; alternatively, all answer question images in the question bank can be initially marked as not having a corresponding user-uploaded image, and when an answer question image is successfully matched with a user-uploaded image, the mark of that answer question image is updated to have a corresponding user-uploaded image. The following explains how to determine the associated target uploaded image in both cases of existing and non-existent user-uploaded images.

[0071] As an optional implementation, when obtaining the target uploaded image associated with the answer question image, if there is no corresponding user uploaded image for the answer question image, the question area image can be cropped from the answer question image first, and the first size information of the question area image can be obtained. Then, based on the first size information, the target answer question image with the smallest difference between the size information of the question area contained in the candidate answer question images with corresponding user uploaded images is determined. The first uploaded image corresponding to the target answer question image is determined as the target uploaded image associated with the answer question image.

[0072] The first size information may include at least one of the following: the width and height of the question area, the aspect ratio of the question area image, and the area of ​​the question area image.

[0073] In this embodiment, the answer question image with a corresponding user-uploaded image can be used as a candidate answer question image. Based on the size information of the question region in the answer question image to be indexed (referred to as the first size information for easy distinction), the difference between the size information of the question region in the candidate answer question image and the first size information is calculated, and the candidate answer question image with the smallest difference is selected as the target answer question image. If there is more than one candidate answer question image with the smallest difference, it can be randomly selected as the target answer question image. Since the target answer question image has a corresponding user-uploaded image, the user-uploaded image corresponding to the target answer question image (referred to as the first uploaded image for easy distinction) can be obtained, and this first uploaded image is determined as the target uploaded image associated with the answer question image. When calculating the difference between size information, if there are multiple size information items, the difference between each size information item can be calculated, and then the differences can be summed, averaged, or weighted summed to determine the final result as the difference between the size information items. For example, if the first size information includes aspect ratio and area, then the aspect ratio and area of ​​the question region in each candidate answer question image can be obtained. A first difference between the aspect ratio of the question region in each candidate answer question image and the aspect ratio in the first size information, and a second difference between the area and the area in the first size information, can be calculated. For the same candidate answer question image, the average of the first and second differences can be calculated, and this average is determined as the difference between the size information of the question region in that candidate answer question image and the first size information. Then, the magnitudes of each difference are compared, and the candidate answer question image with the smallest difference is selected as the target answer question image.

[0074] As an optional implementation, when obtaining the target uploaded image associated with the answer question image, if there is a corresponding user uploaded image (referred to as the second uploaded image for easy distinction), the second uploaded image can be identified as the target uploaded image associated with the answer question image.

[0075] In this embodiment, there is one user-uploaded image corresponding to the answer question image. When multiple user-uploaded images match the same answer question image in the historical search, the user-uploaded image with the best quality can be selected by manual screening and bound to the answer question image as the user-uploaded image corresponding to that answer question image.

[0076] Step 202: Obtain the target title identifier of the target uploaded image.

[0077] In this embodiment of the disclosure, each user-uploaded image corresponds to a unique title identifier.

[0078] For example, when the answer question image is successfully matched for the first time, a unique identifier can be assigned to the user-uploaded image that is currently matched with it. If another user-uploaded image is successfully matched with the answer question image in the future, the user-uploaded image will not be recorded, the question identifier of the user-uploaded image corresponding to the answer question image will not be changed, and the question identifier assigned to the user-uploaded image when the answer question image was successfully matched for the first time will still be retained.

[0079] For example, a unique question identifier can be pre-assigned to each answer question image in the question bank. When an answer question image is successfully matched with a user-uploaded image, the user-uploaded image is taken as the user-uploaded image corresponding to the answer question image, and the question identifier of the answer question image is determined as the question identifier of the user-uploaded image.

[0080] In this embodiment of the disclosure, after determining the target uploaded image, the title identifier corresponding to the target uploaded image, i.e., the target title identifier, can be obtained.

[0081] Step 203: Based on the target title identifier, obtain the first moiré background image corresponding to the target title identifier from the moiré background image library.

[0082] As mentioned earlier, when generating a moiré pattern background image based on a user-uploaded image, the generated moiré pattern background image can be marked with the title identifier of the corresponding user-uploaded image, so that the corresponding moiré pattern background image can be found based on the title identifier of the user-uploaded image. Therefore, in this embodiment, after obtaining the target title identifier of the target uploaded image, a search can be performed in the moiré pattern background image library based on the target title identifier to find the moiré pattern background image corresponding to the target title identifier (for ease of distinction, it is referred to as the first moiré pattern background image).

[0083] Step 204: Determine the first moiré background image as the target moiré background image corresponding to the answer question image.

[0084] In this embodiment of the disclosure, after obtaining the first moiré background image corresponding to the target question identifier from the moiré background image library, the first moiré background image can be determined as the target moiré background image corresponding to the answer question image.

[0085] Then, the answer question image can be fused with the target moiré background image to obtain the simulated question image.

[0086] Furthermore, in one alternative embodiment of this disclosure, such as Figure 2 As shown, the specific process of obtaining the simulation problem image is described in steps 205-208 below.

[0087] Step 205: Crop the question area image from the answer question image.

[0088] In this embodiment of the disclosure, for the obtained answer question image, the location of the question area in the answer question image can be identified first, and then the question area image can be cropped from the answer question image according to the location.

[0089] For example, a pre-trained question detection model can be used to detect the question region in the answer question image. The question detection model outputs the position information of the detection box corresponding to the question region. Based on the position information, the question region can be cropped from the answer question image to obtain the question region image.

[0090] For example, since the background of the answer question image in the question bank is pure white, the question area can be determined by binarization. Specifically, the answer question image is binarized to separate the foreground pixels and background pixels. Then, the top, bottom, left, and right boundaries of all foreground pixels are obtained, which gives the location of the question area in the answer question image. Based on this location, the question area is cropped from the answer question image to obtain the question area image.

[0091] Step 206: Obtain the second size information of the title area in the target uploaded image.

[0092] The second size information may include the width and height of the question area.

[0093] For example, each user-uploaded image can be pre-annotated with question region information, including the width and height of the question region. For instance, the question region in the user-uploaded image can be manually outlined, closely following the top, bottom, left, and right boundaries of the question, and the width and height of the bounding box can be used as the question region information for that user-uploaded image and annotated. Alternatively, a pre-trained question detection model can detect the question region in the user-uploaded image, and the position information of the detection box corresponding to the question region output by the question detection model can be determined as the question region information for the user-uploaded image and annotated. To ensure the accuracy of the question region information, the output of the question detection model can also be manually corrected.

[0094] Therefore, in this embodiment of the disclosure, the title area information corresponding to the target uploaded image can be obtained as the second size information.

[0095] Step 207: Based on the second size information, scale the title area image and the target moiré background image to obtain scaled title area image and scaled moiré background image.

[0096] In this embodiment of the disclosure, after obtaining the second size information of the question area in the target uploaded image, the cropped question area image can be scaled to the same size as the second size information to obtain the scaled question area image, and the determined target moiré background image can also be scaled to the same size as the second size information to obtain the scaled moiré background image, thereby obtaining the scaled moiré background image and the scaled question area image of the same size, which facilitates image fusion.

[0097] Among them, commonly used image scaling techniques can be used to scale the image of the question area and the background image of the target moiré pattern. This disclosure does not limit the specific scaling methods.

[0098] It should be noted that scaling the target moiré background image to match the second size information is only an example and should not be considered a limitation of this disclosure. Other methods can also be used to process the target moiré background image, such as cropping an image block of the same size and second size information from the target moiré background image for fusion with the scaled title area image.

[0099] Step 208: Fuse the scaled question area image and the scaled moiré background image to generate a simulated question image.

[0100] In this embodiment of the disclosure, after obtaining the scaled question area image and the scaled moiré background image, the two can be fused to obtain a simulated question image carrying moiré patterns.

[0101] As an optional implementation, the first weight corresponding to the scaled question area image and the second weight corresponding to the scaled moiré background image can be obtained first, wherein the first weight is greater than the second weight and the sum of the first weight and the second weight is 1. Then, the scaled question area image and the scaled moiré background image are weighted and summed according to the first weight and the second weight to obtain the simulated question image.

[0102] In other words, the simulated problem image = first weight * scaled problem area image + second weight * scaled moiré background image.

[0103] In this embodiment of the disclosure, by setting the first weight corresponding to the scaled question area image to be greater than the second weight corresponding to the scaled moiré background image, the scaled question area image and the scaled moiré background image are weighted and summed to obtain the simulated question image, so that the generated simulated question image can retain more information of the question area and improve the realism of the simulated question image.

[0104] Step 209: Extract features from the simulated problem image to obtain the feature vector corresponding to the simulated problem image.

[0105] Step 210: Determine the feature vector as the question index corresponding to the answer question image.

[0106] It should be noted that, in the embodiments of this disclosure, the explanation of steps 209-210 can be found in the description of steps 104-105 in the foregoing embodiments, and will not be repeated here.

[0107] The question index acquisition method of this embodiment obtains a target uploaded image associated with the answer question image and a target question identifier of the target uploaded image. Based on the target question identifier, a first moiré background image corresponding to the target question identifier is obtained from a moiré background image library. The first moiré background image is determined as the target moiré background image corresponding to the answer question image. The answer question image and the target moiré background image are fused to generate a simulated question image. The features of the simulated question image are then extracted as the question index of the answer question image. Since the target moiré background image is determined by the user-uploaded image, fusing the target moiré background image with the answer question image can obtain a more realistic simulated question image, thereby reducing the feature difference between the question index of the user-uploaded image and the answer question image and improving the accuracy of question search.

[0108] Figure 3 A flowchart of a method for obtaining a question index according to yet another exemplary embodiment of this disclosure is shown, such as... Figure 3 As shown, the method for obtaining the question index may include the following steps:

[0109] Step 300: Obtain the image of the answer question with the index to be obtained.

[0110] Step 301: Obtain the first width and first height of the question area in the answer question image.

[0111] For example, a pre-trained question checking model can be used to detect questions in the answer question image, obtain the position information of the detection box in the question region of the answer question image (e.g., the coordinate information of the four vertices of the detection box), determine the width and height of the detection box based on the position information, and use the width of the detection box as the first width of the question region in the answer question image, and use the height of the detection box as the first width of the question region in the answer question image.

[0112] For example, the answer question image can be binarized to obtain the foreground pixels and background pixels of the answer question image. The length of the upper or lower boundary of the foreground pixel (e.g., the larger of the two values) can be obtained as the first width, and the length of the left or right boundary of the foreground pixel (e.g., the larger of the two values) can be obtained as the first height.

[0113] Step 302: Based on the first width and the first height, select candidate moiré background images from the moiré background image library, wherein the second width of the candidate moiré background image is greater than the first width, and the second height of the candidate moiré background image is greater than the first height.

[0114] In this embodiment of the disclosure, after obtaining the first width and first height of the question area in the answer question image, candidate moiré background images can be selected from the moiré background image library based on the first width and first height. Specifically, the width and height of each moiré background image in the moiré background image library can be obtained, and the width and height of each moiré background image can be compared with the first width and first height respectively. Moiré background images with a width greater than the first width and a height greater than the first height are determined as candidate moiré background images.

[0115] Step 303: Randomly select one of the candidate moiré background images as the target moiré background image corresponding to the answer question image.

[0116] In this embodiment of the disclosure, after determining the candidate moiré background images, one can be randomly selected from the candidate moiré background images as the target moiré background image corresponding to the answer question image.

[0117] Then, the answer question image can be fused with the target moiré background image to obtain the simulated question image.

[0118] Furthermore, in one alternative embodiment of this disclosure, such as Figure 3 As shown, the specific process of obtaining the simulation problem image is described in steps 304-308 below.

[0119] Step 304: Crop the question area from the answer question image to obtain the question area image.

[0120] For example, a pre-trained question detection model can be used to detect the question region in the answer question image. The question detection model outputs the position information of the detection box corresponding to the question region. Based on the position information, the question region can be cropped from the answer question image to obtain the question region image.

[0121] For example, since the background of the answer question image in the question bank is pure white, the question area can be determined by binarization. Specifically, the answer question image is binarized to separate the foreground pixels and background pixels. Then, the top, bottom, left, and right boundaries of all foreground pixels are obtained, which gives the location of the question area in the answer question image. Based on this location, the question area is cropped from the answer question image to obtain the question area image.

[0122] Step 305: Determine the first aspect ratio of the title area image based on the first width and the first height.

[0123] In this embodiment of the disclosure, the ratio of the first width to the first height can be calculated to obtain the aspect ratio of the title area image, which is referred to as the first aspect ratio for ease of use.

[0124] Step 306: Based on the first aspect ratio, determine the user-uploaded image from the set of user-uploaded images that has the smallest difference between the aspect ratio of the question area and the first aspect ratio, and use it as the target image.

[0125] In this set of user-uploaded images, each user-uploaded image is pre-labeled with question area information, including but not limited to the width and height of the question area. The ratio of the width and height can be calculated to obtain the aspect ratio of the question area in each user-uploaded image, and then the target image can be determined based on the aspect ratio.

[0126] As an optional implementation, the difference between the aspect ratio of the title area of ​​each user-uploaded image in the user-uploaded image set and the first aspect ratio can be calculated, and the user-uploaded image with the smallest difference from the first aspect ratio can be selected as the target image.

[0127] As an optional implementation, the aspect ratio variation range of the questions can be obtained, and a random number can be selected from this range as the aspect ratio adjustment range. The sum of the adjustment range and the first aspect ratio is calculated to obtain the second aspect ratio. Then, based on the second aspect ratio, the user-uploaded image with the smallest difference between the aspect ratio of the question area and the second aspect ratio is determined from the set of user-uploaded images as the target image. Thus, the aspect ratio of the question area in the answer question image can be corrected based on a random number from the aspect ratio variation range, and the target image can be determined based on the corrected aspect ratio, making the aspect ratio of the question area closer to that of the question area in the user-uploaded image, thereby improving the accuracy of recalling user-uploaded images.

[0128] The area where the aspect ratio of the question changes can be preset or determined based on the difference in aspect ratio between the question area in the user-uploaded image and the corresponding answer question image.

[0129] As an example, when obtaining the range of aspect ratio changes for questions, multiple question image pairs can be obtained. Each question image pair includes an answer question image and a user-uploaded image corresponding to that answer question image. The number of user-uploaded images can be at least one. Next, for each question image pair, the aspect ratio of the first question region of the answer question image and the aspect ratio of the second question region of each corresponding user-uploaded image can be obtained. The difference between the aspect ratios of the first and second question regions is calculated. If the aspect ratio of the first question region is less than that of the second question region, the difference is negative. For example, assuming the aspect ratio of the first question region of the answer question image is 2.5 and the aspect ratio of the second question region of the corresponding user-uploaded image is 2.6, the difference is 0.1; assuming the aspect ratio of the first question region of the answer question image is 2.5 and the aspect ratio of the second question region of the corresponding user-uploaded image is 2.4, the difference is -0.1. This allows us to obtain multiple difference values, which can then be compared for all aspect ratio differences calculated for each image pair of the question. From these differences, we can determine the minimum and maximum aspect ratio differences. Based on these differences, we can define the range of aspect ratio variation for each question, where the minimum difference is the lower limit and the maximum difference is the upper limit. For example, assuming the maximum difference is 0.2 and the minimum difference is -0.3, the range of aspect ratio variation for the question can be represented as (-0.3, 0.2).

[0130] Step 307: Based on the third width and third height of the question region in the target image, the question region image is scaled to obtain a target question region image with the same width and the same height as the third width.

[0131] In this embodiment of the disclosure, after determining the target image, the third width and third height of the question region in the target image can be obtained. The third width and third height can be manually pre-annotated on the user-uploaded image, or they can be determined based on the detection results of the question region detection on the target image; this disclosure does not impose any limitations on this. Next, based on the obtained third width and third height, the question region image can be scaled to obtain a target question region image with the same width and height as the third width.

[0132] Among them, commonly used image scaling methods can be used to scale the image of the question area to a size where the width and height are the same as the third width and the third height. This disclosure does not limit the specific scaling methods.

[0133] Step 308: After aligning the first center point of the target question area image with the second center point of the target moiré background image, the target question area image and the target moiré background image are fused to generate a simulated question image.

[0134] As an optional implementation, the first center point of the target question area image can be aligned with the second center point of the target moiré background image. Then, the background pixels in the target moiré background image corresponding to the question pixels in the target question area image can be determined. By calculating the mean and weighted sum of the background pixels and the corresponding question pixels, the background pixels and question pixels can be merged. Other background pixels in the target moiré background image retain their original pixel values ​​to obtain the simulated question image.

[0135] As an optional implementation, the first center point of the target question region image can be aligned with the second center point of the target moiré background image. Then, the target question region image and the target moiré background image are fused to obtain an initial simulation image. During fusion, the background pixels in the target moiré background image corresponding to the question pixels in the target question region image can be determined first. The background pixels are then fused with the question pixels by calculating the mean or weighted sum of the background pixels and their corresponding question pixels. Other background pixels in the target moiré background image retain their original pixel values, resulting in the initial simulation image. Next, the target upper boundary value, target lower boundary value, target left boundary value, and target right boundary value of the non-question region can be obtained. Based on these values, the initial simulation image is cropped to obtain the simulated question image. In the simulated question image, the height of the top edge of the non-question region is no greater than the target upper boundary value, the height of the bottom edge is no greater than the target lower boundary value, the width of the left edge is no greater than the target left boundary value, and the width of the right edge is no greater than the target right boundary value.

[0136] Among them, the target upper boundary value, target lower boundary value, target left boundary value and target right boundary value refer to the distance between each boundary of the question area in the image and each boundary of the corresponding image. The specific value of each boundary can be preset according to actual needs, or it can be determined according to the distance between each boundary of the question area in the user-uploaded image and each boundary of the user-uploaded image. This disclosure does not impose any restrictions on this.

[0137] In one optional embodiment of this disclosure, when obtaining the target upper boundary value, target lower boundary value, target left boundary value, and target right boundary value of the non-topic area, multiple user-uploaded images can be obtained first, and then the upper height, lower height, left width, and right width of the non-topic area of ​​each user-uploaded image can be obtained. The specific areas corresponding to the upper height, lower height, left width, and right width are as follows: Figure 4As shown. Thus, multiple top heights, multiple bottom heights, multiple left widths, and multiple right widths are obtained. Furthermore, the maximum top height can be determined from the multiple top heights as the target top boundary value, the maximum bottom height can be determined from the multiple bottom heights as the target bottom boundary value, the maximum left width can be determined from the multiple left widths as the target left boundary value, and the maximum right width can be determined from the multiple right widths as the target right boundary value.

[0138] Step 309: Extract features from the simulated problem image to obtain the feature vector corresponding to the simulated problem image.

[0139] Step 310: Determine the feature vector as the question index corresponding to the answer question image.

[0140] It should be noted that, in the embodiments of this disclosure, the explanation of steps 309-310 can be found in the description of steps 104-105 in the foregoing embodiments, and will not be repeated here.

[0141] The question index acquisition method of this disclosure obtains the first width and first height of the question region in the answer question image, and selects candidate moiré background images from a moiré background image library based on the first width and first height. The second width and second height of the candidate moiré background images are greater than the first width and the first height are greater than the first height. Then, one of the candidate moiré background images is randomly selected as the target moiré background image corresponding to the answer question image. The target moiré background image is then fused with the answer question image to obtain a simulated question image. The features of the simulated question image are extracted as the question index of the answer question image. This reduces the feature difference between the features of the user-uploaded image and the question index of the answer question image, reduces the deviation of the feature matching results, and thus improves the accuracy of question search.

[0142] In one optional embodiment of this disclosure, before feature extraction, the obtained simulation problem image can be subjected to image enhancement operations such as random rotation, random cropping, and random addition of backlighting, and then feature extraction can be performed on the enhanced image.

[0143] This exemplary embodiment also provides a question index acquisition device. Figure 5 A schematic block diagram of a title index acquisition apparatus according to an exemplary embodiment of the present disclosure is shown, such as Figure 5 As shown, the question index acquisition device 50 includes: an answer question acquisition module 510, a background image acquisition module 520, an image fusion module 530, a feature extraction module 540, and an index determination module 550.

[0144] Among them, the answer question acquisition module 510 is used to acquire the answer question image of the index to be acquired;

[0145] Background image acquisition module 520 is used to acquire a target moiré background image corresponding to the answer question image from a pre-built moiré background image library;

[0146] Image fusion module 530 is used to fuse the answer question image with the target moiré background image to generate a simulated question image;

[0147] The feature extraction module 540 is used to extract features from the simulated problem image to obtain the feature vector corresponding to the simulated problem image.

[0148] The index determination module 550 is used to determine the feature vector as the question index corresponding to the answer question image.

[0149] Optionally, the background image acquisition module 520 includes:

[0150] The first acquisition unit is used to acquire the target uploaded image associated with the answer question image;

[0151] The second acquisition unit is used to acquire the target title identifier of the target uploaded image;

[0152] The third acquisition unit is used to acquire a first moiré background image corresponding to the target question identifier from the moiré background image library according to the target question identifier;

[0153] The first determining unit is used to determine the first moiré background image as the target moiré background image corresponding to the answer question image.

[0154] Optionally, the first acquisition unit is further configured to:

[0155] In response to the absence of a corresponding user-uploaded image for the answer question image, the question area image is cropped from the answer question image;

[0156] Obtain the first size information of the image of the question area;

[0157] Based on the first size information, from the candidate answer question images that have corresponding user-uploaded images, the target answer question image with the smallest difference between the size information of the question area contained therein and the first size information is determined;

[0158] The first uploaded image corresponding to the target answer question image is determined as the target uploaded image associated with the answer question image;

[0159] In response to the existence of a corresponding second uploaded image for the answer question image, the second uploaded image is determined as the target uploaded image associated with the answer question image.

[0160] Further, optionally, the image fusion module 530 includes:

[0161] The first cropping unit is used to crop out the question area image from the answer question image;

[0162] The fourth acquisition unit is used to acquire the second size information of the question area in the target uploaded image;

[0163] The first scaling unit is used to scale the title area image and the target moiré background image based on the second size information to obtain the scaled title area image and the scaled moiré background image.

[0164] The first fusion unit is used to fuse the scaled question area image and the scaled moiré background image to generate a simulated question image.

[0165] Optionally, the first fusion unit is further configured to:

[0166] Obtain the first weight corresponding to the scaled question area image and the second weight corresponding to the scaled moiré background image, wherein the first weight is greater than the second weight;

[0167] Based on the first weight and the second weight, the scaled question area image and the scaled moiré background image are weighted and summed to obtain the simulated question image.

[0168] Optionally, the background image acquisition module 520 includes:

[0169] The fifth acquisition unit is used to acquire the first width and first height of the question area in the answer question image;

[0170] A filtering unit is configured to filter candidate moiré background images from the moiré background image library based on the first width and the first height, wherein the second width of the candidate moiré background image is greater than the first width, and the second height of the candidate moiré background image is greater than the first height;

[0171] The selection unit is used to randomly select one of the candidate moiré background images as the target moiré background image corresponding to the answer question image.

[0172] Further, optionally, the image fusion module 530 includes:

[0173] The second cropping unit is used to crop the question region from the answer question image to obtain a question region image;

[0174] The second determining unit is configured to determine the first aspect ratio of the question area image based on the first width and the first height;

[0175] The third determining unit is used to determine, based on the first aspect ratio, the user-uploaded image with the smallest difference between the aspect ratio of the question area and the first aspect ratio from the set of user-uploaded images, and use it as the target image;

[0176] The second scaling unit is used to scale the question region image based on the third width and third height of the question region in the target image to obtain a target question region image with the same width and the same height as the third width.

[0177] The second fusion unit is used to align the first center point of the target question area image with the second center point of the target moiré background image, and then fuse the target question area image and the target moiré background image to generate a simulated question image.

[0178] Optionally, the third determining unit is further configured to:

[0179] Obtain the range of changes in the aspect ratio of the question.

[0180] A number is randomly selected from the range of aspect ratio changes in the given question as the aspect ratio adjustment range;

[0181] The second aspect ratio is obtained by summing the aspect ratio adjustment range with the first aspect ratio.

[0182] Based on the second aspect ratio, the user-uploaded image with the smallest difference between the aspect ratio of the question area and the second aspect ratio is determined from the set of user-uploaded images and used as the target image.

[0183] Optionally, the question index acquisition device 50 further includes:

[0184] The first statistical module is used to acquire multiple question image pairs, wherein each question image pair includes an answer question image and a user-uploaded image corresponding to the answer question image; acquire the aspect ratio difference between the first question region aspect ratio of the answer question image and the second question region aspect ratio of the user-uploaded image in each question image pair; determine the minimum aspect ratio difference and the maximum aspect ratio difference from the aspect ratio differences corresponding to the multiple question image pairs respectively; and determine the range of variation of the question aspect ratio based on the minimum aspect ratio difference and the maximum aspect ratio difference.

[0185] Optionally, the second fusion unit is further configured to:

[0186] The target question area image and the target moiré background image are fused to obtain an initial simulation image;

[0187] Obtain the target's upper boundary value, lower boundary value, left boundary value, and right boundary value in the non-question area;

[0188] The initial simulation image is cropped based on the target upper boundary value, the target lower boundary value, the target left boundary value, and the target right boundary value to obtain the simulation question image. In the simulation question image, the height of the upper edge of the non-question area is not greater than the target upper boundary value, the height of the lower edge is not greater than the target lower boundary value, the width on the left is not greater than the target left boundary value, and the width on the right is not greater than the target right boundary value.

[0189] Optionally, the question index acquisition device 50 further includes:

[0190] The second statistics module is used to acquire multiple user-uploaded images; acquire the top height, bottom height, left width, and right width of the non-title area of ​​each user-uploaded image; determine the maximum top height from the multiple top heights as the target top boundary value; determine the maximum bottom height from the multiple bottom heights as the target bottom boundary value; determine the maximum left width from the multiple left widths as the target left boundary value; and determine the maximum right width from the multiple right widths as the target right boundary value.

[0191] The question index acquisition device provided in this disclosure can execute any question index acquisition method applicable to electronic devices provided in this disclosure, and has the corresponding functional modules and beneficial effects of the method execution. Content not described in detail in the device embodiments of this disclosure can be referred to the description in any method embodiment of this disclosure.

[0192] Exemplary embodiments of this disclosure also provide an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, the computer program, when executed by the at least one processor, causing the electronic device to perform a title index acquisition method according to embodiments of this disclosure.

[0193] Exemplary embodiments of this disclosure also provide a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform a title index acquisition method according to embodiments of this disclosure.

[0194] Exemplary embodiments of this disclosure also provide a computer program product, including a computer program, wherein, when executed by a computer's processor, the computer program is used to cause the computer to perform a title index acquisition method according to embodiments of this disclosure.

[0195] refer to Figure 6 The present invention describes a structural block diagram of an electronic device 1100 that can serve as a server or client of the present disclosure, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0196] like Figure 6 As shown, the electronic device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1102 or a computer program loaded from a storage unit 1108 into a random access memory (RAM) 1103. The RAM 1103 may also store various programs and data required for the operation of the device 1100. The computing unit 1101, ROM 1102, and RAM 1103 are interconnected via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0197] Multiple components in electronic device 1100 are connected to I / O interface 1105, including: input unit 1106, output unit 1107, storage unit 1108, and communication unit 1109. Input unit 1106 can be any type of device capable of inputting information to electronic device 1100. Input unit 1106 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of electronic device. Output unit 1107 can be any type of device capable of presenting information and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 1108 may include, but is not limited to, disk and optical disk. Communication unit 1109 allows electronic device 1100 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth™ devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0198] The computing unit 1101 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1101 performs the various methods and processes described above. For example, in some embodiments, the question index retrieval method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1108. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 1100 via ROM 1102 and / or communication unit 1109. In some embodiments, the computing unit 1101 can be configured to perform the question index retrieval method by any other suitable means (e.g., by means of firmware).

[0199] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0200] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0201] As used in this disclosure, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0202] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0203] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0204] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

Claims

1. A method for obtaining a question index, wherein, The method includes: Retrieve the image of the answer question for the index to be retrieved; Obtain the target moiré background image corresponding to the answer question image from a pre-built moiré background image library; The answer question image is fused with the target moiré background image to generate a simulated question image; Feature extraction is performed on the simulated problem image to obtain the feature vector corresponding to the simulated problem image; The feature vector is determined as the question index corresponding to the answer question image; The step of obtaining the target moiré background image corresponding to the answer question image from a pre-built moiré background image library includes: Obtain the first width and first height of the question region in the answer question image; Based on the first width and the first height, candidate moiré background images are selected from the moiré background image library, wherein the second width of the candidate moiré background image is greater than the first width, and the second height of the candidate moiré background image is greater than the first height; Randomly select one of the candidate moiré background images as the target moiré background image corresponding to the answer question image.

2. The question index acquisition method as described in claim 1, wherein, The step of obtaining the target moiré background image corresponding to the answer question image from a pre-built moiré background image library includes: Obtain the target uploaded image associated with the answer question image; Obtain the target title identifier of the target uploaded image; Based on the target question identifier, obtain the first moiré background image corresponding to the target question identifier from the moiré background image library; The first moiré background image is determined as the target moiré background image corresponding to the answer question image.

3. The question index acquisition method as described in claim 2, wherein, The step of obtaining the target uploaded image associated with the answer question image includes: In response to the absence of a corresponding user-uploaded image for the answer question image, the question area image is cropped from the answer question image; Obtain the first size information of the image of the question area; Based on the first size information, from the candidate answer question images that have corresponding user-uploaded images, the target answer question image with the smallest difference between the size information of the question area contained therein and the first size information is determined; The first uploaded image corresponding to the target answer question image is determined as the target uploaded image associated with the answer question image; In response to the existence of a corresponding second uploaded image for the answer question image, the second uploaded image is determined as the target uploaded image associated with the answer question image.

4. The question index acquisition method as described in claim 2, wherein, The step of fusing the answer question image with the target moiré background image to generate a simulated question image includes: The question area image is cropped from the answer question image; Obtain the second size information of the question area in the target uploaded image; Based on the second size information, the question area image and the target moiré background image are scaled to obtain scaled question area image and scaled moiré background image. The scaled question area image and the scaled moiré background image are fused to generate a simulated question image; The step of fusing the scaled question area image and the scaled moiré background image to generate a simulated question image includes: Obtain the first weight corresponding to the scaled question area image and the second weight corresponding to the scaled moiré background image, wherein the first weight is greater than the second weight; Based on the first weight and the second weight, the scaled question area image and the scaled moiré background image are weighted and summed to obtain the simulated question image.

5. The question index acquisition method as described in claim 1, wherein, The step of fusing the answer question image with the target moiré background image to generate a simulated question image includes: The question area is cropped from the answer question image to obtain the question area image; Based on the first width and the first height, determine the first aspect ratio of the question area image; Based on the first aspect ratio, the user-uploaded image with the smallest difference between the aspect ratio of the question area and the first aspect ratio is determined from the set of user-uploaded images and used as the target image. Based on the third width and third height of the question region in the target image, the question region image is scaled to obtain a target question region image with the same width and the same height as the third width. After aligning the first center point of the target question area image with the second center point of the target moiré background image, the target question area image and the target moiré background image are fused to generate a simulated question image.

6. The question index acquisition method as described in claim 5, wherein, The step of determining the user-uploaded image with the smallest difference between the aspect ratio of the question area and the first aspect ratio from the set of user-uploaded images, based on the first aspect ratio, as the target image, includes: Obtain the range of changes in the aspect ratio of the question. A number is randomly selected from the range of aspect ratio changes in the given question as the aspect ratio adjustment range; The second aspect ratio is obtained by summing the aspect ratio adjustment range with the first aspect ratio. Based on the second aspect ratio, the user-uploaded image with the smallest difference between the aspect ratio of the question area and the second aspect ratio is determined from the set of user-uploaded images and used as the target image. The process of obtaining the aspect ratio variation range of the question includes: Multiple question image pairs are obtained, wherein each question image pair includes an answer question image and a user-uploaded image corresponding to the answer question image; The aspect ratio difference between the first question region aspect ratio of the answer question image and the second question region aspect ratio of the user-uploaded image is obtained for each question image pair; From the aspect ratio differences corresponding to the multiple image pairs, determine the minimum and maximum aspect ratio differences; Based on the minimum aspect ratio difference and the maximum aspect ratio difference, the range of aspect ratio variation of the title is determined.

7. The question index acquisition method as described in claim 5, wherein, The process of fusing the target question region image and the target moiré background image to generate a simulated question image includes: The target question area image and the target moiré background image are fused to obtain an initial simulation image; Obtain the target's upper boundary value, lower boundary value, left boundary value, and right boundary value in the non-question area; The initial simulation image is cropped based on the target upper boundary value, the target lower boundary value, the target left boundary value, and the target right boundary value to obtain the simulation question image. In the simulation question image, the height of the upper edge of the non-question area is not greater than the target upper boundary value, the height of the lower edge is not greater than the target lower boundary value, the width on the left is not greater than the target left boundary value, and the width on the right is not greater than the target right boundary value. The step of obtaining the target's upper boundary value, lower boundary value, left boundary value, and right boundary value in the non-question area includes: Get images uploaded by multiple users; Obtain the top height, bottom height, left width, and right width of the non-title area of ​​each user-uploaded image from the plurality of user-uploaded images; The maximum upper height is determined from multiple upper heights and used as the target upper boundary value; The maximum lower boundary height is determined from multiple lower boundary heights and used as the target lower boundary value; The maximum left width among multiple left widths is determined as the target left boundary value; The maximum right width is determined from multiple right widths and used as the target right boundary value.

8. A question index acquisition device, wherein, The device includes: The answer / question retrieval module is used to retrieve the image of the answer / question for the index to be retrieved. The background image acquisition module is used to acquire the target moiré background image corresponding to the answer question image from a pre-built moiré background image library; An image fusion module is used to fuse the answer question image with the target moiré background image to generate a simulated question image; The feature extraction module is used to extract features from the simulated problem image to obtain the feature vector corresponding to the simulated problem image. An index determination module is used to determine the feature vector as the question index corresponding to the answer question image; The background image acquisition module is specifically used for: Obtain the first width and first height of the question region in the answer question image; Based on the first width and the first height, candidate moiré background images are selected from the moiré background image library, wherein the second width of the candidate moiré background image is greater than the first width, and the second height of the candidate moiré background image is greater than the first height; Randomly select one of the candidate moiré background images as the target moiré background image corresponding to the answer question image.

9. An electronic device, comprising: processor; as well as Stored program memory, The program includes instructions that, when executed by the processor, cause the processor to perform the question index acquisition method according to any one of claims 1-7.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to execute the question index acquisition method according to any one of claims 1-7.

Images