Title search method and device, equipment and storage medium

Abstract

The present disclosure provides a question search method, device and equipment and a storage medium, which are used for removing moire in a question image to improve the accuracy of test question recommendation. The method comprises the following steps: obtaining a moire question image; removing moire from the moire question image by using a moire filtering model to obtain a question image; extracting features of the question image to obtain a feature vector of the question image; and obtaining recommended test questions from a test question library based on the similarity between the feature vector of the question image and the feature vectors of the test questions in the test question library.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and particularly relates to a question searching method and device, equipment and a storage medium. BACKGROUND

[0002] In recent years, with the development of Internet education, more and more users use the test question bank to search for test questions to obtain the problem solving ideas or answers of the test questions. The most commonly used method is to search for test questions by taking pictures.

[0003] In the related art, a user can take a question image against the screen of an electronic device, and upload the image to a question answering system. After analyzing the image, the question answering system pushes the stems of a plurality of test questions, test question answers and test question analyses that are most similar to the question image to the user for reference. SUMMARY

[0004] According to an aspect of the present application, a question searching method is provided, which includes:

[0005] obtaining a moire question image;

[0006] removing moire from the moire question image by using a moire filtering model to obtain a question image, wherein the loss in the training stage of the moire filtering model is determined by a first sub-loss, a second sub-loss and a third sub-loss, the first sub-loss is determined based on a moire prediction image and a moire image label, the second sub-loss is determined based on a question prediction image and a question image label, and the third sub-loss is determined based on a moire question sample image and a moire question prediction image, the moire question prediction image being determined by the moire prediction image and the question prediction image;

[0007] performing feature extraction on the question image to obtain a feature vector of the question image;

[0008] obtaining recommended test questions from a test question bank based on the similarity between the feature vector of the question image and the feature vectors of the test questions in the test question bank.

[0009] According to another aspect of the present application, a question searching device is provided, which includes:

[0010] The processing module is configured to acquire a moire subject image, perform moire removal processing on the moire subject image by using a moire filtering model, and obtain a subject image, wherein a loss in a training stage of the moire filtering model is determined by a first sub-loss, a second sub-loss, and a third sub-loss, the first sub-loss is determined based on a moire image label and a moire predicted image, the second sub-loss is determined based on a subject image label and a subject predicted image, and the third sub-loss is determined based on a moire subject predicted image and a moire subject sample image.

[0011] The search module is configured to perform feature extraction on the subject image to obtain a feature vector of the subject image, and obtain recommended test questions from a test question library based on a similarity between the feature vector of the subject image and feature vectors of test questions in the test question library.

[0012] According to another aspect of the present disclosure, an electronic device is provided, characterized by comprising:

[0013] a processor; and

[0014] a memory storing a program;

[0015] The program includes instructions that, when executed by the processor, cause the processor to perform the method provided by the exemplary embodiments of the present disclosure.

[0016] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, characterized by storing computer instructions for causing a computer to perform the method provided by the exemplary embodiments of the present disclosure.

[0017] In one or more of the technical solutions provided in the embodiments of the present disclosure, a moire topic image is obtained, and a moire removal model is used to remove the moire in the moire topic image to obtain a topic image. Since the loss of the moire removal model in the training stage is determined by a first sub-loss, a second sub-loss, and a third sub-loss, the first sub-loss is determined based on a moire image label and a moire prediction image, the second sub-loss is determined based on a topic image label and a topic prediction image, and the third sub-loss is determined based on a moire topic prediction image and a moire topic sample image, the moire topic prediction image being determined by the moire prediction image and the topic prediction image. As can be seen, the supervision signal of the moire removal model used in the method of the exemplary embodiments of the present disclosure includes not only the topic image label and the moire image label, but also the moire topic prediction image obtained by reconstructing the moire prediction image and the topic prediction image, so that in the training process of the moire removal model, not only is it required that the moire removal model can separate the moire prediction image and the topic prediction image, but also it is required that the moire topic prediction image obtained by reconstructing the separated moire prediction image and topic prediction image can restore the moire topic sample image, so that the moire removal model can be trained under the supervision of the three kinds of supervision signals, the accuracy of the moire removal model is improved, and the effect of removing the moire in the moire topic image by the moire removal model is improved. Based on this, the moire in the moire topic image is removed by using the moire removal model, and the removal effect of the obtained topic image is better.

[0018] Then, the feature vector of the topic image is obtained by performing feature extraction on the obtained topic image. Since the moire removal effect of the topic image is good, when the similarity between the feature vector of the topic image and the feature vector of the test questions in the test question library is used to obtain recommended test questions from the test question library, the interference of the moire feature on the test question search is small, so that the obtained recommended test questions are more accurate. BRIEF DESCRIPTION OF DRAWINGS

[0019] In the following description of the exemplary embodiments in conjunction with the accompanying drawings, more details, features and advantages of the present disclosure are disclosed, in which:

[0020] Figure 1 A schematic diagram of an example system in which the various methods described herein can be implemented in accordance with exemplary embodiments of the present disclosure is shown;

[0021] Figure 2 A schematic flowchart of a topic search method in accordance with exemplary embodiments of the present disclosure is shown;

[0022] Figure 3 A schematic flowchart of a training method of a moire removal model in accordance with exemplary embodiments of the present disclosure is shown;

[0023] Figure 4 A structural schematic diagram of a moire filter model is shown according to an example embodiment of the present disclosure;

[0024] Figure 5A A structural schematic diagram of a moire filter model is shown according to an example embodiment of the present disclosure;

[0025] Figure 5B A structural schematic diagram of a first encoder is shown according to an example embodiment of the present disclosure;

[0026] Figure 5C A structural schematic diagram of a second encoder is shown according to an example embodiment of the present disclosure;

[0027] Figure 5D A structural schematic diagram of a third encoder is shown according to an example embodiment of the present disclosure;

[0028] Figure 5E A structural schematic diagram of a fourth encoder is shown according to an example embodiment of the present disclosure;

[0029] Figure 5F A structural schematic diagram of a fifth encoder is shown according to an example embodiment of the present disclosure;

[0030] Figure 6 A schematic flow chart of a method for obtaining a subject image is shown according to an example embodiment of the present disclosure;

[0031] Figure 7 A schematic flow chart of a method for obtaining a feature vector of a subject image is shown according to an example embodiment of the present disclosure;

[0032] Figure 8 A schematic flow chart of a method for subject search based on a feature vector is shown according to an example embodiment of the present disclosure;

[0033] Figure 9 A functional module schematic block diagram of a subject search device is shown according to an example embodiment of the present disclosure;

[0034] Figure 10 A schematic block diagram of a chip is shown according to an example embodiment of the present disclosure;

[0035] Figure 11 A structural block diagram of an example electronic device that can be used to implement embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0036] Embodiments of the present disclosure will be described below in greater detail with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be construed as being limited to the embodiments set forth herein, but rather, the embodiments are provided so that the present disclosure can be more thoroughly and completely understood. It should be understood that the drawings of the present disclosure are only for illustrative purposes and should not be construed as limiting the scope of protection of the present disclosure.

[0037] It should be understood that each step recited in the method embodiments of the present disclosure can be performed in different orders and / or in parallel. In addition, the method embodiments can include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0038] The term "comprising" and variations thereof as used herein are open-ended, that is, "comprising but not limited to." The term "based on" is "based, at least in part, on." The term "one embodiment" means "at least one embodiment." The term "another embodiment" means "at least one additional embodiment." The term "some embodiments" means "at least some embodiments." Related definitions are given throughout the description below. It should be noted that the concepts mentioned in the present disclosure are merely used for distinguishing different apparatuses, modules or units, and are not intended to limit the functions of the apparatuses, modules or units.

[0039] It should be noted that the terms "one", "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise explicitly stated in the context, it should be understood as "one or more".

[0040] The names of the messages or information exchanged between the plurality of apparatuses in the embodiments of the present disclosure are only for illustrative purposes, and are not intended to limit the scope of the messages or information.

[0041] Before introducing the embodiments of the present disclosure, first, the related terms involved in the embodiments of the present disclosure are explained as follows:

[0042] Moiré is a kind of high-frequency interference stripe that appears on the light-sensitive element of a digital camera or a scanner and the like, and is a kind of high-frequency irregular stripe that makes the picture appear colored.

[0043] Robust is the transliteration of Robust, which means strong and strong. It refers to the ability of a system to survive in abnormal and dangerous situations. For example, whether the computer software can not die and not crash in the case of input error, disk failure, network overload or intentional attack is the robustness of the software. The so-called "robustness" also refers to the characteristics of the control system to maintain certain performance under certain (structure, size) parameter perturbation.

[0044] Frame extraction is a process that simulates taking a photo every certain time interval and splicing it into a video by extracting a certain number of frames at intervals in a video (i.e. low-speed shooting). The biggest advantage of frame extraction is to save storage space.

[0045] MobileNet is a convolutional neural network applied to mobile and embedded devices proposed by Google in 2017. Its main application scenarios include smartphones, drones, robots, autonomous driving, augmented reality, etc.

[0046] ResNet is also called residual network. ResNet is built by residual blocks (Residual Building Block). The residual block is composed of multiple cascaded convolution layers and a shortcut connection. The output values of the two are added and then passed through the ReLU activation layer to get the output of the residual block. Multiple residual blocks can be connected in series to realize a deeper network.

[0047] EfficientNet model opens up a new direction in improving image recognition accuracy and enhancing performance. It adjusts the size of the input image, the number of network layers and the width of the three dimensions to optimize efficiency and accuracy.

[0048] In the question-answering system, users often upload images taken by pointing at the screen. These images have moire patterns, which greatly interfere with image recognition and search for illustrations, making the question-answering system perform very unstably and providing users with a poor experience.

[0049] Currently, the effect of filtering moire patterns in the illustration topic scene using traditional methods is not good. Traditional moire pattern filtering methods will cause some loss to geometric line drawing data. Among the methods of filtering moire patterns through models, since the general model is relatively large and has poor robustness, it is not suitable for the illustration topic scene.

[0050] To solve the above problems, the exemplary embodiments of the present disclosure provide a question searching method. The moire pattern in a moire pattern question image is removed by using a moire pattern filtering model with three kinds of supervision signals to obtain a question image, so that the removal effect of the moire pattern in the question image is better. Then, based on the similarity of the feature vector of the question image and the feature vector of the test questions in the test question library, the accuracy of the recommended test questions is better. It should be understood that the related question images involved in the question searching method of the exemplary embodiments of the present disclosure can be table question images, text question images, illustration question images, etc. in terms of their categories, and are not limited thereto. In terms of their sources, they can refer to various basic subject question images of schools, such as mathematics, Chinese, and various foreign language question images such as Japanese, English, and French. They can also be various science and engineering question images or liberal arts question images, and can also be various test question images, etc. and are not limited thereto. The moire pattern question image herein can be a question image with moire patterns.

[0051] Figure 1 A schematic diagram of an example system in which the various methods described herein can be implemented in accordance with the exemplary embodiments of the present disclosure is shown. As Figure 1 shown, the system 100 of the exemplary embodiments of the present disclosure can include a user device 110, a computing device 120, and a data storage system 130.

[0052] As Figure 1 shown, the user device 110 described above can communicate with the computing device 120 through a communication network. The communication network can be a wired communication network or a wireless communication network. The wired communication network can be a communication network based on power line carrier technology, and the wireless communication network can be a local wireless network or a wide area wireless network. The local wireless network can be a WIFI wireless network, a Zigbee wireless network, a mobile communication network, or a satellite communication network, etc.

[0053] As Figure 1As shown in the figure, the user device 110 can include a computer, a mobile phone, or an intelligent terminal such as an information processing center. The user device 110 can serve as an image acquisition end of the moire subject image and initiate a request to the computing device 120. The computing device 120 can be a cloud server, a network server, an application server, a management server, or other servers with data processing functions, to implement the method of the exemplary embodiments of the present disclosure. The server can be configured with a deep learning processor, which can be a neuron of a single-core deep learning processor (DLP-S) or a multi-core deep learning processor (DLP-M). The multi-core deep learning processor is a multi-core expansion based on the single-core deep learning processor. The multiple single-core deep learning processors are interconnected through a network-on-chip (Noc), multicast, inter-core synchronization, and other protocols for inter-core communication to complete deep learning tasks and subject image generation tasks.

[0054] As shown in the figure, Figure 1 The data storage system 130 can store a database of multiple reference feature vectors of the subject image. The database can be stored on the computing device 120 or on other network servers. The data storage system 130 can be separate from the computing device 120 or integrated into the computing device 120. The data storage system 130 can not only input data uploaded by the user device 110, but also store program instructions, neuron data, weight data, and the like. The neuron data and weight data can be trained data or untrained data, depending on the method implemented by the computing device 120. In addition, the data storage system 130 can also store the processing results obtained by the computing device 120 into the data storage system 130.

[0055] In actual applications, as shown in the figure, Figure 1 The user device 110 can have an image acquisition function, so that the user device 110 can not only initiate a request to the computing device 120 through an interactive interface, but also collect images during the training phase and send the images to the computing device 120 through the interactive interface. Based on this, the moire subject image used by the computing device 120 when implementing the training method can be obtained not only from the data storage system 130 but also from the user device 110 through the interactive interface.

[0056] As shown in the figure, Figure 1As shown, when the user uploads the moire subject image to the computing device 120 through the user device 110, the moire filtering model can remove the moire from the moire subject image to obtain a moire-removed subject image, and then determine the test question matched with the subject image in the test question library as the recommended test question through feature extraction and feature matching, and recommend it to the user to improve the user experience.

[0057] The subject search method of the exemplary embodiments of the present disclosure can be applied to a server or a chip in the server, and the subject search method of the exemplary embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0058] Figure 2 An exemplary flowchart of the subject search method of the exemplary embodiments of the present disclosure is shown. As shown, the subject search method comprises: Figure 2

[0059] Step 201: Obtain a moire subject image. It should be understood that the moire subject image involved in the exemplary embodiments of the present disclosure can be a to-be-processed moire subject image photographed by a user device and sent to a computing device, or a to-be-processed moire subject image stored in a data storage system, and then the to-be-processed moire subject image is obtained by the computing device and processed.

[0060] Step 202: Remove the moire from the moire subject image by using the moire filtering model to obtain a subject image. The loss of the moire filtering model in the training stage is determined by a first sub-loss, a second sub-loss and a third sub-loss, the first sub-loss is determined based on the moire prediction image and the moire image label, the second sub-loss is determined based on the subject prediction image and the subject image label, and the third sub-loss is determined based on the moire subject sample image and the moire subject prediction image, the moire subject prediction image is determined by the moire prediction image and the subject prediction image.

[0061] As can be seen from the above, the supervision signal of the moire filtering model used in the method of the exemplary embodiments of the present disclosure not only includes the subject image label and the moire image label, but also includes the moire subject prediction image obtained after the reconstruction of the moire prediction image and the subject prediction image, so that in the training process of the moire filtering model, not only is it required that the moire filtering model can separate the moire prediction image and the subject prediction image, but also it is required that the moire subject prediction image obtained by the reconstruction of the separated moire prediction image and the subject prediction image can restore the moire subject sample image, so that the moire filtering model can be trained under the supervision of the three kinds of supervision signals, the accuracy of the moire filtering model is improved, and the effect of removing the moire in the moire subject image by the moire filtering model is improved. Based on this, the moire filtering model is used to remove the moire from the moire subject image, and the removal effect of the obtained subject image is better.​

[0062] Step 203: Feature extraction is performed on the question image to obtain a feature vector of the question image. Since the moire removal effect of the question image obtained through the moire removal model is good, the interference of the moire feature vector in the feature vector of the question image obtained here is small.

[0063] In actual application, the features of the question image can be extracted using a deep learning method. For example, a feature extraction model can be trained using multiple question images as samples, and after the feature extraction model converges, the feature vector of the question image is extracted using the trained feature extraction model. It should be understood that the feature extraction model of the example embodiment of the present disclosure can be a MobileNetV3 model, which has high processing efficiency while achieving high accuracy, and has good effect in actual application. The feature extraction module can also be other excellent models, such as ResNet series models, EfficientNet series models, etc.

[0064] Step 204: Recommended test questions are obtained from the test question library based on the similarity of the feature vector of the question image and the feature vector of the test questions in the test question library. Since the moire in the question image is less, the interference of the moire on the search results is small when searching for test questions, so that the recommended test questions searched are more accurate, and can be the original question.

[0065] In actual application, after the user device sends the user-shot moire question image to be processed to the computing device, the computing device processes the moire question image to be processed using the moire filter-out model to obtain a moire-removed question image, and then inputs the moire-removed question image into the feature extraction model to extract the feature vector of the question image, and inputs the extracted feature vector into the test question library to obtain recommended test questions. The computing device pushes the recommended test questions to the user device.

[0066] Figure 3 A schematic flowchart of a training method of a moire filter-out model according to an example embodiment of the present disclosure is shown.

[0067] As shown in Figure 3 the training method of the moire filter-out model includes:

[0068] Step 301: Obtain a moire question sample image, a moire image label of the moire question sample image, and a question image label of the moire question sample image. It should be understood that the images involved in the example embodiment of the present disclosure can be images obtained by frame extraction processing of a shot video, or photos shot by a camera, etc., and are not limited thereto. The video can be a video shot by a camera or a camera with video recording function, etc.

[0069] For example, when the image is a video, a plurality of pure color background videos can be taken by a camera against the display screen as the moire image label, and then the plurality of pure color background videos are frame extracted to obtain a plurality of moire background images. It should be noted that, in order to simulate the user shooting scene as much as possible, the moire image label can be a moire image label containing environmental noise and / or lens noise. The color of the pure color background is not limited here, and the moire image of the pure color background can reduce the interference of the background factor on the image processing.

[0070] The environmental noise can be introduced into the moire image label by transforming the collection time or collection environment of the moire image label. When the environmental noise is introduced into the moire image label by transforming the collection time of the moire image label, the light brightness will be different due to different collection times, for example, the collection time of the moire image label can be morning, noon, afternoon, evening, etc., and is not limited thereto. When the environmental noise is introduced into the moire image label by transforming the collection environment of the moire image label, the light intensity will be different due to different collection environments, for example, the collection environment of the moire image label can be indoor, outdoor, sunny, cloudy, rainy, etc., and is not limited thereto.

[0071] The lens noise can be introduced into the moire image label by transforming the camera lens posture of the moire image label. The camera lens posture can include camera pitch angle, camera yaw angle, and camera roll angle, etc. By simulating one or more of these angles through camera simulation noise, the influence of camera lens shaking on the moire image label can be simulated.

[0072] In actual application, when the plurality of pure color background videos are frame extracted, a video editing software or a command line tool can be used to manually select the frames to be extracted in the video as the moire image label. An image processing library in a programming language can also be used to automatically select the frames to be extracted in the video as the moire image label.

[0073] The question image label can be a question image extracted from a question bank. It should be noted that, in order to reduce background interference, the question image can be a pure color background question image. It should be noted that the background color of the question image label and the moire image label is consistent, and the specific color can be selected according to the actual situation, which is not limited here. By synthesizing the question image label and the moire image label, a plurality of moire question sample images can be obtained.

[0074] Step 302: training the moire filter model based on the moire question sample image, the moire image label and the question image label, the moire filter model being used to obtain the moire predicted image and the question predicted image based on the moire question sample image.

[0075] On this basis, the loss of the moire filter model in the training stage is determined by the first sub-loss, the second sub-loss and the third sub-loss. The first sub-loss is determined based on the moire image label and the moire predicted image, the second sub-loss is determined based on the question image label and the question predicted image, and the third sub-loss is determined based on the moire question predicted image and the moire question sample image, the moire question predicted image being determined by the moire predicted image and the question predicted image.

[0076] As can be seen from the above, after the moire predicted image and the question predicted image are obtained by the moire filter model, not only can the moire filter model be trained by using the question image label and the moire image label as two kinds of supervision information, but also the moire question predicted image can be obtained by reconstructing the moire predicted image and the question predicted image, and the moire filter model can be trained by using the moire question predicted image as the third supervision information. By adding the third supervision signal, it is required not only that the moire filter model can separate the moire predicted image and the question predicted image, but also that the moire question predicted image reconstructed by the separated moire predicted image and the question predicted image can restore the moire question sample image, so that the moire filter model can be trained under the supervision of the three kinds of supervision signals, the accuracy of the moire filter model is improved, and the effect of removing the moire in the moire question image by the moire filter model is improved.

[0077] In actual application, since the first sub-loss can be determined by the moire image label and the moire predicted image, the first sub-loss can be represented as Fm=Loss(MImg,PMImg); in the formula, MImg is the moire image label, and PMImg is the moire predicted image output by the moire filter model.

[0078] Since the second sub-loss can be determined by the question image label and the question predicted image, the second sub-loss can be represented as Fq=Loss(QImg,PQImg); in the formula, QImg is the question image label, and PQImg is the question predicted image output by the moire filter model.

[0079] Since the third sub-loss can be determined by the moire question prediction image and the moire question sample image, the third sub-loss can be expressed as: Fmq=Loss(MQImg,PMQImg); wherein, MQImg is the moire question sample image, and PMQImg is the moire question prediction image, which can be determined by the question prediction image and the moire prediction image, i.e., PMQImg=PQImg+PMImg.

[0080] As a possible implementation, the moire question sample image can be determined by the question image label and the moire image label in a weighted manner. The moire question sample image can be expressed as: MQImg=W1*MImg+W2*QImg; wherein, MQImg is the moire question sample image, MImg is the moire image label, and QImg is the question image label.

[0081] For example, when synthesizing the moire question sample image by using the moire image label and the question image label, different weights W1 and W2 can be respectively given to the moire image label and the question image label in a weighted sum manner to obtain a moire question sample image close to the actual shooting effect of the user, so as to avoid the phenomenon that the question is not clear due to the excessive weight of the moire image label.

[0082] In some optional manners, in order to obtain a moire question sample image close to the actual shooting effect of the user and reduce the influence of the moire image label on the question clarity in the moire question sample image, the weight of the question image label can be set to be greater than the weight of the moire image label, i.e., W2>W1. For example, the weight W2 of the question image label can be set to be 0.5-0.9, the weight W1 of the moire image label can be set to be 0.1-0.5, and the sum of the weight of the question image label and the weight of the moire image label is 1, i.e., W1+W2=1.

[0083] For example, the weight of the question image label can be W2=0.8, and the weight of the moire image label can be W1=0.2, so that the sum of the weight of the question image label and the weight of the moire image label is W1+W2=0.2+0.8=1.

[0084] For another example, the weight of the question image label can be W2=0.5, and the weight of the moire image label can be W1=0.5, so that the sum of the weight of the question image label and the weight of the moire image label is W1+W2=0.5+0.5=1.

[0085] As a possible implementation, Figure 4 A structural schematic diagram of a moire filtering model according to an exemplary embodiment of the present disclosure is shown. As shown in FIG. 1, the moire filtering model can include a question image label generator 101, a moire image label generator 102, a question prediction image generator 103, a moire prediction image generator 104, a moire question sample image generator 105, and a loss function 106. Figure 4As shown, the above moire filter-out model can include an encoding unit 401, a decoding unit 402, and a feature fusion layer 403.

[0086] As shown, the encoding unit 401 is configured to extract an encoding feature map of the moire subject sample image. The decoding unit 402 is configured to obtain a moire prediction image and a subject prediction image based on the encoding feature map, so as to increase the number of supervision signals of the moire prediction model of the exemplary embodiments of the present disclosure, so that the prediction accuracy of the moire prediction model is better. Figure 4

[0087] As an example, the encoding unit can include N-level encoders in series, where N represents an integer greater than or equal to 2. As the level of the encoder increases, the network depth of the encoding unit is deeper, and the scale of the encoding feature map output by the encoding unit is smaller. The encoders can be residual block-based feature extractors or U-Net feature extractors based on a fully convolutional neural network architecture, but are not limited thereto.

[0088] As shown, in the training phase, the number of decoding units 402 included in the above moire filter-out model can be two, and the architectures of the two decoding units 402 are the same. For example, the decoding unit 402 can include a first decoding unit 4021 and a second decoding unit 4022, where the first decoding unit 4021 is configured to obtain a moire prediction image based on the moire subject sample image, and the second decoding unit 4022 is configured to obtain a subject prediction image based on the moire subject sample image. Each decoding unit can include M-level decoders in series, where M represents an integer greater than or equal to 2, and N-M=1. Figure 4 As an example, the decoders are configured to output decoding feature maps of different scales. The M-level decoders can be connected in series, and as the level of the decoder increases, the scale of the decoding feature map output by the decoder becomes larger and larger. The architecture of the decoder can be similar to that of the encoder. The decoder can be a residual block-based feature extractor or a U-Net feature extractor based on a fully convolutional neural network architecture, but is not limited thereto.

[0089] In order to learn as many local features and global features as possible, for the current decoding feature map, the same scale current encoding feature map and the current decoding feature map can be input into the same decoder, so that the decoder can fuse the same scale encoding feature map and decoding feature map, thereby obtaining a decoding feature map of the next level.

[0090] As shown, the feature fusion layer 403 is configured to fuse the decoding feature maps output by the two decoding units 402, so as to obtain a final decoding feature map.

[0091] Figure 4 ​​As shown, the feature fusion layer 403 is configured to determine the prediction image based on the multiple decoded feature maps greater than or equal to the preset scale in the decoded feature maps of different scales, so as to improve the efficiency of the moire filter model. The prediction image can include a moire prediction image and a topic prediction image. It should be understood that the multiple decoded feature maps greater than or equal to the preset scale herein account for a ratio of the number of decoded feature maps of different scales greater than 0 and less than 100%.

[0092] For example, when the moire filter model includes 7 decoders, since the deep decoders have poor extraction effect on detail features, the decoders in the lower 3 layers of the moire filter model can be removed to avoid the decoded feature maps output by the 3 layers of decoders interfering with the final prediction image, and the operation efficiency is improved.

[0093] As shown, Figure 4 The feature fusion layer 403 can include a first feature fusion layer 4031 and a second feature fusion layer 4032. When the prediction image is a moire prediction image, the feature fusion layer 403 corresponding to the first decoding unit 4021 can be the first feature fusion layer 4031. The first feature fusion layer 4031 corresponding to the first decoding unit 4021 is configured to upsample the multiple moire decoded feature maps greater than or equal to the preset scale in the moire decoded feature maps of different scales until the size of the input moire topic sample image, and fuse the multiple images to obtain the moire prediction image.

[0094] As shown, Figure 4 When the prediction image is a topic prediction image, the feature fusion layer 403 corresponding to the second decoding unit 4022 can be the second feature fusion layer 4032. The second feature fusion layer 4032 corresponding to the second decoding unit 4022 is configured to upsample the multiple topic decoded feature maps greater than or equal to the preset scale in the topic decoded feature maps of different scales until the size of the input moire topic sample image, and fuse the multiple images to obtain the topic prediction image.

[0095] In order to facilitate understanding of the method of the example embodiments of the present disclosure, the moire filter process of the moire filter model of the U2Net network architecture is described below.

[0096] Figure 5A The architecture schematic diagram of the moire filter model according to the example embodiments of the present disclosure is shown. As shown, Figure 5AAs shown, the moire filter model 500 is a U2Net network architecture, which includes an encoding unit and a decoding unit. The encoding unit includes 6 encoders, each of which has a network structure of U-Net structure or a full convolutional neural network, and the input feature scale and the output feature scale of each encoder are the same. The decoding unit includes 5 decoders, each of which is a U-Net structure or a full convolutional neural network, and the input feature scale and the output feature scale of each decoder are the same. The 6 encoders are defined as a first encoder 511, a second encoder 512, a third encoder 513, a fourth encoder 514, a fifth encoder 515 and a sixth encoder 516 respectively, and are connected in turn. The 5 decoders are defined as a first decoder 521, a second decoder 522, a third decoder 523, a fourth decoder 524 and a fifth decoder 525 respectively, and are connected in turn. Moreover, the architecture of the first encoder 511 is the same as that of the fifth decoder 525, the architecture of the second encoder 512 is the same as that of the fourth decoder 524, the architecture of the third encoder 513 is the same as that of the third decoder 523, the architecture of the fourth encoder 514 is the same as that of the second decoder 522, and the architecture of the fifth encoder 515 is the same as that of the first decoder 521.

[0097] Figure 5B The schematic diagram of the architecture of the first encoder of the exemplary embodiment of the present disclosure is shown. Figure 5C The schematic diagram of the architecture of the second encoder of the exemplary embodiment of the present disclosure is shown. Figure 5D The schematic diagram of the architecture of the third encoder of the exemplary embodiment of the present disclosure is shown. Figure 5E The schematic diagram of the architecture of the fourth encoder of the exemplary embodiment of the present disclosure is shown. Figure 5B to Figure 5E As can be seen, the network architecture of the first encoder 511, the second encoder 512, the third encoder 513, the fourth encoder 514 and the fifth encoder 515 is a U-Net network architecture, and correspondingly, the network architecture of the second decoder 522, the third decoder 523, the fourth decoder 524 and the fifth decoder 525 is a U-Net network architecture.

[0098] The network architecture of the fifth encoder and the sixth encoder in the exemplary embodiment of the present disclosure is the same. Figure 5F The schematic diagram of the architecture of the fifth encoder of the exemplary embodiment of the present disclosure is shown. As Figure 5F As shown, in the fifth encoder 515, the sizes of the encoding feature maps extracted by different network depths are the same.

[0099] The exemplary moire filter model architecture of the present disclosure can be divided into a downsampling stage and an upsampling stage in the process of processing an image. The following describes a 512x512 image processing process as an example.

[0100] AsFigure 5A As shown, in the downsampling stage, the 512×512 image is processed by the first encoder 511 to extract features. The scale of the first-level coded feature map output by the first encoder 511 remains 512×512. Before being input into the second encoder 512, a first downsampling operation can be performed, making the scale of the first-level coded feature map 254×254. Then it is input into the second encoder 512. The second-level coded feature map output by the second encoder 512 is processed by a second downsampling operation before being input into the third encoder 513, making the scale of the second-level coded feature map 128×128. Then it is input into the third encoder 513. The third encoder 513 outputs... Before the Level 3 coded feature map is input into the fourth encoder 514, it can undergo a third downsampling operation, making its scale 44×44. Before the Level 4 coded feature map output by the fourth encoder 514 is input into the fifth encoder 515, it can undergo a fourth downsampling operation, making its scale 32×32. Before the Level 5 coded feature map output by the fifth encoder 515 is input into the sixth encoder 516, it can undergo a fifth downsampling operation, making its scale 14×14. Therefore, the scale of the Level 6 coded feature map output by the sixth encoder 516 is 14×14.

[0101] During the upsampling phase, each feature input to the decoder undergoes both feature concatenation and upsampling operations before entering the decoder. The order of the upsampling and concatenation operations can be set according to the actual situation. The following description of the upsampling phase assumes that the upsampling operation is performed first, followed by the concatenation operation.

[0102] like Figure 5AAs shown, the scale of the 6th level encoded feature map output by the sixth encoder 516 is 14x14, and after the first upsampling operation, the scale is increased to 32x32, and the 5th level encoded feature map (the scale is 32x32) output by the fifth encoder 515 is spliced, and the first spliced feature map formed after splicing is input into the first decoder 521, therefore, the scale of the 1st level decoded feature map output by the first decoder 521 is 32x32, and the information of the first spliced feature is fused. After the second upsampling operation of the 1st level decoded feature map, the scale is increased to 44x44, and the 4th level encoded feature map (the scale is 44x44) output by the fourth encoder 514 is spliced, and the second spliced feature formed after splicing is input into the second decoder 522, therefore, the scale of the 2nd level decoded feature map output by the second decoder 522 is 44x44, and the information of the second spliced feature is fused. After the third upsampling operation of the 2nd level decoded feature map, the scale is increased to 128x128, and the 3rd level encoded feature map (128x128) output by the third encoder 513 is spliced, and the third spliced feature formed after splicing is input into the third decoder 523, therefore, the scale of the 3rd level decoded feature map Sup3 output by the third decoder 523 is 128x128, and the information of the third spliced feature is fused. After the fourth upsampling operation of the 3rd level decoded feature map Sup3, the scale is increased to 254x254, and the 2nd level encoded feature map (the scale is 254x254) output by the second encoder 512 is spliced, and the fourth spliced feature formed after splicing is input into the fourth decoder 524, therefore, the scale of the 3rd level decoded feature map Sup2 output by the fourth decoder 524 is 254x254, and the information of the third spliced feature is fused; after the fifth upsampling operation of the 4th level decoded feature map Sup2, the scale is increased to 512x512, and the 1st level encoded feature map (the scale is 512x512) output by the first encoder 511 is spliced, and the fifth spliced feature formed after splicing is input into the fifth decoder 525, therefore, the scale of the 5th level decoded feature map Sup1 output by the fifth decoder 525 is 512x512, and the information of the fifth spliced feature is fused, and the image scale is restored. On this basis, the decoded feature map output by the third decoder 523, the decoded feature map output by the fourth decoder 524 and the decoded feature map output by the fifth decoder 525 are spliced, thereby obtaining a prediction image S with rich context information fuse .

[0103] As a possible implementation manner, since the moire filter-out model can have two decoding modules, the moire prediction image and the question prediction image can be respectively predicted, so as to increase the number of supervision signals in the retraining stage of the moire filter-out model, so that the training of the moire filter-out model is carried out in a more accurate direction. Therefore, the loss of the moire filter-out model in the training stage can include a first sub-loss, a second sub-loss and a third sub-loss, and the loss can be determined by the first sub-loss, the second sub-loss and the third sub-loss in a weighted manner. It should be understood that the loss function used herein can be a mean square error loss function (MSE), a mean absolute error loss function (MAE), etc., and is not limited thereto.

[0104] For example, the loss Loss of the moire filter-out model in the training stage (总) = X1*Fm+X2*Fq+X3*Fmq, wherein Fm is the first sub-loss, X1 is the weight of the first sub-loss, Fq is the second sub-loss, X2 is the weight of the second sub-loss, Fmq is the third sub-loss, and X3 is the weight of the third sub-loss.

[0105] In a possible implementation manner, Figure 6 A schematic flowchart of a method for obtaining a question image according to an example embodiment of the present disclosure is shown. As Figure 6 shown, the moire removal of the moire question image by using the moire filter-out model to obtain the question image includes:

[0106] Step 601: Extract the encoding feature map of the moire question image by using an encoding unit. Specifically, the moire question image can be input into the moire filter-out model trained by using the training method described above, and the encoding feature map of the moire question image can be extracted by using the encoding unit of the moire filter-out model.

[0107] For example, the moire question image can be input into the N-level encoder in series, so that the encoding feature maps of different scales are output by different level encoders. As the number of levels of the encoder increases, the scale of the encoding feature map of the moire question image becomes smaller. The input of the nth level encoder is the encoding feature map output by the (n-1)th level encoder, and n represents an integer greater than or equal to 2 and less than or equal to N.

[0108] Step 602: Output the decoding feature maps of different scales based on the encoding feature map by using a decoding unit. Specifically, the encoding feature map can be input into the decoding unit of the moire filter-out model trained by using the training method described above, and the decoding feature maps of different scales can be obtained by using the decoding unit of the moire filter-out model.

[0109] Exemplarily, the encoded feature map output by the Nth encoder can be input into the concatenated M-level decoder, so that the different-level decoders output decoded feature maps of different scales, wherein the scale of the decoded feature map of the moire subject image increases as the number of levels of the decoder increases, the input of the mth decoder includes the decoded feature map output by the m-1th decoder and the encoded feature map output by the N-m+1th encoder; m is greater than or equal to 2 and less than or equal to M.

[0110] Step 603: determining the subject image based on the multiple decoded feature maps of different scales that are greater than or equal to the preset scale by using the feature fusion layer.

[0111] Exemplarily, the feature fusion layer of the moire filter-out model trained by the training method described above can be used to fuse the multiple decoded feature maps of different scales that are greater than or equal to the preset scale, to obtain the subject image, wherein the ratio of the multiple decoded feature maps of different scales that are greater than or equal to the preset scale to the number of decoded feature maps of different scales is greater than 0 and less than 100%. For example, the ratio of the multiple decoded feature maps of different scales that are greater than or equal to the preset scale to the number of decoded feature maps of different scales can be 10%, 20%, 50%, 70% and 90%, etc., and is not limited thereto. This is to avoid interference of deep features on the final prediction image, and to improve the operation efficiency.

[0112] In a possible implementation, before the moire-removed subject image is input into the feature extraction model, the moire-removed subject image can be preprocessed first, and then the preprocessed subject image is input into the feature extraction model. Figure 7 A schematic flowchart of a method for obtaining a feature vector of a subject image according to an exemplary embodiment of the present disclosure is shown. As shown in Figure 7 The above feature extraction of the subject image to obtain the feature vector of the subject image can include:

[0113] Step 701: obtaining the gray scale data of the pixel points of the subject image. It should be understood that the gray scale data here can be the value of each pixel point of the subject image after the subject image is grayed.

[0114] Exemplarily, each pixel point can be grayed based on the RGB value of each pixel point in the subject image to obtain the gray scale data of the pixel points of the subject image. It should be understood that the graying method here can include floating-point algorithm, integer method, shift method, average value method, only green method, etc.

[0115] Step 702: If the gray scale data of the pixel point is less than the preset gray scale threshold, the pixel point is deleted, and a processed question image is obtained. It should be understood that the preset gray scale threshold herein can be set to any value between 0 and 255 according to actual conditions, which is not limited herein. For example, the preset gray scale threshold can be 1, 5, etc., which is not limited herein.

[0116] For example, the preset gray scale threshold can be set to 1, the gray scale data of each pixel point in the gray-scaled question image is compared with the preset gray scale threshold 1, the pixel points with gray scale data greater than 1 are retained, and the pixel points with gray scale data less than 1 are deleted to obtain the processed question image.

[0117] Step 703: Feature extraction is performed on the processed question image to obtain a feature vector of the question image. At this time, the feature extraction model described above can be used to perform feature extraction on the processed question image to obtain the feature vector of the question image.

[0118] It should be understood that when the question image of the exemplary embodiment of the present disclosure is image preprocessed, at least one of the following image processing methods can also be performed on the question image:

[0119] First, the moire-removed question image is scaled to a size suitable for the input image of the image preprocessing module, for example, the moire-removed question image can be scaled to a square or rectangular image, etc., which can be determined according to actual conditions.

[0120] Second, the moire-removed question image is standardized. For example, the moire-removed question image is centered by mean removal, according to the knowledge of convex optimization theory and data probability distribution, data centering conforms to the data distribution law, and the generalization effect after training is easier to achieve. The image data can also be normalized, for example, the pixel value range of the illustration image and the illustration foreground image is converted from 0-255 to 0-1, thereby facilitating subsequent processing, etc.

[0121] Figure 8 A schematic flowchart of a question search method based on a feature vector according to an exemplary embodiment of the present disclosure is shown. As shown in Figure 8 The above method can include:

[0122] Step 801: Based on the feature vector of the question image, a plurality of comparison question feature vectors are obtained from a question bank.

[0123] For example, the feature vector model described above can be used to extract feature vectors from the question image. In this case, the question bank can be a feature vector search library. The feature vector of the question image to be searched can be input into the feature vector search library. The feature vector search library compares the feature vector of the question image to be searched with the feature vectors of the questions contained in the feature vector search library to determine the feature vectors of multiple matching questions that match the feature vector of the question image to be searched.

[0124] The feature vectors for each type of question included in the aforementioned feature vector search library can be extracted using the feature vector extraction model mentioned above. These questions can include various types of questions, such as multiple-choice questions, application questions, geometry questions, etc. It should be understood that the feature vector search library used in the exemplary embodiments of this disclosure can be the ElasticSearch search library, or it can be the goLang search library, the milkvus search library, etc., and is not limited to these.

[0125] Step 802: Based on the feature vector of each comparison question and the feature vector of the question image, determine the feature vector similarity of multiple comparison questions. Here, similarity comparison methods such as edit distance, cosine similarity, Euclidean distance, and longest common subsequence can be used to compare the similarity between the feature vector of the question image to be searched and the feature vector of the comparison questions, thereby obtaining the feature vector similarity of multiple comparison questions.

[0126] Step 803: Based on the feature vector similarity of multiple comparison questions, select multiple comparison questions that meet the similarity screening criteria as recommended questions.

[0127] The similarity filtering criteria mentioned above may include: the similarity of the feature vectors of the comparison questions is greater than a preset similarity. It may also include: sorting the comparison questions in descending order of their feature vector similarity, recommending the k-th comparison question as the recommended question, where k is greater than 0 and less than or equal to K, where K represents an integer less than the total number of comparison questions. Alternatively, a combination of these two criteria can be used to filter comparison questions.

[0128] When the similarity filtering criteria include a feature vector similarity greater than a preset similarity, the preset similarity can be a range value or a point value. For example, assuming the preset similarity is 97%, the feature vector similarity of the first comparison question is 99%, and the feature vector similarity of the second comparison question is 95%, it can be seen that the feature vector similarity of the first comparison question is greater than the preset similarity, while the feature vector similarity of the second comparison question is less than the preset similarity. Therefore, the first comparison question is determined to be the recommended question. Multiple recommended questions are then selected from the multiple comparison questions and pushed to the user's device.

[0129] When the similarity screening condition includes sorting the plurality of comparison test questions in descending order of the similarity of the feature vectors of the plurality of comparison test questions, the recommended test question is the kth comparison test question. Here, it is essentially limited that the test question to be recommended needs to be sorted in the top k comparison test questions. For example, the plurality of comparison test questions are sorted in descending order of the similarity of the feature vectors, it is assumed that the similarity of the feature vector of the first comparison test question ranked first is 100%, and the similarity of the feature vector of the kth comparison test question ranked kth is 95%, the top k comparison test questions with a similarity greater than or equal to 95% are taken as the recommended test questions, and the recommended test questions are pushed to the user equipment.

[0130] When the similarity screening condition includes a combination of the above two, not only the comparison test questions ranked in the top k are selected, but also the similarity of the feature vectors of the comparison test questions ranked in the top k is greater than a certain preset value. Continuing the previous example, since it is required that the similarity of the feature vector of the comparison test question is greater than 97%, and there are comparison test questions with a similarity of the feature vector less than 97% in the similarity of the comparison test questions ranked in the top k, the comparison test questions ranked in the top k cannot be directly determined as the recommended test questions, the comparison test questions with a similarity of the feature vector less than 97% in the comparison test questions ranked in the top k need to be removed, it is assumed that x comparison test questions are removed, then the comparison test questions ranked in the top k-x are determined as the recommended test questions, and the recommended test questions are pushed to the user equipment.

[0131] The above mainly introduces the scheme provided by the embodiments of the present disclosure from the perspective of the server. It can be understood that the server includes a hardware structure and / or a software module for executing each function in order to implement the above functions. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present disclosure can be realized in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed by hardware or computer software driven hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present disclosure.

[0132] The embodiments of the present disclosure can divide the function units of the server according to the above method examples, for example, each function module can be divided corresponding to each function, or two or more functions can be integrated in one processing module. The above integrated module can be realized in the form of hardware or in the form of a software function module. It should be noted that the division of the modules in the embodiments of the present disclosure is illustrative, and is only a logical function division. There can be another division way when actually implemented.

[0133] In the case of dividing each functional module according to each function, the example embodiment of the present disclosure provides a question searching device which can be a server or a chip applied to a server. Figure 9 A functional module schematic block diagram of a question searching device according to an example embodiment of the present disclosure is shown. As shown in Figure 9 The question searching device 900 includes:

[0134] The processing module 901 is configured to obtain a moire question image, perform moire removal processing on the moire question image by using a moire filtering model, and obtain a question image, wherein a loss in a training stage of the moire filtering model is determined by a first sub-loss, a second sub-loss, and a third sub-loss, the first sub-loss is determined based on a moire image label and a moire predicted image, the second sub-loss is determined based on a question image label and a question predicted image, and the third sub-loss is determined based on a moire question predicted image and a moire question sample image.

[0135] The searching module 902 is configured to perform feature extraction on the question image to obtain a feature vector of the question image, and obtain recommended test questions from a test question library based on a similarity between the feature vector of the question image and feature vectors of the test questions in the test question library.

[0136] As a possible implementation manner, the above-mentioned moire filtering model includes an encoding unit, a decoding unit, and a feature fusion layer, and the question searching device further includes an extraction module 903 configured to extract an encoded feature map of the moire question image by using the encoding unit.

[0137] The question searching device further includes a determination module 904 configured to output decoding feature maps of different scales based on the encoded feature map by using the decoding unit, and determine the question image based on a plurality of decoding feature maps greater than or equal to a preset scale in the decoding feature maps of different scales by using the feature fusion layer.

[0138] In some optional manners, the above-mentioned encoding unit includes N-level encoders connected in series, the decoding unit includes M-level decoders connected in series, N-M = 1, M and N both represent integers greater than or equal to 2, and the extraction module 903 is further configured to input the moire question image into the N-level encoders connected in series, so that the different-level encoders output encoded feature maps of different scales, wherein the scale of the encoded feature map of the moire question image becomes smaller as the number of levels of the encoder increases, the input of an n-level encoder is the encoded feature map output by an n-1-level encoder, n represents an integer greater than or equal to 2 and less than or equal to N.

[0139] The determination module 904 is further configured to input the encoded feature map of the output of the Nth encoder into the concatenated M-level decoder, so that the decoding feature maps of different scales are output by different levels of the decoder, and the scale of the decoding feature map of the moire subject image increases as the number of levels of the decoder increases. The input of the mth level of the decoder includes the decoding feature map output by the m-1th level of the decoder and the encoded feature map output by the N-m+1th encoder. m is greater than or equal to 2 and less than or equal to M.

[0140] In some optional manners, the ratio of the number of the plurality of decoding feature maps greater than or equal to the preset scale to the number of the decoding feature maps of different scales is greater than 0 and less than 100%.

[0141] As a possible implementation, the processing module 901 is further configured to obtain the gray scale data of the pixel points of the subject image, delete the pixel points if the gray scale data of the pixel points is less than a preset gray scale threshold, obtain the processed subject image, and perform feature extraction on the processed subject image to obtain the feature vector of the subject image.

[0142] As a possible implementation, the search module 902 is further configured to obtain the feature vectors of a plurality of comparison test questions from the test question library based on the feature vector of the subject image, determine the feature vector similarity of the plurality of comparison test questions based on the feature vector of each comparison test question and the feature vector of the subject image, and obtain a plurality of comparison test questions that satisfy a similarity screening condition as recommended test questions from the plurality of comparison test questions based on the feature vector similarity of the plurality of comparison test questions. The similarity screening condition includes that the feature vector similarity of the comparison test question is greater than a preset similarity, and / or

[0143] The plurality of comparison test questions are sorted in descending order of the feature vector similarity of the plurality of comparison test questions, the recommended test question is the kth comparison test question, k is greater than 0 and less than or equal to K, and K represents an integer less than the total number of comparison test questions.

[0144] As a possible implementation, the moire subject sample image is determined by the subject image label and the moire image label in a weighted manner, the weight of the subject image label is greater than the weight of the moire image label,

[0145] In the training stage, the moire filtering model includes two decoding units, the two decoding units have the same architecture, one of the two decoding units is configured to obtain a moire predicted image based on the moire subject sample image, and the other decoding unit is configured to obtain a subject predicted image based on the moire subject sample image.

[0146] Figure 10 A schematic block diagram of a chip according to an example embodiment of the present disclosure is shown. As shown in FIG. 1, the chip includes a processor 10, a memory 20, a bus 30, an input device 40, an output device 50, and a communication device 60. The bus 30 is configured to connect the processor 10, the memory 20, the input device 40, the output device 50, and the communication device 60. Figure 10As shown, the chip 1000 includes one or more (including two) processors 1001 and a communication interface 1002. The communication interface 1002 can support the server in performing the data transmission and reception steps in the above-described image processing method, and the processor 1001 can support the server in performing the data processing steps in the above-described image processing method.

[0147] Optional, such as Figure 10 As shown, the chip 1000 also includes a memory 1003, which may include read-only memory and random access memory, and provides operation instructions and data to the processor. A portion of the memory may also include non-volatile random access memory (NVRAM).

[0148] In some implementations, such as Figure 10 As shown, processor 1001 executes corresponding operations by calling operation instructions stored in memory (which may be stored in the operating system). Processor 1001 controls the processing operations of any terminal device; processor can also be called a central processing unit (CPU). Memory 1003 may include read-only memory and random access memory, and provides instructions and data to processor 1001. A portion of memory 1003 may also include NVRAM. For example, in applications, memory, communication interfaces, and other components are coupled together via a bus system, which may include, in addition to a data bus, a power bus, a control bus, and a status signal bus, etc. However, for clarity, in... Figure 10 The general labeled all buses as Bus System 1004.

[0149] The method disclosed in the embodiments of the present disclosure can be applied to a processor or implemented by the processor. The processor can be an integrated circuit chip having a signal processing capability. In the implementation process, the steps of the above method can be completed by hardware integrated logic circuits in the processor or by instructions in the form of software. The processor mentioned above can be a general processor, a digital signal processor (DSP), an ASIC, a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiments of the present disclosure can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor or the like. The steps of the method disclosed in combination with the embodiments of the present disclosure can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register or other mature storage medium in the art. The storage medium is located in the memory, and the processor reads the information in the memory and combines the hardware to complete the steps of the above method.

[0150] The exemplary embodiments of the present disclosure further provide an electronic device, including at least one processor, and a memory connected with the at least one processor in communication. The memory stores a computer program capable of being executed by the at least one processor, and the computer program, when executed by the at least one processor, is configured to cause the electronic device to perform the method according to the embodiments of the present disclosure.

[0151] The exemplary embodiments of the present disclosure further provide a non-transitory computer readable storage medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is configured to cause the computer to perform the method according to the embodiments of the present disclosure.

[0152] The exemplary embodiments of the present disclosure further provide a computer program product, including a computer program, wherein the computer program, when executed by a processor of a computer, is configured to cause the computer to perform the method according to the embodiments of the present disclosure.

[0153] Reference Figure 11The present invention describes a structural block diagram of an electronic device 1100 that can serve as a server or client of the present disclosure, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0154] like Figure 11 As shown, the electronic device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1102 or a computer program loaded from a storage unit 1108 into a random access memory (RAM) 1103. The RAM 1103 may also store various programs and data required for the operation of the electronic device 1100. The computing unit 1101, ROM 1102, and RAM 1103 are interconnected via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0155] Multiple components in electronic device 1100 are connected to I / O interface 1105, including: input unit 1106, output unit 1107, storage unit 1108, and communication unit 1109. Input unit 1106 can be any type of device capable of inputting information to electronic device 1100. Input unit 1106 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of electronic device. Output unit 1107 can be any type of device capable of presenting information and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 1108 may include, but is not limited to, disk and optical disk. Communication unit 1109 allows electronic device 1100 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth™ devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0156] The computing unit 1101 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, or the like. The computing unit 1101 performs various methods and processes described above. For example, in some embodiments, the methods of the example embodiments of the present disclosure can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 1108. In some embodiments, portions or all of the computer program can be loaded and / or installed onto the electronic device 1100 via the ROM 1102 and / or the communication unit 1109. In some embodiments, the computing unit 1101 can be configured to perform the methods of the example embodiments of the present disclosure by way of other any suitable means, such as by way of firmware.

[0157] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or the block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0158] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable storage media can include, without limitation, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include one or more lines of a system, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0159] As used in this disclosure, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus and / or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal that can be used to provide machine instructions and / or data to a programmable processor.

[0160] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0161] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0162] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0163] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer programs or instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present disclosure are performed. The computer can be a general purpose computer, a special purpose computer, a computer network, a terminal, user equipment or other programmable apparatus. The computer programs or instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another computer readable storage medium, for example, the computer programs or instructions can be transferred from one website site, computer, server or data center to another website site, computer, server or data center through wired or wireless manner. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center and the like integrated with one or more available media. The available media can be a magnetic medium, for example, a floppy disk, a hard disk, a magnetic tape; or an optical medium, for example, a digital video disc (digital video disc, DVD); or a semiconductor medium, for example, a solid state drive (solid state drive, SSD).

[0164] Although the present disclosure is described in conjunction with specific features and embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art in light of the foregoing description. Accordingly, it is intended to embrace all alternatives, modifications and variations that fall within the scope of the present disclosure. Obviously, various modifications and changes are possible in the present disclosure without departing from the spirit and scope of the present disclosure. Accordingly, the present disclosure intends to include all such modifications and changes in the scope of the present disclosure. Obviously, those skilled in the art can make various modifications and changes to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and changes of the present disclosure fall within the scope of the claims of the present disclosure and their equivalent technologies, the present disclosure also intends to include these modifications and changes.

Claims

1. A question search method, characterized in that, The method includes: Obtain the moiré pattern title image; The moiré pattern removal model is used to remove moiré patterns from the moiré pattern question image to obtain the question image. The loss of the moiré pattern removal model during the training phase is determined by a first sub-loss, a second sub-loss, and a third sub-loss. The first sub-loss is determined based on the moiré pattern prediction image and the moiré pattern image label. The second sub-loss is determined based on the question prediction image and the question image label. The third sub-loss is determined based on the moiré pattern question sample image and the moiré pattern question prediction image. The moiré pattern question prediction image is determined by the moiré pattern prediction image and the question prediction image. Feature extraction is performed on the question image to obtain the feature vector of the question image; Based on the similarity between the feature vector of the question image and the feature vector of the questions in the question bank, recommended questions are obtained from the question bank.

2. The method according to claim 1, characterized in that, The moiré pattern removal model includes an encoding unit, a decoding unit, and a feature fusion layer. The process of removing moiré patterns from the moiré question image using the moiré pattern removal model to obtain the question image includes: Using the encoding unit, the encoded feature map of the moiré pattern title image is extracted; Using the decoding unit, decoded feature maps of different scales are output based on the encoded feature map; Using the feature fusion layer, the title image is determined based on multiple decoded feature maps of different scales that are greater than or equal to a preset scale.

3. The method according to claim 2, characterized in that, The encoding unit includes an N-level encoder connected in series, and the decoding unit includes an M-level decoder connected in series, where NM=1, and M and N both represent integers greater than or equal to 2. The step of extracting the encoded feature map of the moiré pattern image using the encoding unit includes: The moiré pattern question image is input into an N-level encoder connected in series, so that the encoders at different levels output encoded feature maps of different scales. As the number of encoder levels increases, the scale of the encoded feature map of the moiré pattern question image decreases. The input of the n-th level encoder is the encoded feature map output by the (n-1)-th level encoder, where n represents an integer greater than or equal to 2 and less than or equal to N. The step of using the decoding unit to output decoded feature maps of different scales based on the encoded feature map includes: The encoded feature map output by the Nth level encoder is input into the M-level decoder in series, so that the decoders at different levels output decoded feature maps of different scales. As the number of decoder levels increases, the scale of the decoded feature map of the moiré title image increases. The input of the m-th level decoder includes the decoded feature map output by the (m-1)-th level decoder and the encoded feature map output by the (N-m+1)-th level encoder; where m is greater than or equal to 2 and less than or equal to M.

4. The method according to claim 2, characterized in that, The ratio of the number of decoded feature maps of a preset scale to the number of decoded feature maps of different scales is greater than 0 and less than 100%.

5. The method according to any one of claims 1-4, characterized in that, Feature extraction is performed on the question image to obtain the feature vector of the question image, including: Obtain the grayscale data of the pixels in the image of the question; If the grayscale data of the pixel is less than the preset grayscale threshold, the pixel is deleted to obtain the processed title image. Feature extraction is performed on the processed question image to obtain the feature vector of the question image.

6. The method according to any one of claims 1-4, characterized in that, The process of using the feature vector to perform question search includes: Based on the feature vectors of the question images, feature vectors of multiple comparison questions are obtained from the question bank; Based on the feature vector of each comparison question and the feature vector of the question image, the feature vector similarity of multiple comparison questions is determined. Based on the feature vector similarity of multiple comparison test items, several comparison test items that meet the similarity screening criteria are selected as recommended test items. The similarity filtering conditions include: The feature vector similarity of the comparison test questions is greater than the preset similarity; and / or, The multiple comparison questions are sorted in descending order of feature vector similarity. The recommended questions are the first k comparison questions, where k is greater than 0 and less than or equal to K, and K represents an integer less than the total number of comparison questions.

7. The method according to any one of claims 1-4, characterized in that, The moiré pattern question sample image is determined by a weighted average of the question image label and the moiré pattern image label, where the weight of the question image label is greater than the weight of the moiré pattern image label. The moiré pattern filtering model includes two decoding units during the training phase. The two decoding units have the same architecture. One decoding unit is used to obtain the moiré pattern prediction image based on the moiré pattern question sample image, and the other decoding unit is used to obtain the question prediction image based on the moiré pattern question sample image.

8. A question search device, characterized in that, The device includes: The processing module is used to acquire a moiré pattern question image, and to perform moiré pattern removal processing on the moiré pattern question image using a moiré pattern filtering model to obtain a question image. The loss of the moiré pattern filtering model during the training phase is determined by a first sub-loss, a second sub-loss, and a third sub-loss. The first sub-loss is determined based on the moiré pattern image label and the predicted moiré pattern image. The second sub-loss is determined based on the question image label and the predicted question image. The third sub-loss is determined based on the predicted moiré pattern question image and the sample moiré pattern question image. The predicted moiré pattern question image is determined by the predicted moiré pattern image and the predicted question image. The search module is used to extract features from the question image to obtain the feature vector of the question image, and to obtain recommended questions from the question bank based on the similarity between the feature vector of the question image and the feature vector of the questions in the question bank.

9. An electronic device, characterized in that, include: processor; as well as, Memory for stored programs; The program includes instructions that, when executed by the processor, cause the processor to perform the method according to any one of claims 1-7.

10. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions for causing the computer to perform the method according to any one of claims 1-7.

Images