Method and apparatus for detecting duplicate topics, electronic device, and storage medium

Abstract

The application provides a kind of heavy subject detection method, device, electronic equipment and storage medium, the method comprises: obtaining the subject to be detected;From a plurality of candidate subjects, determine the similar subject of the subject to be detected;Based on the first similarity, the subject characteristics of the subject to be detected and the subject characteristics of the similar subject, determine whether the subject to be detected and the similar subject are heavy subject;The first similarity is the similarity between the first text image feature of the subject to be detected and the second text image feature of the similar subject, the first text image feature is determined based on the text of the subject to be detected and the image corresponding to the subject to be detected, and the second text image feature is determined based on the text of the similar subject and the image corresponding to the similar subject.The heavy subject detection method, device, electronic equipment and storage medium provided by the application can improve the accuracy of heavy subject detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of text processing, and particularly relates to a duplicate question detection method and device, electronic equipment and a storage medium. BACKGROUND

[0002] With the maturity of artificial intelligence (AI) technology and the continuous subdivision of the education industry form, AI+ education has developed rapidly. In the field of AI+ education, how to quickly and efficiently detect questions that do not need to be processed repeatedly from a large number of to-be-detected questions, that is, how to detect duplicate questions, has become a hot issue in the industry.

[0003] At present, the existing duplicate question detection scheme mainly realizes through keyword retrieval and rule duplicate judgment system, that is, first, the keywords contained in the to-be-detected question are retrieved to retrieve relevant questions with the same keywords as the to-be-detected question from a candidate question library to roughly recall part of the questions; then, based on the rule duplicate judgment system, the relevant questions retrieved are compared and judged one by one with the to-be-detected question to detect whether the to-be-detected question is a duplicate question.

[0004] However, in the above duplicate question detection process, if the to-be-detected question contains numbers, formulas or patterns and the like, a large number of irrelevant questions will be retrieved, thereby reducing the accuracy of duplicate question detection. SUMMARY

[0005] The present application provides a duplicate question detection method, device, electronic equipment and storage medium to solve the problem of low accuracy of duplicate question detection in the prior art, and achieves the purpose of improving the accuracy of duplicate question detection.

[0006] The present application provides a duplicate question detection method, comprising:

[0007] obtaining a to-be-detected question;

[0008] determining a similar question of the to-be-detected question from a plurality of candidate questions;

[0009] determining whether the to-be-detected question and the similar question are duplicate questions based on a first similarity, a question feature of the to-be-detected question and a question feature of the similar question; the first similarity is a similarity between a first text image feature of the to-be-detected question and a second text image feature of the similar question, the first text image feature is determined based on a text of the to-be-detected question and an image corresponding to the to-be-detected question, and the second text image feature is determined based on a text of the similar question and an image corresponding to the similar question.

[0010] The method for detecting duplicate questions provided in the application comprises the following steps:

[0011] determining a second similarity between the question feature of the to-be-detected question and the question feature of the similar question;

[0012] performing keyword detection on the to-be-detected question, and determining a first weight corresponding to the first similarity and a second weight corresponding to the second similarity based on the detected keywords;

[0013] determining a target similarity between the to-be-detected question and the similar question based on the first similarity, the first weight, the second similarity and the second weight;

[0014] determining whether the to-be-detected question and the similar question are duplicate questions based on the target similarity.

[0015] The method for detecting duplicate questions provided in the application further comprises the following steps:

[0016] inputting the text of the to-be-detected question and the image corresponding to the to-be-detected question into a text-image feature extraction model to obtain a first text-image feature output by the text-image feature extraction model;

[0017] The text-image feature extraction model is constructed based on a gating feature fusion mechanism and a self-attention mechanism, and is trained by using a text sample and an image sample obtained by image acquisition on the text sample.

[0018] The method for detecting duplicate questions provided in the application further comprises the following steps:

[0019] determining a first answer of the to-be-detected question and a second answer of the similar question respectively in a case where the to-be-detected question and the similar question are determined to be non-duplicate questions;

[0020] updating the model parameters of the text-image feature extraction model based on the similar question in a case where the first answer and the second answer are the same.

[0021] The method for detecting duplicate questions provided in the application further comprises the following steps:

[0022] adding the to-be-detected question into a candidate question library in a case where there is no similar question of the to-be-detected question in the plurality of candidate questions or the first answer and the second answer are not the same, wherein the candidate question library comprises the plurality of candidate questions.

[0023] The method for detecting duplicate questions provided in the application further comprises the following steps:

[0024] determine at least one text line of the to-be-detected question based on the text of the to-be-detected question;

[0025] For each text line, obtain the text box position where the text line is located;

[0026] Based on the text line and the text box position, determine the text encoding feature of the text line, and based on the image corresponding to the to-be-detected question and the text box position where the text line is located, determine the visual feature of the text line;

[0027] Fuse the text encoding feature and the visual feature of the text line to obtain a fusion feature;

[0028] Based on the fusion feature of at least one text line, determine the first text image feature of the to-be-detected question.

[0029] According to the heavy question detection method provided by the application, the text encoding feature and the visual feature of the text line are fused to obtain a fusion feature, which comprises:

[0030] Based on the text encoding feature and the visual feature of the text line, a gating feature fusion mechanism is used to determine the first feature corresponding to the text line;

[0031] According to the first feature corresponding to the text line, determine the attention score between the text line and other text lines in the to-be-detected question, and according to the attention score, determine the fusion feature of the text line.

[0032] The application also provides a heavy question detection device, comprising:

[0033] The acquisition module is used to acquire a to-be-detected question;

[0034] The determination module is used to determine a similar question of the to-be-detected question from a plurality of candidate questions;

[0035] The determination module is also used to determine whether the to-be-detected question and the similar question are heavy questions based on a first similarity, a question feature of the to-be-detected question and a question feature of the similar question; the first similarity is the similarity between the first text image feature of the to-be-detected question and the second text image feature of the similar question, the first text image feature is determined based on the text of the to-be-detected question and the image corresponding to the to-be-detected question, and the second text image feature is determined based on the text of the similar question and the image corresponding to the similar question.

[0036] The application further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the question detection method according to any one of the above when executing the program.

[0037] The application further provides an electronic device, comprising a camera, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the camera is used to collect a question to be detected; the processor is used to acquire the question to be detected, determine a similar question of the question to be detected from a plurality of candidate questions, and determine whether the question to be detected and the similar question are duplicate questions based on a first similarity, a question feature of the question to be detected, and a question feature of the similar question, wherein the first similarity is a similarity between a first text image feature of the question to be detected and a second text image feature of the similar question, the first text image feature is determined based on a text of the question to be detected and an image corresponding to the question to be detected, and the second text image feature is determined based on a text of the similar question and an image corresponding to the similar question.

[0038] The application further provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program is executable on a processor to implement the question detection method according to any one of the above.

[0039] The application further provides a computer program product comprising a computer program, wherein the computer program is executable on a processor to implement the question detection method according to any one of the above.

[0040] The question detection method, device, electronic device, and storage medium provided by the application can determine whether the question to be detected and the similar question are duplicate questions based on the first text image feature determined based on the text of the question to be detected and the image corresponding to the question to be detected, the first similarity between the second text image feature of the similar question and the first text image feature of the question to be detected, and the question features of the question to be detected and the similar question. Since the question detection method using two detection rules, i.e., the text image feature and the question feature, is used to determine whether the question to be detected and the similar question are duplicate questions, the accuracy of the duplicate question detection of the question to be detected can be improved. In addition, since the image corresponding to the question to be detected contains comprehensive features, the first text image feature of the question to be detected contains not only single-modal text features but also multi-modal features such as numbers, formulas, and pictures when the question to be detected contains numbers, formulas, or pictures. Therefore, the accuracy of the duplicate question detection of the question to be detected can be further improved when the question to be detected is detected based on the first similarity between the first text image feature of the question to be detected and the second text image feature of the similar question, the question features of the question to be detected and the similar question. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0042] Figure 1 is one of the flowcharts of the topic detection method provided by the embodiments of the present application;

[0043] Figure 2 is the flowchart of the first text image feature extraction method provided by the embodiments of the present application;

[0044] Figure 3 is the structural diagram of the text image feature extraction model provided by the embodiments of the present application;

[0045] Figure 4 is the flowchart of the fusion feature extraction method provided by the embodiments of the present application;

[0046] Figure 5 is the second flowchart of the topic detection method provided by the embodiments of the present application;

[0047] Figure 6 is the structural diagram of the topic detection device provided by the embodiments of the present application;

[0048] Figure 7 is one of the structural diagrams of the electronic device provided by the present application;

[0049] Figure 8 is the second structural diagram of the electronic device provided by the present application. DETAILED DESCRIPTION

[0050] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely in combination with the drawings in the present application. Obviously, the described embodiments are some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all the other embodiments obtained by those skilled in the art without creative effort belong to the protection scope of the present application.

[0051] In the field of education, it is often necessary to process the questions, that is, to scan the questions in the teaching aids into pictures, and then store the questions in the question resource library after recognizing the questions in the pictures into texts through the OCR (Optical Character Recognition) technology. However, with the continuous expansion of the question resource library, a large number of repeated questions inevitably appear. Therefore, when the questions in the question resource library are processed in detail, a large amount of repetitive work will be generated, in other words, when the same questions are processed, the questions will be processed again in the case that some questions have been processed in detail. Thus, the processing cost of the questions is greatly increased. As can be seen, in the field of question processing, especially in the aspect of reducing the cost in the field of question processing, it is very important to detect the repeated questions for the questions to be processed. The questions to be processed are also called the questions to be detected.

[0052] Specifically, when the questions are processed, the primary problem of reducing the cost is "how to avoid repeated processing", that is, in the case that some questions have been processed in detail, when the subsequent questions to be processed are processed, if there is a repeated question corresponding to the question to be processed, only the repeated question can be reused, and the question to be processed does not need to be processed in detail from zero. Thus, it is not difficult to see that the problem of "how to avoid repeated processing" is actually a problem of how to detect the repeated questions for the questions to be processed in the processed question library, that is, in the candidate question library.

[0053] Further, since the questions are processed in batches, the more and more accurate repeated questions detected before the questions are processed into the library, the more questions that can be reused, that is, the shorter the processing time of the questions to be processed. Thus, in the case of single processing, the shorter the processing time of a question, the shorter the time of another question entering the processing, so that the number of questions processed in the same time period is more, thereby further improving the effect of "avoiding repeated processing".

[0054] Therefore, in the process of question processing, the key to solving the problem of repeated processing is how to develop a more accurate repeated question detection method for the questions to be processed, so that more processed questions, that is, the questions to be processed with repeated questions, can be efficiently and accurately detected in near real time, thereby avoiding repeated processing to the greatest extent.

[0055] Based on this, the prior art proposes a duplicate question detection scheme based on keyword search and rule duplicate judgment system. The scheme first needs to prepare a keyword table to facilitate keyword extraction of the to-be-detected question, and then searches for similar questions containing the same keywords in the candidate question library based on the extracted keywords. Then, based on the rules artificially summarized in the rule duplicate judgment system, the to-be-detected question and the similar question are compared and judged one by one to obtain the duplicate question detection result of the to-be-detected question.

[0056] However, the rules artificially summarized in the existing rule duplicate judgment system have a very limited coverage, that is, the energy of the rule writer is limited and it is impossible to traverse all the questions, so the rules artificially summarized often have one-sidedness and are highly targeted, thus greatly increasing the error rate of the duplicate question detection of the to-be-detected question. Moreover, since the length and content of the to-be-detected question may be different, the number of keywords that can be extracted from the to-be-detected question is usually different. Therefore, when keyword search is performed on the to-be-detected question containing more keywords, the search time is too long due to the large number of keywords, thereby reducing the efficiency of the duplicate question detection of the to-be-detected question. In addition, in the above duplicate question detection process, if the to-be-detected question contains numerical, formula or pattern content, a large number of irrelevant questions will be retrieved, thereby further reducing the accuracy of the duplicate question detection of the to-be-detected question.

[0057] To overcome this defect, the embodiment of the present application proposes a duplicate question detection method. In this method, considering that the image corresponding to the to-be-detected question obtained by scanning the to-be-detected question and the image corresponding to the similar question obtained by scanning the similar question both contain relatively comprehensive features, the similarity between the first text image feature determined by the text of the to-be-detected question and the image corresponding to the to-be-detected question and the second text image feature determined by the text of the similar question and the image corresponding to the similar question can be further considered on the basis of the question features of the to-be-detected question and the question features of the similar question to determine whether the to-be-detected question and the similar question are duplicate questions. This way avoids matching by keywords, thus improving the efficiency of duplicate question detection. In addition, since the image corresponding to the to-be-detected question contains relatively comprehensive features, the first text image feature corresponding to the to-be-detected question contains not only single-mode text features but also multi-modal features such as numbers, formulas and pictures when facing multi-modal to-be-detected questions containing numbers, formulas or pictures. Therefore, when the to-be-detected question is detected for duplicate questions based on the first similarity between the first text image feature of the to-be-detected question and the second text image feature of the similar question and the question features of the to-be-detected question and the question features of the similar question, the accuracy of the duplicate question detection of the to-be-detected question can be further improved.

[0058] The following will be described in conjunction with Figures 1-5The method for detecting duplicate questions provided by the embodiment of the present application can be applied to any detection scene that needs to detect whether it is the same question. The subject executing the method can be a duplicate question detection device connected with an image acquisition device or an image scanning device, such as a mobile phone, a computer, a computer or any other electronic device capable of detecting duplicate questions.

[0059] Figure 1 One of the flowcharts of the method for detecting duplicate questions provided by the embodiment of the present application is shown in FIG. 1, which comprises the following steps. Figure 1

[0060] Step 101: Obtain a question to be detected.

[0061] The question to be detected can be a question composed of any one or more languages, which is not specifically limited. The question to be detected is a question that needs to be processed and prepared for addition to the test question resource library.

[0062] Specifically, the question to be detected can be obtained by test question scanning, test question shooting or network test question downloading.

[0063] Step 102: Determine the similar question of the question to be detected from a plurality of candidate questions.

[0064] The candidate question refers to a processed question, such as a text question recognized from a picture. The plurality of candidate questions are stored in a candidate question library.

[0065] Specifically, a plurality of questions that are not the same as each other can be first screened from the processed questions and used as candidate questions. After obtaining the question to be detected and the plurality of candidate questions, the Vearch retrieval model is used to retrieve the similar question of the question to be detected from the plurality of candidate questions. The Vearch retrieval model is based on Faiss (Facebook AISimilarity Search, Facebook Similarity Vector Search) and is a distributed vector search system. In use, it can be used to store and calculate a large number of feature vectors. In management, it provides Restful API similar to Elasticsearch distributed search engine, which can facilitate the management and query of data and table structure. Therefore, the system has been widely applied to various machine learning fields such as image, audio and video, and natural language processing.

[0066] ​Similar question recall in the embodiment of the application is also implemented through the text image features of the candidate questions, similar to the similar question recall in some machine learning projects through the text image features of the candidate questions in the Vearch retrieval library. Specifically, the similarity between the text image features of the to-be-detected question and the text image features of each candidate question is calculated first; then, the similarity between each candidate question and the to-be-detected question is compared with the similarity threshold one by one, so as to take each candidate question corresponding to a similarity greater than the similarity threshold as the similar question of the to-be-detected question. The Vearch retrieval library, also called the candidate question library, stores the text image features of all candidate questions, the corresponding relationship between the candidate questions and the text image features of the candidate questions, and the candidate questions themselves.

[0067] Further, the text image features of all candidate questions need to be used to train the Vearch retrieval model to improve the accuracy of the Vearch retrieval model, and thus improve the accuracy of retrieving similar questions from multiple candidate questions.

[0068] The Vearch retrieval model trained by the above method can only complete the retrieval function of retrieving the candidate questions stored in the Vearch retrieval library. Therefore, in order to ensure the accuracy of model retrieval, the candidate questions in the Vearch retrieval library and their text image features, and the corresponding relationship between the candidate questions and the text image features of the candidate questions need to be updated, that is, the to-be-detected question that has been retrieved is added as a new candidate question, and the candidate question and its text image features, and the corresponding relationship between the candidate questions and the text image features of the candidate questions are added to the Vearch retrieval library, so as to avoid repeated retrieval of the to-be-detected question, and thus improve the accuracy of retrieving similar questions of the to-be-detected question from multiple candidate questions.

[0069] Step 103: determining whether the to-be-detected question and the similar question are duplicate questions based on the first similarity, the question features of the to-be-detected question, and the question features of the similar question.

[0070] The first similarity is the similarity between the first text image features of the to-be-detected question and the second text image features of the similar question, the first text image features are determined based on the text of the to-be-detected question and the image corresponding to the to-be-detected question, and the second text image features are determined based on the text of the similar question and the image corresponding to the similar question. It should be understood that the first similarity can also be understood as the similarity score between the first text image features of the to-be-detected question and the second text image features of the similar question.

[0071] In a possible implementation, when determining whether the to-be-detected question and the similar question are duplicate questions based on the first similarity, the question feature of the to-be-detected question, and the question feature of the similar question, the first text image feature of the to-be-detected question and the second text image feature of the similar question can be extracted first; then, the first similarity between the first text image feature and the second text image feature is calculated based on the first text image feature and the second text image feature, and the question feature of the to-be-detected question and the question feature of the similar question are calculated in a statistical manner, so as to determine whether the to-be-detected question and the similar question are duplicate questions based on the first similarity, the question feature of the to-be-detected question, and the question feature of the similar question. The question feature is a feature that can be obtained from the question text itself, such as the content length of the question, the question type of the question, the text character repetition ratio of the question, and the like, which will not be listed one by one here.

[0072] The first text image feature and the second text image feature can be extracted by any one of the following text image feature extraction methods.

[0073] First, the first text image feature extraction method is a text image feature extraction scheme based on single-character input. The scheme first encodes each character in the question through an encoding layer; then, the image corresponding to each character is encoded through a visual backbone network, and the character encoding and image encoding are spliced and then input into a transformer network to obtain the fusion feature of each character, and the fusion features of each character are sequentially spliced to obtain the text image feature of the question.

[0074] Further, the second text image feature extraction method is a text image feature extraction scheme based on sentence input. The scheme first extracts the semantic features of each sentence in the question through a BERT (Bidirectional Encoder Representation from Transformers) model; then, the image corresponding to each sentence is encoded through a visual backbone network to obtain the visual feature of each sentence, and the semantic features and visual features of each sentence are spliced and then fused through a transformer network to obtain the fusion feature of each sentence, and then the fusion features of each sentence are sequentially spliced to obtain the text image feature of the question.

[0075] Optionally, in order to improve the accuracy of text image feature extraction, the transformer network can also be pre-trained in a manner of pre-training tasks such as text masking, image masking, and text-image alignment, to obtain a more accurate transformer network, so that the accuracy of the extracted text image feature is higher.

[0076] In another possible implementation, when determining whether the to-be-detected question and the similar question are duplicate questions based on the first similarity, the question features of the to-be-detected question, and the question features of the similar question, a duplicate detection algorithm can be used to perform duplicate detection on the to-be-detected question.

[0077] Specifically, any one or more duplicate detection algorithms can be used to perform duplicate detection on the to-be-detected question, such as an XGBoost (eXtreme Gradient Boosting) machine learning algorithm. The XGBoost machine learning algorithm is a tree ensemble model that can not only well fit the classification task of data but also well complete the regression task of data. Unlike most ensemble learning models, the algorithm supports distributed parallel computing, runs extremely efficiently, and has good algorithm portability. Currently, the algorithm has been widely used in many machine learning competitions and large-scale data projects in the industry.

[0078] Since the duplicate question detection problem of the to-be-detected question is still a typical binary classification problem in machine learning, it is not difficult to understand that a machine learning duplicate detection model based on a duplicate detection algorithm can be used to perform duplicate detection on the to-be-detected question. The input of the machine learning duplicate detection model based on the duplicate detection algorithm is the comparison features or the fusion features of the to-be-detected question and the similar question, and the output is whether the to-be-detected question and the similar question are duplicate questions.

[0079] Further, the machine learning duplicate detection model based on the duplicate detection algorithm can also be trained using a supervised learning training method. In this way, the model can learn the parameter fitting of the existing to-be-detected question pairs that are duplicate questions, and can better mine the important features of the to-be-detected question and the similar question, such as comparison features or fusion, and the relationship between the duplicate question label and the to-be-detected question and the similar question. The to-be-detected question pair, also called the duplicate question pair, refers to the to-be-detected question and the similar question that are duplicate questions.

[0080] Similar to general machine learning projects, the machine learning duplicate detection model based on the duplicate detection algorithm also needs to go through data collection, data preprocessing, feature engineering, model training and testing, and model optimization operations to improve the accuracy of the model.

[0081] Next, the specific implementation process of the machine learning duplicate detection model based on the duplicate detection algorithm is described in detail. For ease of description, the machine learning duplicate detection model based on the duplicate detection algorithm is referred to as the duplicate detection model.

[0082] First, the label sample data of the duplicate detection model is obtained by using the Vearch retrieval model + manual annotation method. The label sample data is the duplicate question pair and the non-duplicate question pair, and the ratio between the duplicate question pair and the non-duplicate question pair is 1:1.

[0083] Specifically, in the case that the Vearch retrieval model retrieves similar questions of the to-be-detected question, the duplicate questions of the to-be-detected question are determined from the similar questions of the to-be-detected question by means of manual duplicate detection, wherein the to-be-detected question and the similar questions that are not duplicate questions are referred to as duplicate question pairs.

[0084] Further, in order to ensure that the proportion of duplicate question pairs and non-duplicate question pairs is 1:1, the non-duplicate question pairs obtained are randomly selected so that the number of non-duplicate question pairs is the same as the number of duplicate question pairs, and then the same number of duplicate question pairs and non-duplicate question pairs are manually annotated to obtain label sample data of the duplicate detection model.

[0085] Then, the label sample data of the duplicate detection model obtained is divided into a training set, a development set and a test set according to a ratio of 7:1:2, and based on the characteristics of the label sample data on the development set and artificial experience, key features of the label sample data are extracted.

[0086] The key features of the label sample data can be understood as important features that affect the duplicate detection model, such as GraphDoc text image features, question content length features, question content length difference features, and text character repetition ratio features of the question, and the like, which are not listed one by one. In addition, a visual correlation analysis method can be used to analyze the correlation between part of the features of the label sample data and the duplicate question labels, filter out the key features of the label sample data, and extract the key features of the label sample data in the training set according to the key features, so as to train the duplicate detection model by means of the Xgboost model tool.

[0087] Finally, after the duplicate detection model training is completed, the key features of the label sample data in the test set are extracted to test the duplicate detection effect of the trained duplicate detection model. When the duplicate detection effect of the duplicate detection model meets the standard, no optimization of the duplicate detection model is performed.

[0088] Further, when the duplicate detection effect of the duplicate detection model does not meet the standard, the key features of the label sample data in the validation set are extracted, and the multiple duplicate detection results output by the trained duplicate detection model are analyzed, and then the analysis results are used to modify the key features or the selection scheme of the key features to obtain new key features, and the above model training and optimization process is repeated until the duplicate detection effect of the trained duplicate detection model meets the standard.

[0089] Therefore, the key features obtained through the above model training, testing and verification have more accurate duplication judgment pertinence, that is, the key features include more accurate distinguishing features between duplicate questions and non-duplicate questions, so that the accuracy of the duplication judgment model trained based on the key features of the label sample data in the training set is higher, and the accuracy of the duplicate question detection of the to-be-detected question can be improved.

[0090] In summary, the first text image feature includes text features and image features, and the image features are determined based on the picture obtained after image scanning of the to-be-detected question. The picture includes not only text information, but also other information such as patterns, formulas and tables, and the like, in addition to the text. Therefore, the first text image feature contains more comprehensive features of the to-be-detected question. Similarly, the second text image feature contains more comprehensive features of the similar question. Therefore, on the basis of comparing the question features of the to-be-detected question and the question features of the similar question, and combining the first similarity between the first text image feature and the second text image feature, the duplicate question judgment can be further improved.

[0091] The duplicate question detection method provided by the embodiment of the application can determine whether the to-be-detected question and the similar question are duplicate questions based on the first similarity between the first text image feature determined by the text of the to-be-detected question and the image corresponding to the to-be-detected question and the second text image feature determined by the text of the similar question and the image corresponding to the similar question, and the question features of the to-be-detected question and the question features of the similar question. Since the duplicate question detection method using two detection rules including the text image feature and the question feature is used to determine whether the to-be-detected question and the similar question are duplicate questions, the accuracy of the duplicate question detection of the to-be-detected question can be improved. In addition, since the image corresponding to the to-be-detected question contains more comprehensive features, the first text image feature corresponding to the to-be-detected question contains not only single-modal text features, but also multi-modal features such as numbers, formulas and pictures when the to-be-detected question contains numbers, formulas or pictures. Therefore, the accuracy of the duplicate question detection of the to-be-detected question can be further improved based on the first similarity between the first text image feature of the to-be-detected question and the second text image feature of the similar question, the question features of the to-be-detected question and the question features of the similar question.

[0092] On the basis of the above-mentioned embodiments, when determining whether the to-be-detected question and the similar question are the same question based on the first similarity, the question feature of the to-be-detected question and the question feature of the similar question, the following method can be used: determining a second similarity between the question feature of the to-be-detected question and the question feature of the similar question; performing keyword detection on the to-be-detected question, and determining a first weight corresponding to the first similarity and a second weight corresponding to the second similarity based on the detected keywords; determining a target similarity between the to-be-detected question and the similar question based on the first similarity, the first weight, the second similarity and the second weight; and determining whether the to-be-detected question and the similar question are the same question based on the target similarity.

[0093] The keywords mentioned above can be understood as words used to represent the modality of the to-be-detected question, such as “formula”, “ Figure 1 ”, “table”, “reference diagram” and the like.

[0094] Specifically, after obtaining the question feature of the to-be-detected question and the question feature of the similar question, the second similarity between the question feature of the to-be-detected question and the question feature of each similar question is calculated first; then, keyword detection is performed on the to-be-detected question based on the keywords in the pre-set keyword table, so that after detecting the keywords, the initial first weight corresponding to the first similarity and the initial second weight corresponding to the second similarity are adjusted based on the detected keywords to obtain the first weight corresponding to the first similarity and the second weight corresponding to the second similarity, and then the target similarity between the to-be-detected question and the similar question can be calculated based on the first similarity, the first weight, the second similarity and the second weight. The target similarity between the to-be-detected question and the similar question is calculated by the following target similarity formula (1). After the target similarity is determined, whether the to-be-detected question and the similar question are the same question can be determined based on the target similarity.

[0095] S = W1S1 + W2S2 (1)

[0096] Wherein, S represents the target similarity; W1 represents the first weight corresponding to the first similarity S1; and W2 represents the second weight corresponding to the second similarity S2.

[0097] For example, taking the detected keywords “ Figure 1 ” and “ Figure 2 ” as an example, wherein the keywords “ Figure 1 ” and “ Figure 2It is indicated that the to-be-detected question includes at least two images, that is, the to-be-detected question includes not only text information but also at least two pictures. As can be seen, the to-be-detected question contains a large amount of picture information. Since the second similarity between the question characteristics of the to-be-detected question and the question characteristics of the similar question usually takes more text information into account, the first weight of the first similarity between the first text image feature of the to-be-detected question and the second text image feature of the similar question can be increased, and the second weight of the second similarity between the question characteristics of the to-be-detected question and the question characteristics of the similar question can be correspondingly reduced, so that the target similarity calculated according to formula (1) is more accurate, and thus the result of determining whether the to-be-detected question and the similar question are duplicate questions based on the target similarity is also more accurate, that is, the accuracy of duplicate question detection of the to-be-detected question is improved.

[0098] In addition, in the above embodiment, when extracting the first text image feature of the to-be-detected question, the following method can be adopted: inputting the text of the to-be-detected question and the image corresponding to the to-be-detected question into a text image feature extraction model to obtain the first text image feature output by the text image feature extraction model. The text image feature extraction model is constructed based on a gating feature fusion mechanism and a self-attention mechanism and is trained by a text sample and an image sample obtained by image acquisition on the text sample.

[0099] Specifically, the gating feature fusion mechanism and the self-attention mechanism can be applied to any one of the text image feature extraction methods in the foregoing embodiments to construct the text image feature extraction model.

[0100] It should be understood that after the text image feature extraction model is constructed, the following model training needs to be performed:

[0101] First, a large number of text samples are collected from a question resource library.

[0102] The question resource library represents an entity or a virtual storage point, such as a bookstore, a library or a network, which stores a large number of different questions. The text sample represents any question in various test questions obtained from the question resource library.

[0103] Specifically, after selecting certain question resource libraries, the target test questions are searched in the question resource libraries, such as searching for the test question of “5-year high school entrance examination 3-year simulation” in a bookstore, and after the target test question is found, all the questions in the target test question are taken as the text samples.

[0104] Then, on the basis of obtaining the text samples, the text samples are subjected to image acquisition to obtain image samples for model training.

[0105] Specifically, each text sample obtained can be collected in a text scanning or text shooting manner to obtain image samples for model training, so as to train the text image feature extraction model and obtain a more accurate text image feature extraction model.

[0106] Further, when the text image feature extraction model is used to extract text image features of the to-be-detected question, the first text image features obtained are also more accurate.

[0107] The gating feature fusion mechanism proposed in this embodiment can fully fuse visual and semantic information at different levels, fuse image features and text features, and obtain whole-page features, that is, the gating feature fusion mechanism can fully fuse various modal information such as text, formula, table, and pattern in the to-be-detected question, thereby greatly improving the accuracy and efficiency of the duplicate question detection of the to-be-detected question. On this basis, by introducing the self-attention mechanism in the construction of the text image feature extraction model, the model can pay more attention to important features when extracting text image features, so that the accuracy of the first text image features of the to-be-detected question is higher.

[0108] For example, Figure 2 The flowchart of the first text image feature extraction method provided by the embodiment of the present application is shown in Figure 2 The method comprises the following steps:

[0109] Step 201: Based on the text of the to-be-detected question, at least one text line of the to-be-detected question is determined.

[0110] The to-be-detected question includes at least one text line, and the text line can be understood as a line of text.

[0111] Specifically, Figure 3 The structure diagram of the text image feature extraction model provided by the embodiment of the present application is shown in Figure 3 After obtaining the whole image corresponding to the text of the to-be-detected question, that is, the image corresponding to the to-be-detected question, the image corresponding to the to-be-detected question is recognized by using the OCR technology to obtain each text line in the to-be-detected question. The image corresponding to the text of the to-be-detected question is an image obtained by scanning the to-be-detected question.

[0112] Step 202: For each text line, the text box position where the text line is located is obtained.

[0113] Further, in the process of recognizing the text line of the to-be-detected question by using the OCR technology, the text box position where each text line in the to-be-detected question is located can also be obtained.

[0114] It should be noted that in some cases, each sentence in the text of the to-be-detected question can be a text line, and the i-th text line will be introduced as an example in the following.

[0115] Step 203: Determine the text encoding features of the text line based on the text line and the text box position, and determine the visual features of the text line based on the image corresponding to the to-be-detected question and the text box position where the text line is located.

[0116] For example, after obtaining each text line in the to-be-detected question and the text box position corresponding to each text line, first, encode the text box position information to obtain the position encoding features of the text line. Specifically, the text box position of the i-th text line can be represented as (x i0 , y i0 , x i1 , y i1 , w i , h i ), where (x i0 , y i0 ) is the coordinate of the upper left corner of the text box where the i-th text line is located, (x i1 , y i1 ) represents the coordinate of the lower right corner of the text box where the i-th text line is located, w i , h i represent the width and height of the text box where the i-th text line is located, respectively, i∈(1, 2, 3, …, N), N represents the number of text lines in the to-be-detected question, and N is a positive integer.

[0117] Since the 2D text box position of each text line in the to-be-detected question is helpful for question detection, an embedding layer is used to encode the text box position of the text line in this embodiment, and the specific encoding formula is as follows:

[0118] I i =Emb(x i0 , y i0 , x i1 , y i1 , w i , h i ) (2)

[0119] where I i represents the position encoding features of the i-th text line; x i0 , y i0 represent the coordinates of the upper left corner of the text box where the i-th text line is located, x i1 , y i1 represent the coordinates of the lower right corner of the text box where the i-th text line is located, w i , h i represent the width and height of the text box where the i-th text line is located, respectively.

[0120] Then, each text line obtained is input into the text encoding network of the text image feature extraction model to extract semantic features of each text line.

[0121] Specifically, the Sentence-Bert network can be used as the text encoding network of the text image feature extraction model to extract semantic features of each text line in the to-be-detected question.

[0122] Further, the semantic features of each text line are input into a fully connected layer for encoding to obtain first semantic encoding features of the text line, and the first semantic encoding features are summed with the position encoding features of the corresponding text line to obtain text encoding features of the text line, and the specific calculation formula is as follows:

[0123] S i i = f (SentenceEmb (t i i )) + I i (3)

[0124] wherein S i represents the text encoding features of the i-th text line; I i represents the position encoding features of the i-th text line; f(SentenceEmb(t i i )) represents the first semantic encoding features of the i-th text line; f represents a fully connected layer; SentenceEmb(t i i ) represents the first semantic features of the i-th text line; t i represents the i-th text line.

[0125] In the present implementation, after obtaining the image corresponding to the to-be-detected question, the image corresponding to the to-be-detected question can be input into the image encoding network of the text image feature extraction model for image encoding to obtain visual features of the text line.

[0126] As shown in Figure 3 , the visual backbone network is used as the image encoding network of the text image feature extraction model, and specifically, a Swin Transformer network connected with a Feature Pyramid Network (FPN) can be used as the visual backbone network, so that the image corresponding to the to-be-detected question input can be first preprocessed in size; then, it is input into the visual backbone network to obtain a visual feature map of the whole image, and according to the position of the text box where each text line is located, the visual features corresponding to the i-th text line are obtained through ROI, and the specific calculation formula is as follows:

[0127] V i= ROI (Pool (backbone (I))) (4)

[0128] wherein, V i represents the visual feature of the ith text line; Backbone represents a visual backbone network; Pool represents a pooling layer; and I represents an image corresponding to the to-be-detected question.

[0129] Step 204: Fuse the text coding feature and the visual feature of the text line to obtain a fusion feature.

[0130] Specifically, after obtaining the text coding feature of the text line and the visual feature of the text line through step 203, the two are jointly input into a fusion network of a text image feature extraction model to obtain the fusion feature corresponding to the text line. The fusion network of the text image feature extraction model can be a transformer network itself, or a fusion network combined by a transformer network and other networks or mechanisms, and no specific limitation is made to this.

[0131] Step 205: Determine the first text image feature of the to-be-detected question based on the fusion feature of at least one text line.

[0132] Specifically, after obtaining the fusion feature of each text line through step 204, the fusion features of the text lines are spliced according to the positions of the text boxes where each text line is located to determine the first text image feature of the to-be-detected question.

[0133] In summary, the present scheme can utilize the correlation between different modalities, that is, utilize the feature that different types of information such as text, formula, table and pattern in the to-be-detected question are correlated with each other, jointly model the text (each text line), image (the image corresponding to the to-be-detected question) and layout position information (the text box position of each text line) in the to-be-detected question, generate the first text image feature of the to-be-detected question, and use it for the recall of similar questions of the to-be-detected question. Thus, when facing a multi-modal to-be-detected question containing formula, table or picture, a more comprehensive feature of the to-be-detected question can be obtained, so that the accuracy of the re-question detection based on the first text image feature is higher.

[0134] Similarly, the second text image feature of the similar question of the to-be-detected question can also be extracted by the above-mentioned extraction method of the first text image feature, and the specific implementation manner of the second text image feature is the same as that in the foregoing embodiments, which will not be described herein.

[0135] An exemplary flowchart of the fusion feature extraction method provided by the embodiments of the present application is shown in FIG. 1. Figure 4 As shown in FIG. 1, the method comprises the following steps. Figure 4 ​

[0136] Step 401: determining the first feature corresponding to the text line based on the text encoding feature and the visual feature of the text line, using a gated feature fusion mechanism.

[0137] The text encoding feature of the text line is also called the second semantic encoding feature of the text line.

[0138] In the existing fusion network, the semantic features and visual features in different network layers may represent different meanings, for example, the semantic features of the shallow layer represent the semantics of a single word, and the semantic features of the deep layer represent the semantics of the entire text line. Therefore, in order to better fuse the semantic features and visual features in the to-be-detected question, in the embodiment, when fusing the semantic features and visual features in each layer of the fusion network, a gated feature fusion mechanism is used for fusion, so that after the semantic features and visual features of the text line are input into the fusion network, the gated feature fusion mechanism is used for fusion in each layer of the fusion network, thereby obtaining the first feature of each text line.

[0139] Specifically, in the process of fusing the semantic features and visual features of the text line, the fusion network autonomously learns the weights of the semantic features and visual features in each layer of the fusion network, so as to find the best fusion mode for the semantic features and visual features of different network layers, for example, different colors and different sizes of fonts in the to-be-detected question may represent different meanings. The gated feature fusion mechanism calculates the weights of the semantic features and visual features in each layer of the fusion network through the following calculation formulas (5) and (6). For example, when distinguishing the meanings of different sentences in the to-be-detected question, a larger weight is assigned to the visual features; and when facing text lines that are similar in vision, a larger weight is assigned to the semantic features.

[0140]

[0141]

[0142] wherein, is a gating coefficient, used to represent the weight corresponding to the semantic features and visual features of the i-th text line in the l-th layer; is the output of the gating mechanism, used to represent the first feature corresponding to the i-th text line in the l-th transformer network layer; represents the text line feature corresponding to the i-th text line output by the l-1-th transformer network layer; σ is an activation function, and W and b are weight matrix and bias respectively.

[0143] In order to better represent the interaction between the semantic features and the visual features of the text lines in the to-be-detected question, in a possible implementation, a text line mask prediction task is adopted as a preset text line training task of the text image feature extraction model,

[0144] Specifically, the text line samples of the text line mask can be obtained according to a preset proportion, for example, 30% of the text line samples are randomly selected, and part of the text in the text line samples is replaced with a preset symbol [MASK], so as to input the text line samples into the initial text image feature extraction model for training, and obtain the text image feature extraction model. Wherein, the visual features and the position encoding features of the text line samples need to be preserved, and the text encoding features of the masked text lines in these text line samples are pre-labeled for training the initial text image feature extraction model.

[0145] After obtaining the text line samples, each text line sample is sequentially input into the initial text image feature extraction model, so as to predict the text encoding features of the masked text lines in the text line samples by using the visual features and the position encoding features of the masked text lines, and the visual features and the text encoding features of the unmasked text lines, and compare the prediction results with the corresponding pre-labeled results, and perform multiple rounds of model training according to the difference between the two and a preset objective function, until the loss value of the objective function output by the model is less than a preset value, at which time, the text image feature extraction model can be generated. Wherein, the specific structure of the initial text image feature extraction model is not limited, which can select a Sentence-Bert network, an initial visual backbone network and an initial transformer network, etc.

[0146] In the above model training process, the objective function can adopt the objective function smoothL1 shown in the following formula (7):

[0147]

[0148] Wherein, s i represents the expected value of the text encoding features of the masked text lines; represents the text encoding features of the masked text lines; V represents all the features input into the model, such as visual features, position encoding features, etc. represents the prediction result of the text image feature extraction model to s i , that is, the predicted value of the text encoding features of the masked text lines.

[0149] It should be noted that after the text image feature extraction model is trained by using the text line mask prediction task, the trained text image feature extraction model can be directly migrated to the processing of other text line analysis tasks, such as image mask, text image alignment, and the like, which only needs a small amount of training data for fine-tuning.

[0150] In this embodiment, after the first feature of the text line is obtained, the first feature of the text line can be analyzed and processed to obtain an analysis result of each text line in the to-be-detected question, so that more accurate and rapid result analysis of the to-be-detected question can be realized, that is, the analysis effect of the to-be-detected question is improved.

[0151] For example, in the element extraction task of the to-be-detected question, such as the extraction of elements such as the type of the to-be-detected question, formula or keyword, first, the text line in the to-be-detected question and the text box position of the text line are extracted by using the OCR tool; then, the text line and the image corresponding to the text line are input into the fusion network of the text image feature extraction model to obtain the fusion feature of the text line in the to-be-detected question, and then the fusion feature can be classified, and according to the classification result, the element category corresponding to each text line is recognized, such as which text line corresponds to the type, which text line corresponds to the formula, and which text line corresponds to the keyword element, so as to realize the element extraction of the to-be-detected question.

[0152] Step 402: determining the attention score between the text line and other text lines in the to-be-detected question according to the first feature corresponding to the text line, and determining the fusion feature of the text line according to the attention score.

[0153] In the traditional fusion network, the self-attention mechanism is usually used to calculate the attention score between each text line and other text lines in the to-be-detected question, but this calculation method not only increases the complexity of calculation, but also may cause the fusion network to fail to pay attention to more important text lines. Therefore, in order to improve the calculation accuracy of the fusion network and reduce the complexity of calculation, in one possible implementation, a graph self-attention mechanism is added to the fusion network.

[0154] Specifically, the nearest text line to the text line in the to-be-detected question is calculated by using the graph attention mechanism through the Euclidean distance, and the attention score between the text line and the nearest text line is calculated according to the first feature of the text line and the first feature of the nearest text line to the text line in the to-be-detected question; then, the fusion feature of the text line is calculated according to the attention score. In this way, when calculating the attention, more attention is paid to some text lines that are closest to the text line in the to-be-detected question, so that the calculation amount of the network is greatly reduced while the performance of the network is improved.

[0155] Specifically, assuming that the to-be-detected question contains N text lines, and the first features of the N text lines in the lth layer are respectively calculated by formula (6) The attention score between the ith text line and the jth text line can be calculated by formula (8) as follows:

[0156]

[0157] wherein e ij represents the attention score between the ith text line and the jth text line; W Q ,W K respectively represent weight matrices; represents the first feature of the ith text line in the lth layer; represents the first feature of the jth text line in the lth layer.

[0158] Further, after the K text lines closest to the ith text line in the to-be-detected question are calculated by the Euclidean distance, the fusion feature of the text line can be calculated by formula (9) as follows.

[0159]

[0160] wherein, represents the fusion feature of the ith text line in the lth layer; D(i) represents the neighborhood of the fusion feature of the ith text line in the to-be-detected question, that is, the K text lines closest to the ith text line in the to-be-detected question calculated by the Euclidean distance, and K is a positive integer greater than 0. In this embodiment, K is 36.

[0161] In this embodiment, the text encoding feature and the visual feature of the text line are fused by the gating feature fusion mechanism, so that the first feature of each text line in the to-be-detected question fuses visual and semantic information at different levels, that is, the first feature of the text line contains various modal feature information such as text, formula, table, and pattern in the to-be-detected question, thereby improving the accuracy of the first feature of each text line in the to-be-detected question. In addition, after the text encoding feature and the visual feature of the text line in the to-be-detected question are fused by the gating feature fusion mechanism to obtain the first feature of each text line, the attention score between each text line and other text lines in the to-be-detected question is also utilized, that is, the correlation between each text line and other text lines. Therefore, the first feature of the important text line can be more easily focused, so that the accuracy of the fusion feature of the text line determined based on the attention score is higher.

[0162] Further, it is not difficult to understand that when the duplicate detection result of the to-be-detected question is incorrect, it indicates that the accuracy of the duplicate detection is not enough, and therefore, the correct duplicate detection result of the to-be-detected question can be deduced reversely, and the text image feature extraction model in the duplicate detection can be updated based on the correct duplicate detection result.

[0163] In a possible implementation, in a case where it is determined that the to-be-detected question and the similar question are not duplicates, the first answer of the to-be-detected question and the second answer of the similar question can be determined respectively, so that in a case where the first answer and the second answer are the same, the model parameters of the text image feature extraction model are updated based on the similar question.

[0164] Specifically, in a case where the machine learning duplicate detection model based on the duplicate detection algorithm outputs a non-duplicate, first, the to-be-detected question and the similar question of the to-be-detected question are corrected by the correction system, so that in a case where the answers of the two are consistent, the similar question is recycled periodically or in real time by a question recycling mechanism, and is used for online learning (OL).

[0165] The process of the online learning includes a prediction result of an output model, collection of similar questions of the to-be-detected question, and updating of the model.

[0166] Specifically, the online learning is a prediction-update-re-prediction model updating system, which uses a back propagation algorithm to update various model parameters in the model, and repeatedly iterates the above process according to a prediction error until the model converges or a target loss reaches a current minimum, and the updating is stopped, that is, the model after each learning is the current optimal model, so that the accuracy of the obtained model is higher. In addition, since the online learning itself is a dynamic process and is based on real-time data, it is not a final result, that is, after each online learning, the text image feature extraction model is updated and corrected, so that the model can be adjusted in real time, so that the model can timely reflect the changes on the line, thereby improving the accuracy of online prediction.

[0167] In the embodiment, in the case that it is determined that the to-be-detected question and the similar question are not duplicate questions, the first answer of the to-be-detected question and the second answer of the similar question are determined respectively, and in the case that the first answer and the second answer are the same, the model parameters of the text-image feature extraction model are updated based on the similar question. In this way, in the case that the model judges that the duplicate question is wrong, the model parameters of the text-image feature extraction model are adjusted, and a more accurate text-image feature extraction model is obtained. Therefore, based on the more accurate text-image feature extraction model, a more accurate first text-image feature of the to-be-detected question and a more accurate second text-image feature of the similar question can be extracted. The first similarity calculated based on the first text-image feature of the to-be-detected question and the second text-image feature of the similar question is more accurate. Therefore, based on the first similarity, the question feature of the to-be-detected question and the question feature of the similar question, a more accurate detection result of whether the to-be-detected question and the similar question are duplicate questions is determined, that is, the accuracy of duplicate question detection of the to-be-detected question is improved.

[0168] Further, since the educational resources are a long-term and continuous processing process, when facing the repeated to-be-detected question, only by continuously adding the to-be-detected question which is not a duplicate question with the candidate question, that is, the new question, to the candidate question library, the problem of repeated detection of the same to-be-detected question can be avoided, so that the duplicate question detection effect of the to-be-detected question is guaranteed. Therefore, in order to further improve the accuracy of duplicate question detection of the to-be-detected question, the to-be-detected question needs to be added to the candidate question library in the case that the similar question of the to-be-detected question does not exist in the plurality of candidate questions, or the first answer and the second answer are not the same, that is, the to-be-detected question which is not a duplicate question with the candidate question is added to the candidate question library. In this way, the types of the candidate questions in the candidate question library are more, so that when searching for the similar question of the to-be-detected question based on the more candidate questions in the candidate question library, the accuracy of the similar question obtained is also higher. Therefore, when performing duplicate question detection on the to-be-detected question based on the more accurate similar question, the duplicate question detection result obtained is also more accurate, that is, the precision of duplicate question detection of the to-be-detected question is improved.

[0169] For example, Figure 5 The flowchart of the duplicate question detection method provided by the embodiment of the present application is shown in Figure 2. Figure 5 Before the method is executed, first, the text-image feature extraction model is used to extract the first text-image feature of the to-be-detected question and the second text-image feature of the similar question. Figure 5The GraphDoc model is used offline to extract the text image features of each candidate question to form a candidate question library, i.e., the aforementioned Vearch retrieval library. Then, the question to be tested is input into the GraphDoc model to extract the first text image features of the question to be tested. Then, based on the similarity between the calculated first text image features of the question to be tested and the text image features of each candidate question in the candidate question library, similar questions are determined from multiple candidate questions, and the first similarity between the first text image features of the question to be tested and the second text image features of the similar questions is calculated. Figure 5 The GraphDoc text-image similarity score is used to determine whether a question is duplicated. The model then compares the question's features with those of similar questions using statistical methods, along with the GraphDoc text-image similarity score. This process determines whether the question is duplicated. If the question is duplicated, the model outputs "duplicate"; otherwise, it outputs "non-duplicate". Questions that cannot be duplicated are directly classified as non-duplicate.

[0170] Furthermore, when the machine learning deduplication model outputs a non-duplicate question, the grading system grades the question to be tested and similar questions. If the first answer of the question to be tested and the second answer of the similar question are the same, the question retrieval mechanism is used to retrieve the similar questions of the question to be tested. Then, the GraphDoc model is updated in real time through online learning, thereby improving the accuracy of duplicate question detection for the question to be tested.

[0171] In addition, in this embodiment, the first similarity between the question to be detected and similar questions, namely the GraphDoc text-image similarity score, is also incorporated into the machine learning deduplication model. This makes the deduplication accuracy of the machine learning deduplication model higher, thereby improving the accuracy of detecting duplicate questions.

[0172] The duplicate title detection device provided in the embodiments of the present invention is described below. The duplicate title detection device described below can be referred to in correspondence with the duplicate title detection method described above.

[0173] Figure 6 This is a schematic diagram of the structure of the duplicate title detection device provided in an embodiment of the present invention, as shown below. Figure 6 As shown, the device includes:

[0174] Module 610 is used to acquire the questions to be tested;

[0175] The determination module 620 is used to identify similar questions to the question to be tested from multiple candidate questions;

[0176] The determination module 620 is further configured to determine whether the to-be-detected question and the similar question are duplicate questions based on the first similarity, the question feature of the to-be-detected question, and the question feature of the similar question, wherein the first similarity is a similarity between a first text image feature of the to-be-detected question and a second text image feature of the similar question, the first text image feature is determined based on a text of the to-be-detected question and an image corresponding to the to-be-detected question, and the second text image feature is determined based on a text of the similar question and an image corresponding to the similar question.

[0177] The duplicate question detection device provided in the embodiment of the present application can determine whether the to-be-detected question and the similar question are duplicate questions based on the first similarity between the first text image feature of the to-be-detected question and the second text image feature of the similar question, and the question feature of the to-be-detected question and the question feature of the similar question, which are determined by the obtaining module 610 based on the text of the to-be-detected question and the image corresponding to the to-be-detected question, and the text of the similar question and the image corresponding to the similar question, and the determination module 620. Since the duplicate question detection method using two detection rules including the text image feature and the question feature is used to determine whether the to-be-detected question and the similar question are duplicate questions, the accuracy of duplicate question detection of the to-be-detected question can be improved. In addition, since the image corresponding to the to-be-detected question contains comprehensive features, the first text image feature corresponding to the to-be-detected question contains not only the text feature of a single mode, but also the multi-modal features such as numbers, formulas, and pictures. Therefore, the accuracy of duplicate question detection of the to-be-detected question can be further improved when the to-be-detected question is detected based on the first similarity between the first text image feature of the to-be-detected question and the second text image feature of the similar question, the question feature of the to-be-detected question, and the question feature of the similar question.

[0178] Optionally, the determination module 620 is specifically configured to:

[0179] determine a second similarity between the question feature of the to-be-detected question and the question feature of the similar question;

[0180] perform keyword detection on the to-be-detected question, and determine a first weight corresponding to the first similarity and a second weight corresponding to the second similarity based on the detected keywords;

[0181] determine a target similarity between the to-be-detected question and the similar question based on the first similarity, the first weight, the second similarity, and the second weight;

[0182] determine whether the to-be-detected question and the similar question are duplicate questions based on the target similarity.

[0183] Optionally, the device further comprises:

[0184] The extraction module is configured to input the text of the to-be-detected question and the image corresponding to the to-be-detected question into the text-image feature extraction model to obtain first text-image features output by the text-image feature extraction model.

[0185] The text-image feature extraction model is constructed based on a gating feature fusion mechanism and an attention mechanism, and is trained by using text samples and image samples obtained by image acquisition on the text samples.

[0186] Optionally, the determination module 620 is further configured to, in a case where it is determined that the to-be-detected question and the similar question are not non-duplicate questions, determine a first answer of the to-be-detected question and a second answer of the similar question respectively.

[0187] The updating module is configured to, in a case where the first answer and the second answer are the same, update model parameters of the text-image feature extraction model based on the similar question.

[0188] Optionally, the apparatus further includes:

[0189] The adding module is configured to, in a case where there is no similar question of the to-be-detected question in the plurality of candidate questions, or the first answer and the second answer are not the same, add the to-be-detected question to a candidate question library, the candidate question library including the plurality of candidate questions.

[0190] Optionally, the determination module 620 is further configured to determine at least one text line of the to-be-detected question based on the text of the to-be-detected question.

[0191] The acquisition module 610 is configured to acquire, for each text line, a text box position where the text line is located.

[0192] The determination module 620 is further configured to determine a text encoding feature of the text line based on the text line and the text box position, and determine a visual feature of the text line based on the image corresponding to the to-be-detected question and the text box position where the text line is located.

[0193] The apparatus further includes:

[0194] The fusion module is configured to fuse the text encoding feature and the visual feature of the text line to obtain a fusion feature.

[0195] The determination module 620 is further configured to determine the first text-image feature of the to-be-detected question based on the fusion feature of the at least one text line.

[0196] Optionally, the fusion module is specifically configured to:

[0197] determine a first feature corresponding to the text line based on the text encoding feature and the visual feature of the text line by using a gating feature fusion mechanism.

[0198] According to the first feature corresponding to the text line, an attention score between the text line and other text lines in the to-be-detected question is determined, and a fusion feature of the text line is determined according to the attention score.

[0199] The device of the embodiment can be used to execute the method of any one of the above-mentioned device-side method embodiments of duplicate question detection, and the specific implementation process and technical effects are similar to those of the above-mentioned device-side method embodiments of duplicate question detection. For details, refer to the above-mentioned device-side method embodiments of duplicate question detection.

[0200] Figure 7 An example of a schematic diagram of a physical structure of an electronic device is shown in FIG. 1. Figure 7 As shown in FIG. 1, the electronic device can include a processor 710, a communications interface 720, a memory 730, and a communications bus 740. The processor 710, the communications interface 720, and the memory 730 can communicate with each other through the communications bus 740. The processor 710 can invoke a logical instruction in the memory 730 to execute a duplicate question detection method. The method includes obtaining a to-be-detected question, determining a similar question of the to-be-detected question from a plurality of candidate questions, and determining whether the to-be-detected question and the similar question are duplicate questions based on a first similarity, a question feature of the to-be-detected question, and a question feature of the similar question. The first similarity is a similarity between a first text image feature of the to-be-detected question and a second text image feature of the similar question. The first text image feature is determined based on a text of the to-be-detected question and an image corresponding to the to-be-detected question. The second text image feature is determined based on a text of the similar question and an image corresponding to the similar question.

[0201] In addition, the logical instruction in the memory 730 described above can be implemented in the form of a software function unit and sold or used as an independent product. When stored in a computer-readable storage medium, the technical solutions of the present application can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0202] Figure 8 An example of a schematic diagram of a physical structure of an electronic device is shown in FIG. 1.Figure 8 As shown, the electronic device can include a processor 810, a communications interface 820, a memory 830, a communications bus 840, and a camera 850, wherein the processor 810, the communications interface 820, the memory 830, and the camera 850 complete mutual communication through the communications bus 840. The camera 850 is used to collect a to-be-detected question, and the processor 810 can call a logical instruction in the memory 830 to execute a duplicate question detection method, which includes: obtaining a to-be-detected question; determining a similar question of the to-be-detected question from a plurality of candidate questions; determining whether the to-be-detected question and the similar question are duplicate questions based on a first similarity, a question feature of the to-be-detected question, and a question feature of the similar question; the first similarity is a similarity between a first text image feature of the to-be-detected question and a second text image feature of the similar question, the first text image feature is determined based on a text of the to-be-detected question and an image corresponding to the to-be-detected question, and the second text image feature is determined based on a text of the similar question and an image corresponding to the similar question.

[0203] In addition, the logical instructions in the memory 830 described above can be implemented in the form of a software function unit and sold or used as an independent product, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program codes that can be stored in the medium.

[0204] In another aspect, the present application also provides a computer program product comprising a computer program, the computer program being stored in a non-transitory computer-readable storage medium, and the computer program being executable by a processor to enable a computer to perform the duplicate question detection method provided by any of the above methods, the method comprising: obtaining a to-be-detected question; determining a similar question of the to-be-detected question from a plurality of candidate questions; and determining whether the to-be-detected question and the similar question are duplicate questions based on a first similarity, a question feature of the to-be-detected question, and a question feature of the similar question, the first similarity being a similarity between a first text-image feature of the to-be-detected question and a second text-image feature of the similar question, the first text-image feature being determined based on a text of the to-be-detected question and an image corresponding to the to-be-detected question, and the second text-image feature being determined based on a text of the similar question and an image corresponding to the similar question.

[0205] In yet another aspect, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, the computer program being executable by a processor to implement a duplicate question detection method provided by any of the above methods, the method comprising: obtaining a to-be-detected question; determining a similar question of the to-be-detected question from a plurality of candidate questions; and determining whether the to-be-detected question and the similar question are duplicate questions based on a first similarity, a question feature of the to-be-detected question, and a question feature of the similar question, the first similarity being a similarity between a first text-image feature of the to-be-detected question and a second text-image feature of the similar question, the first text-image feature being determined based on a text of the to-be-detected question and an image corresponding to the to-be-detected question, and the second text-image feature being determined based on a text of the similar question and an image corresponding to the similar question.

[0206] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0207] Those skilled in the art can clearly understand from the above description of the embodiments that the embodiments can be implemented by means of software and a necessary general hardware platform, and of course can also be implemented by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform the methods described in each embodiment or some parts of the embodiments.

[0208] It should be pointed out finally that the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features therein can be replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for detecting duplicate questions, characterized in that, include: Obtain the questions to be tested; Identify similar questions to the question to be tested from multiple candidate questions; Based on the first similarity, the question features of the question to be tested, and the question features of the similar questions, it is determined whether the question to be tested and the similar questions are duplicate questions; the first similarity is the similarity between the first text image features of the question to be tested and the second text image features of the similar questions, the first text image features are determined based on the text of the question to be tested and the image corresponding to the question to be tested, and the second text image features are determined based on the text of the similar questions and the image corresponding to the similar questions; The step of determining whether the question to be tested and the similar question are duplicate questions based on the first similarity, the question features of the question to be tested, and the question features of the similar questions includes: Determine the second similarity between the question features of the question to be detected and the question features of the similar questions; Keyword detection is performed on the question to be detected, and a first weight corresponding to the first similarity and a second weight corresponding to the second similarity are determined based on the detected keywords. Based on the first similarity, the first weight, the second similarity, and the second weight, the target similarity between the question to be detected and the similar questions is determined; Based on the target similarity, determine whether the question to be detected and the similar question are duplicate questions; The question features include at least one of the following: the length of the question content, the question type, and the percentage of repeated text characters in the question.

2. The duplicate question detection method according to claim 1, characterized in that, The method further includes: The text of the question to be tested and the image corresponding to the question to be tested are input into the text image feature extraction model to obtain the first text image feature output by the text image feature extraction model; The text image feature extraction model is constructed based on a gated feature fusion mechanism and a self-attention mechanism, and is trained using text samples and image samples obtained by image acquisition of the text samples.

3. The duplicate question detection method according to claim 2, characterized in that, The method further includes: If it is determined that the question to be tested and the similar question are not duplicate questions, then the first answer to the question to be tested and the second answer to the similar question are determined respectively. If the first answer and the second answer are the same, the model parameters of the text image feature extraction model are updated based on the similar questions.

4. The duplicate question detection method according to claim 3, characterized in that, The method further includes: If there are no similar questions to the question to be tested among the multiple candidate questions, or if the first answer and the second answer are different, the question to be tested is added to the candidate question bank, which includes multiple candidate questions.

5. The duplicate question detection method according to any one of claims 1-4, characterized in that, The method further includes: Based on the text of the question to be tested, at least one line of text of the question to be tested is determined; For each line of text, obtain the position of the text box containing that line of text; Based on the text line and the text box position, the text encoding features of the text line are determined, and based on the image corresponding to the question to be detected and the text box position where the text line is located, the visual features of the text line are determined; The text encoding features and visual features of the text line are fused to obtain the fused features; Based on the fusion features of at least one line of text, the first text image features of the question to be detected are determined.

6. The duplicate question detection method according to claim 5, characterized in that, The process of fusing the text encoding features and the visual features of the text line to obtain fused features includes: Based on the text encoding features and visual features of the text line, a first feature corresponding to the text line is determined using a gating feature fusion mechanism; Based on the first feature corresponding to the text line, the attention score between the text line and other text lines in the question to be tested is determined, and based on the attention score, the fusion feature of the text line is determined.

7. A duplicate question detection device, characterized in that, include: The acquisition module is used to acquire the questions to be tested. A determination module is used to identify similar questions to the question to be detected from multiple candidate questions; The determining module is further configured to determine whether the question to be tested and the similar question are duplicate questions based on a first similarity, the question features of the question to be tested, and the question features of the similar questions; the first similarity is the similarity between the first text image features of the question to be tested and the second text image features of the similar questions, the first text image features are determined based on the text of the question to be tested and the image corresponding to the question to be tested, and the second text image features are determined based on the text of the similar questions and the image corresponding to the similar questions; The device is also used for: Determine the second similarity between the question features of the question to be detected and the question features of the similar questions; Keyword detection is performed on the question to be detected, and a first weight corresponding to the first similarity and a second weight corresponding to the second similarity are determined based on the detected keywords. Based on the first similarity, the first weight, the second similarity, and the second weight, the target similarity between the question to be detected and the similar questions is determined; Based on the target similarity, determine whether the question to be detected and the similar question are duplicate questions; The question features include at least one of the following: the length of the question content, the question type, and the percentage of repeated text characters in the question.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the duplicate title detection method as described in any one of claims 1 to 6.

9. An electronic device comprising a camera, a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The camera is used to capture the question to be tested; the processor is used to acquire the question to be tested; determine similar questions from multiple candidate questions; and determine whether the question to be tested and the similar questions are duplicate questions based on a first similarity, the question features of the question to be tested, and the question features of the similar questions; the first similarity is the similarity between a first text image feature of the question to be tested and a second text image feature of the similar questions, wherein the first text image feature is determined based on the text of the question to be tested and the image corresponding to the question to be tested, and the second text image feature is determined based on the text of the similar questions and the image corresponding to the similar questions; The step of determining whether the question to be tested and the similar question are duplicate questions based on the first similarity, the question features of the question to be tested, and the question features of the similar questions includes: Determine the second similarity between the question features of the question to be detected and the question features of the similar questions; Keyword detection is performed on the question to be detected, and a first weight corresponding to the first similarity and a second weight corresponding to the second similarity are determined based on the detected keywords. Based on the first similarity, the first weight, the second similarity, and the second weight, the target similarity between the question to be detected and the similar questions is determined; Based on the target similarity, determine whether the question to be detected and the similar question are duplicate questions; The question features include at least one of the following: the length of the question content, the question type, and the percentage of repeated text characters in the question.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the duplicate title detection method as described in any one of claims 1 to 6.

Images