Text detection method, training method, device, equipment, medium and program

Abstract

To provide a text detection method, a training method, a device, an apparatus, a medium, and a program.SOLUTION: A training method includes processing a sample text pair and a sample tag by using an initial model in response to the reliability of a first sample detection result being lower than a prescribed threshold and obtaining a second sample detection result, obtaining a first loss value based on the second sample detection result and the sample tag on the basis of a first loss function, and adjusting a model parameter of the initial model on the basis of the first loss value and obtaining a trained target model. The reliability of the first sample detection result is obtained by detection of the sample text pair by using a first model. The sample text pair includes a sample inquiry text and a sample answer text. The second sample detection result includes a determination result and a determination factor for indicating an answer matching degree between the sample answer text and the sample inquiry text.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to the field of artificial intelligence, particularly to technical fields such as large-scale language models, deep learning, and text processing, and more particularly to a text detection method, a training method, an apparatus, a device, a medium, and a program. [Background technology]

[0002] With the spread of Internet technology, users can post questions over the Internet and receive answers to those questions from other users.

[0003] Since questions relate to many fields, problems such as "incorrect answers" or "question not answered" often occur. Therefore, a method is needed that can accurately detect the matching degree between answers and questions, thereby improving the user experience. Summary of the Invention

[0004] The present disclosure provides a text detection method, a training method, an apparatus, a device, a medium, and a program.

[0005] According to one aspect of the present disclosure, there is provided a model training method, including: in response to a confidence level of a first sample detection result being smaller than a predetermined threshold, processing the sample text pairs and the sample tags using an initial model to obtain a second sample detection result; obtaining a first loss value based on the second sample detection result and the sample tags according to a first loss function; and adjusting model parameters of the initial model based on the first loss value to obtain a trained target model, wherein the confidence level of the first sample detection result is obtained by detecting the sample text pairs using the first model, the sample text pairs including corresponding sample question texts and sample answer texts, the second sample detection result including a discrimination result representing an answer matching degree between the sample answer text and the sample question text and a discrimination cause corresponding to the discrimination result, the sample tags including a discrimination result tag and a discrimination cause tag, and the initial model is a type different from that of the first model.

[0006] According to another aspect of the present disclosure, there is provided a text detection method, including: detecting a plurality of to-be-detected text pairs using a first model, and obtaining a first detection result and a confidence level corresponding to the first detection result, wherein the first model is a discriminant model, and the plurality of to-be-detected text pairs include a plurality of question texts and a plurality of answer texts that correspond to each other; determining a target to-be-detected text pair from the plurality of to-be-detected text pairs based on the confidence level, wherein the confidence level of the detection result corresponding to the target to-be-detected text pair is smaller than a predetermined threshold; detecting the target to-be-detected text pair using a second model, and obtaining a second detection result, wherein the second model is a trained target model obtained by the training method according to any one of claims 1 to 4; and generating a target detection result representing an answer matching degree between the answer text and the corresponding question text based on the first detection result and the second detection result.

[0007] According to another aspect of the present disclosure, there is provided a model training apparatus including a first processing module, a first loss calculation module, and a first adjustment module.

[0008] The first processing module, in response to the confidence of the first sample detection result being smaller than a predetermined threshold, processes the sample text pair and the sample tag using an initial model to obtain a second sample detection result, wherein the confidence of the first sample detection result is obtained by detecting the sample text pair using the first model, the sample text pair includes a sample question text and a sample answer text that correspond to each other, the second sample detection result includes a discrimination result for representing the answer matching degree between the sample answer text and the sample question text and a discrimination cause corresponding to the discrimination result, the sample tag includes a discrimination result tag and a discrimination cause tag, and the initial model is different from the type of the first model.

[0009] The first loss calculation module obtains a first loss value based on the second sample detection result and the sample tag according to the first loss function.

[0010] The first adjustment module adjusts model parameters of the initial model based on the first loss value to obtain a trained target model.

[0011] According to another aspect of the present disclosure, there is provided a text detection apparatus including a second detection module, a second determination module, a third detection module, and a generation module.

[0012] The second detection module detects a plurality of text pairs to be detected using a first model, obtains a first detection result and a confidence level corresponding to the first detection result, and the first model is a discriminant model, and the plurality of text pairs to be detected include a plurality of question texts and a plurality of answer texts corresponding to each other.

[0013] The second determination module determines a target text pair to be detected from the plurality of text pairs to be detected based on the confidence level, and the confidence level of the detection result corresponding to the target text pair to be detected is smaller than a predetermined threshold.

[0014] The third detection module uses the second model to detect the text pairs to be targeted, and obtains a second detection result, where the second model is the trained target model obtained by the above training method.

[0015] The generation module generates a target detection result representing an answer matching degree between the answer text and the corresponding question text based on the first detection result and the second detection result.

[0016] According to another aspect of the present disclosure, there is provided an electronic device including at least one processor and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the above-described method.

[0017] According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium having stored thereon computer instructions that cause the computer to perform the method described above.

[0018] According to another aspect of the present disclosure, there is provided a computer program which, when executed by a processor, implements the above method.

[0019] It should be understood that the content described in this section is not intended to identify key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will be readily apparent from the following specification. [Brief explanation of the drawings]

[0020] The drawings are for a better understanding of the invention and are not intended to limit the disclosure. [Figure 1] FIG. 1 schematically illustrates an exemplary system architecture to which the model training method or text detection method and apparatus according to the embodiments of the present disclosure can be applied. [Figure 2] FIG. 2 is a schematic diagram illustrating a flowchart of a model training method according to an embodiment of the present disclosure. [Figure 3A] FIG. 3A illustrates a schematic diagram of model training according to an embodiment of the present disclosure. [Figure 3B] FIG. 3B schematically illustrates a model training diagram according to another embodiment of the present disclosure. [Figure 4] FIG. 4 schematically illustrates a flowchart of a text detection method according to an embodiment of the present disclosure. [Figure 5] FIG. 5 illustrates a schematic diagram of text detection according to an embodiment of the present disclosure. [Figure 6] FIG. 6 is a schematic block diagram of a model training apparatus according to an embodiment of the present disclosure. [Figure 7] FIG. 7 is a schematic block diagram of a text detection device according to an embodiment of the present disclosure. [Figure 8] FIG. 8 illustrates a block diagram of an electronic device suitable for implementing a model training method or a text detection method according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0021]

[0023] The following description of exemplary embodiments of the present disclosure will be made with reference to the drawings. For ease of understanding, various details of the embodiments of the present disclosure are included, but these details are merely illustrative. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for clarity and conciseness, the following description will omit descriptions of known functions and structures.

[0022] In the scenario of text question answer quality discrimination, there are many fields related to the question text in the sample data. In the process of training the discriminant model multiple times based on a large amount of sample data, the model is prone to overfitting.

[0023] If the input question-answer text pair to be detected deviates slightly from the sample data distribution, the model inference performance fluctuates significantly, resulting in a significant decrease in the accuracy of the model output results.

[0024] In addition, the model in the related example detects text question-answering quality based on surface features such as the linguistic fluency of the answer text and the lexical relationship between the question and the answer, and does not sufficiently process deep features such as the logical relationship between the question and the answer, resulting in low model accuracy.

[0025] Generative Large Language Models (LLMs) are natural language processing models with pre-trained logical reasoning capabilities that utilize deep learning techniques.

[0026] Therefore, an embodiment of the present disclosure provides a model training method, in which the discrimination results and discrimination causes are used as tags to guide the initial model to perform detection based on deep features such as the logical relationship between the question text and the answer text, thereby improving the model accuracy, realizing accurate detection of sample text pairs that are difficult for the first model to detect, and obtaining high-confidence quality detection results for question-answer text pairs.

[0027] FIG. 1 schematically illustrates an exemplary system architecture to which the model training method or text detection method and apparatus according to the embodiments of the present disclosure can be applied.

[0028] 1 merely illustrates an example of a system architecture to which the embodiments of the present disclosure can be applied, and does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments, or scenes. For example, in other embodiments, an exemplary system architecture to which the model training method and apparatus can be applied may include a terminal device, but the terminal device can implement the model training method and apparatus provided by the embodiments of the present disclosure without needing to interact with a server.

[0029] 1, the system architecture 100 according to the present embodiment includes a terminal device 110, a first server 150_1, and a second server 150_2. The terminal device 110 may be various electronic devices having processing capabilities, including, but not limited to, smartphones, tablet computers, laptop portable computers, desktop computers, and smart wearable devices.

[0030] The first server 150_1 executes a training method for a first model in the model training method provided by the embodiment of the present disclosure to obtain a discriminant model 140_1. The second server 150_2 executes a training method for a target model in the model training method provided by the embodiment of the present disclosure to obtain a generative formula model 140_2.

[0031] The terminal device can load the trained discriminant model 140_1 and the trained generative model 140_2, process the question text answer text 120 based on the loaded discriminant model 140_1, and obtain a preliminary detection result. The question text answer text whose reliability in the preliminary detection result is lower than a predetermined threshold can be processed using the generative model 140_2, and obtain a detection result 130.

[0032] It should be noted that the model training method or text detection method according to the embodiments of the present disclosure may generally be performed by the terminal device 110. Accordingly, the model training device or text detection device provided by the embodiments of the present disclosure may be provided in the terminal device 110.

[0033] Alternatively, the model training method or the text detection method provided by the embodiments of the present disclosure may be generally performed by the first server 150_1 and the second server 150_2. Accordingly, the model training device or the text detection device provided by the embodiments of the present disclosure may be generally installed in the first server 150_1 and the second server 150_2. The model training method or the text detection method provided by the embodiments of the present disclosure may be performed by a server or a server cluster different from the first server 150_1 and the second server 150_2 and capable of communicating with the terminal device 110 and / or the first server 150_1 and the second server 150_2. Accordingly, the model training device or the text detection device provided by the embodiments of the present disclosure may be installed in a server or a server cluster different from the first server 150_1 and the second server 150_2 and capable of communicating with the terminal device 110 and / or the first server 150_1 and the second server 150_2.

[0034] It should be understood that the number of terminal devices and servers in Figure 1 is merely an example, and any number of terminal devices and servers may be included as desired.

[0035] In the technical solution disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure, application, etc. of such user personal information shall all comply with the provisions of relevant laws and regulations, take necessary security measures, and not violate public order and morals.

[0036] In the technical solution disclosed herein, the user's approval or consent is obtained before obtaining or collecting the user's personal information.

[0037] FIG. 2 is a schematic diagram illustrating a flowchart of a model training method according to an embodiment of the present disclosure.

[0038] As shown in FIG. 2, the method 200 includes operations S210 to S230.

[0039] In operation S210, in response to the confidence of the first sample detection result being less than a predetermined threshold, the sample text pairs and sample tags are processed using the initial model to obtain a second sample detection result.

[0040] In operation S220, a first loss value is obtained based on the second sample detection result and the sample tag according to the first loss function.

[0041] In operation S230, based on the first loss value, adjust the model parameters of the initial model to obtain a trained target model.

[0042] According to an embodiment of the present disclosure, the initial model is different in type from the first model. The initial model may be a generative model, such as a sentence mind model, a large language model (Large Language Model Meta AI, LlaMA), or a generative pre-trained transformer (GPT) model. The first model may be a discriminant model, such as a convolutional neural network model or an enhanced language representation with informative entities (ERNIE).

[0043] According to an embodiment of the present disclosure, the first model and the initial model may be trained in two stages. In the first stage, the first model may be supervisedly trained using initial sample texts to obtain a trained first model, so that the loss value between the results output by the first model and the manually labeled result tags reaches a convergence condition. The convergence condition may be a maximum number of training rounds or a predetermined loss value, and the present disclosure does not specifically limit this.

[0044] After the first stage of training is completed, the trained first model processes the initial sample text pairs to obtain first sample detection results. The first sample detection results may include classification results corresponding to the initial sample text pairs and confidence levels corresponding to the classification results. The confidence levels corresponding to the classification results may represent the classification difficulty of the first model for the initial sample text pairs.

[0045] For example, the initial sample text pairs include multiple text pairs consisting of sample question texts and sample answer texts: Q1-A1, Q2-A2, …, Q n -A n The first sample detection results obtained by processing the initial sample text pairs using the first model may include Q1-A1(I, 95%), Q2-A2(II, 55%), ..., Q n -A n (III, 75%), where I indicates that the answer text A1 does not solve the question text Q1, II indicates that the answer text A2 partially solves the question text Q2, and III indicates that the answer text A n is the question text Q n The 95%, 55%, and 75% represent the confidence levels of the detection results corresponding to each text pair by the first model, respectively. The higher the confidence level, the lower the difficulty of the first model in discriminating the text pair, and the more accurate the detection result.

[0046] Therefore, by setting a predetermined threshold, sample text pairs whose confidence is less than the predetermined threshold are screened based on the first sample detection result, and are used for training the generative formula model in the second stage.

[0047] For example, the predetermined threshold may be 80%, and the sample text pairs screened based on the first sample detection result are at least Q2-A2(II, 55%), ..., Q n -A n (III, 75%).

[0048] According to an embodiment of the present disclosure, before starting training in the second stage, the manually marked digital discrimination result tags I-III corresponding to the screened sample text pairs can be converted into corresponding text tags: <the question is not resolved>, <the question is partially resolved> and <the question is completely resolved> to meet the input needs of the generative model.

[0049] The generative model may be trained using the discrimination causes corresponding to the discrimination result tags as tags to guide the initial model to detect deep features such as logical relationships in the sample text pairs. Exemplarily, the discrimination cause tags may be manually marked based on the discrimination result tags.

[0050] For example, for the sample text pair Q2-A2, the discrimination result tag may be "partially solves the question" and the discrimination cause tag may be "A2 solves only the first part of Q2's question, and the second part of the question is not solved."

[0051] According to an embodiment of the present disclosure, the initial model can be used to process the sample text pairs and sample tags to obtain a second sample detection result. The second sample detection result may include a discrimination result representing the answer matching degree between the sample answer text and the sample question text, and a discrimination cause corresponding to the discrimination result. For example, [Partially solves the question; Cause: A2 only solves the first part of the question in Q2].

[0052] According to an embodiment of the present disclosure, a first loss value can be obtained based on the second sample detection result and the sample tag based on a first loss function. The first loss function may be a cross-entropy loss function, for example:

[0053]

number

[0054] According to an embodiment of the present disclosure, the model parameters of the initial model can be adjusted based on the first loss value until a convergence condition is met, thereby obtaining a trained target model. The convergence condition may be a maximum number of training rounds, a loss value reaching a predetermined threshold, or the loss value converging. The embodiment of the present disclosure does not specifically limit the convergence condition.

[0055] According to an embodiment of the present disclosure, the discrimination results and discrimination causes are used as tags to guide the initial model, and deep features such as the logical relationship between the question text and the answer text are used for detection, thereby improving the model accuracy, realizing accurate detection of sample text pairs that are difficult for the first model to detect, and obtaining reliable quality detection results for question-answer text pairs.

[0056] According to the embodiment of the present disclosure, the first model is fine-tuned based on sample text pairs in specific application scenarios, thereby improving the discrimination accuracy of the first model.

[0057] For example, the first model may be an Enhanced Representation from knowledge Integration (ERNIE) model, which is built based on a Transformer encoder, and the pre-training tasks performed in the pre-training stage may include a Masked Language Modeling (MLM) task and a Next Sentence Prediction (NSP) task.

[0058] In an embodiment of the present disclosure, a branch for outputting a confidence level can be added to the output layer of the ERNIE model to determine the discrimination difficulty of the ERNIE model for the input sample text pair.

[0059] Then, the preliminary training model is used to detect sample text pairs, sample detection results and confidence levels of the sample detection results are generated, and a second loss value is obtained based on the sample detection results and sample discrimination result tags according to a second loss function. The model parameters of the preliminary training model are adjusted based on the second loss value to obtain a first model.

[0060] According to an embodiment of the present disclosure, the construction format of the sample text pair may be <question text><delimiter><answer text>. The sample discrimination result tag may be manually marked.

[0061] According to an embodiment of the present disclosure, the second loss function may be a cross-entropy loss function, for example:

[0062]

number

[0063] In an embodiment of the present disclosure, the categories of sample discrimination results may include three, and the sample discrimination result tags are respectively: I - the question is not resolved; II - the question is partially resolved; and III - the question is completely resolved.

[0064] The category of the sample discrimination result can be set according to the needs of the actual application scene, and the present disclosure does not specifically limit it.

[0065] According to an embodiment of the present disclosure, the pre-training model may include at least a feature extraction layer and a feature processing layer. When adjusting the model parameters of the pre-training model based on the second loss value, the model parameters of the feature extraction layer may be fixed first, and then the model parameters of the feature processing layer may be fine-tuned to improve model training efficiency. If the convergence speed of the second loss value is slow, the model parameters of the feature extraction layer and the feature processing layer may be adjusted in parallel to improve the convergence speed of the second loss value and shorten the model training cycle.

[0066] According to an embodiment of the present disclosure, by increasing the confidence output branch, the discrimination difficulty of the sample text pairs of the first model can be determined based on the confidence, and sample text pairs with high discrimination difficulty can be screened to train a generative model.

[0067] Hereinafter, the method shown in FIG. 2 will be further described in conjunction with specific examples with reference to FIGS. 3A and 3B.

[0068] FIG. 3A illustrates a schematic diagram of model training according to an embodiment of the present disclosure.

[0069] As shown in FIG. 3A, the embodiment 300A may include an ERNIE model 310 and an LlaMA model 320.

[0070] First, we consider the Q&A (Query & Answer) text pair [Q1-A1, , Q i -A i , , Q I -A I ] 311 is input to the ERNIE model 310, which outputs a detection result 312 and a confidence level 313. The detection result 312 and confidence level 313 correspond to the Q&A text pair. For example, the detection result for the Q1-A1 text pair is I (the question is not resolved) with a confidence level of 67%. The detection result for the Q1-A1 text pair is II (the question is partially resolved) with a confidence level of 97%.

[0071] Then, based on a predetermined threshold of the reliability, for example, 90%, text pairs with a reliability lower than the predetermined threshold are screened and used as sample data for training the LlaMA model 320. j -A j , …, Q j -A j] 314 is obtained. The low confidence of the detection results corresponding to the screened Q&A text pairs indicates that it is difficult for the ERNIE model 310 to distinguish these text pairs, and accordingly, the accuracy of the detection results for these text pairs by the ERNIE model 310 is also low.

[0072] Then, we apply the manually marked tags 321 to the Q&A text pairs [Q1-A1, …, Q j -A j , …, Q j -A j ] 314 is input to the LlaMA model 320 for training, and a discrimination result + discrimination cause 323 is generated.

[0073] According to an embodiment of the present disclosure, processing the sample text pairs and the sample tags using the initial model to obtain a second sample detection result may include: stitching the sample text pairs and the sample tags to generate a sample target text; segmenting the sample target text into words to generate a sample word sequence; encoding each word in the sample word sequence according to its sequence position to generate a sample feature sequence; and processing the sample feature sequence based on an attention mechanism to generate a second sample detection result.

[0074] According to an embodiment of the present disclosure, the sample target text may be a text paragraph consisting of a sample text pair, a sample discrimination result tag, and a sample discrimination cause tag. The sample target text may be segmented into words or phrases to form a sample word sequence. The sample word sequence may then be converted into an integer index sequence based on the index values ​​in the corpus of words or phrases. The integer index sequence may then be mapped to a real number vector, and encoded based on the position of each word or phrase in the token sequence to generate a sample feature sequence.

[0075] According to an embodiment of the present disclosure, the LlaMA model 320 is built based on a Transformer decoder and uses an autoregressive approach to generate each token (text unit) in the output sequence one by one. A text unit may be a word, punctuation mark, number, or other linguistic element. During the decoding process, each time a token is generated, the previously generated content is used as context to help predict the next token. The generated token sequence is then linearly transformed by an output layer, typically using a Softmax function to convert the probability distribution at each position into a probability corresponding to the token. Depending on the probability, the token with the highest probability or the predicted result as the model is selected. During the generation task, the above autoregressive generation process is repeated for multiple tokens until an end mark (e.g., a period or end symbol) is reached or a predetermined maximum output length is reached, and a second detection result including the discrimination result and the discrimination cause is output.

[0076] According to an embodiment of the present disclosure, a tag text is generated by stitching the discrimination result tag and the discrimination cause tag, and a first loss value can be obtained based on the tag text and the second sample detection result according to a first loss function.

[0077] For example, for Q1-A1, the classification result tag may be "The question is not solved," and the classification cause may be "The answer is unrelated to the question." "The question is not solved" and "The answer is unrelated to the question" are stitched together to generate tag text "The question is not solved, and the reason is that the answer is unrelated to the question." The second detection result output by the LlaMA model 320 is text including the classification result and the classification cause, so there is no need to perform two stitching operations. For example, the second detection result may be "The question is not solved, and the reason is that the answer is unrelated to the question." Then, based on the above-mentioned formula (1), a loss value 324 is calculated from the tag text and classification result + classification cause 323.

[0078] According to an embodiment of the present disclosure, the manually marked tag 321 includes a discrimination result tag 321_1 and a discrimination cause tag 321_2. Therefore, during the training process of the LlaMA model 320, the LlaMA model 320 learns not only the discrimination result but also the discrimination cause corresponding to the discrimination result based on context learning for the input data, so that the LlaMA model 320 has the ability to infer and discriminate based on deep logic features.

[0079] In the case of quality inspection of question-answering texts, the technical field of question-answering is very broad, and manually marked discriminant causes are limited to the knowledge accumulated and subjective perception of the person involved, which results in variations in the accuracy of the marked discriminant causes. Therefore, the LlaMA model 320 trained using manually marked discriminant cause tags has a certain accuracy bottleneck.

[0080] The generative large-scale language model has strong logical inference capabilities pre-trained based on a rich corpus. Therefore, the embodiment of the present disclosure uses the discriminant causes output by the generative large-scale language model as discriminant cause tags to train the LlaMA model 320, thereby breaking through the bottleneck in accuracy of the LlaMA model 320.

[0081] According to an embodiment of the present disclosure, a presentation text can be constructed based on a sample text pair, a first discrimination result tag corresponding to the sample text pair, and a reference text, where the reference text includes a text pair related to a target sample text, a second discrimination result tag corresponding to the text pair, and a discrimination cause tag corresponding to the text pair, where the first discrimination result tag and the second discrimination result tag are identical, and a large-scale language model is used to process the presentation text to generate a second discrimination cause corresponding to the sample text pair, and the second discrimination cause is determined as the discrimination cause tag.

[0082] FIG. 3B schematically illustrates a model training diagram according to another embodiment of the present disclosure.

[0083] 3B, the embodiment 300B may include an ERNIE model 310, an LlaMA model 320, and a large-scale language model 330. The process of using the ERNIE model 310 to process Q&A text pairs 311 to obtain detection results 312, and screening Q&A text pairs 314 from the Q&A text pairs 311 for training the LlaMA model 320 based on confidence scores 313 is the same as that of the embodiment 300A, and therefore will not be described again.

[0084] Then, a presentation text 331 is constructed based on the Q&A text pair 314 obtained by screening, the manually marked classification results, and the reference text. The reference text is a text pair Q&A in a field close to the Q&A text pair 314. t -A t and text pair Q t -A t It may also include manually marked discrimination results + discrimination causes corresponding to the above.

[0085] For example, the text pair Q j -A j may be a question and answer related to the field of autonomous driving technology, or may randomly screen 1-3 text pairs from other question and answer text pairs in the field of autonomous driving technology, and mark the discrimination results and discrimination causes for the 1-3 text pairs. j -A j and text pair Q j -A j The manually marked judgment results corresponding to the above 1-3 text pairs, and the judgment results and judgment causes corresponding to the above 1-3 text pairs are called Prompt (presented text).

[0086] The tag type of the classification result for the 1-3 text pairs is text pair Q. j -A j The result of the classification is the same as the tag type. For example, for the text pair Q j -A jIf the discrimination result tag of the 1-3 text pairs is “I-Question not resolved”, the discrimination result tag type of the 1-3 text pairs is also “I-Question not resolved”, so that the generative large model can output a cause tag that matches the discrimination result based on the discrimination causes of the 1-3 text pairs.

[0087] Next, the presented text 331 is input to the large-scale language model 330, and the text pair Q j -A j The discrimination cause corresponding to the above is output, and the discrimination cause is identified as the discrimination cause tag 321_3. Then, the discrimination cause tag 321_3 is used to combine the manually marked discrimination result tag, and the LlaMA model 320 is trained. The training process is the same as in the above embodiment 300A, so the description is omitted here.

[0088] According to an embodiment of the present disclosure, the discrimination results and discrimination causes of other text pairs in a field related to the text pair are input into the large model together with reference text, and discrimination cause tags are output for training the LlaMA model 320. By fully utilizing the rich corpus and logical inference capabilities of the generative large model and learning the logical inference capabilities of the generative large model through the LlaMA model 320 in a manner similar to result distillation, the accuracy of the model can be improved and the demands on the hardware resources of the model can be reduced, allowing the LlaMA model 320 to efficiently complete the task of detecting the quality of text pairs even under conditions where hardware resources are limited.

[0089] In the context of question-answering text pair quality detection, not only must the accuracy requirements of the detection results be met, but also the hardware resource requirements of the model operation must be taken into consideration. Therefore, an embodiment of the present disclosure provides a text detection method that combines a discriminant model and a generative model.

[0090] FIG. 4 schematically illustrates a flowchart of a text detection method according to an embodiment of the present disclosure.

[0091] As shown in FIG. 4, the text detection method 400 includes operations S410 to S440.

[0092] In operation S410, a first model is used to detect a plurality of text pairs to be detected, and a first detection result and a confidence corresponding to the first detection result are obtained.

[0093] In operation S420, a target text pair to be detected is determined from the plurality of text pairs to be detected based on the confidence level.

[0094] In operation S430, the second model is used to detect the text pairs to be targeted, and a second detection result is obtained.

[0095] In operation S440, a target detection result is generated based on the first detection result and the second detection result.

[0096] According to an embodiment of the present disclosure, the first model may be a discriminant model, such as the ERNIE model, and the second model may be a target model, such as the LlaMA model, trained using the model training method described above.

[0097] According to an embodiment of the present disclosure, the plurality of text pairs to be detected include a plurality of question texts and a plurality of answer texts that correspond to each other.

[0098] According to an embodiment of the present disclosure, the first model can be used to detect all text pairs to be detected, and a first detection result and a corresponding confidence level for each text pair can be obtained. For example, the first detection result for the text pair Q1-A1 is II (partially solves the question) with a confidence level of 95%. The first detection result for the text pair Q2-A2 is II (partially solves the question) with a confidence level of 55%.

[0099] According to an embodiment of the present disclosure, the predetermined confidence threshold may be set to 80%. Because the confidence of the first detection result for the text pair Q1-A1 is greater than 80%, the first detection result for the text pair Q1-A1 can be used as the final detection result. However, because the confidence of the first detection result for the text pair Q2-A2 is less than 80%, the text pair Q2-A2 can be determined as the target text pair and used as input data for the second model. This can reduce the amount of data processing for the second model and improve detection efficiency.

[0100] According to the embodiment of the present disclosure, the second detection result obtained by detecting the target text pair using the second model includes a discrimination result category of the text pair and a discrimination cause corresponding to the discrimination result category.

[0101] For example, the second model may be used to process the text pair Q2-A2, and the second detection result obtained may be that the reason the question is unresolved is that answer A2 is unrelated to question Q2.

[0102] Finally, the first detection result output by the first model, whose reliability is greater than a predetermined threshold, and the second detection result output by the second model are integrated to obtain a target detection result, which represents the answer matching degree between the answer text and the corresponding question text.

[0103] According to the embodiments of the present disclosure, by comprehensively utilizing a discriminant model and a generative model, compared to the single discriminant model in the related examples, the problem of the discriminant model having low detection accuracy due to data overfitting can be resolved, and the robustness of the model can be improved. Furthermore, when hardware resources are limited, the generative model can be used to detect only low-confidence text pairs, thereby reducing the amount of data processing for the generative model and improving detection efficiency.

[0104] FIG. 5 illustrates a schematic diagram of text detection according to an embodiment of the present disclosure.

[0105] As shown in FIG. 5, an embodiment 500 may include an ERNIE model 510 and an LlaMA model 520 .

[0106] First, the Q&A text pair 501 "Q1-A1, Q2-A2, Q3-A3" is input to the ERNIE model 510, and the detection result 502 "Q1-A1_question partially solved_80%; Q2-A2_question completely solved_60%; Q3-A3_question not solved_50%" is output. Based on the confidence, the Q&A text pair 501 is screened to obtain the Q&A text pair 503 "Q2-A2, Q3-A3".

[0107] Then, the Q&A text pair 503 "Q2-A2, Q3-A3" is input to the LlaMA model 520, and the detection result 504 "Q2-A2_Completely resolves the question_Discriminating cause XXX; Q3-A3_Partially resolves the question_Discriminating cause YYY" is output.

[0108] For example, the target text "Q2-A2" to be detected can be segmented into words to generate a word sequence. Then, the sample word sequence can be converted into an integer index sequence based on the index values ​​in a word or phrase corpus. The integer index sequence can then be mapped to a real-number vector, and encoding can be performed based on the position of each word or phrase in the token sequence to generate a feature sequence. Next, an autoregressive method is used to generate each token (text unit) in the output sequence one by one based on an attention mechanism. A text unit can be a word, punctuation mark, number, or other linguistic element. During the decoding process, each generated token uses the previously generated content as context to help predict the next token. The generated token sequence is then converted into a probability corresponding to each position by an output layer, typically using a linear transformation and a softmax function. Depending on the probabilities, the token with the highest probability or prediction result is selected as the model. In the generation task, the above autoregressive generation process is repeated until an end mark (e.g., a period or a termination code) or a predetermined maximum output length is reached, generating multiple tokens, and the output second detection result may be "Q2-A2_Perfectly resolves the question_Distinguishing cause XXX".

[0109] Finally, the detection result 502 whose reliability is greater than a predetermined threshold is integrated with the detection result 504 to obtain the detection result 505 "Q1-A1_Partially solves the question_80%; Q2-A2_Perfectly solves the question_Discriminating cause XXX; Q3-A3_Partially solves the question_Discriminating cause YYY".

[0110] FIG. 6 is a schematic block diagram of a model training apparatus according to an embodiment of the present disclosure.

[0111] As shown in FIG. 6, the model training apparatus 600 may include a first processing module 610, a first loss calculation module 620, and a first adjustment module 630.

[0112] In response to the confidence level of the first sample detection result being less than a predetermined threshold, the first processing module 610 processes the sample text pairs and the sample tags using an initial model to obtain a second sample detection result, where the confidence level of the first sample detection result is obtained by detecting the sample text pairs using the first model, the sample text pairs include corresponding sample question texts and sample answer texts, the second sample detection result includes a discrimination result representing an answer matching degree between the sample answer texts and the sample question texts and a discrimination cause corresponding to the discrimination result, the sample tags include a discrimination result tag and a discrimination cause tag, and the initial model is a type different from the first model.

[0113] The first loss calculation module 620 obtains a first loss value based on the second sample detection result and the sample tag according to the first loss function.

[0114] The first adjustment module 630 adjusts the model parameters of the initial model based on the first loss value to obtain a trained target model.

[0115] According to an embodiment of the present disclosure, the first loss calculation module may include a first stitching sub-module and a calculation sub-module. The first stitching sub-module stitches the discrimination result tag and the discrimination cause tag to generate tag text. The calculation sub-module obtains a first loss value based on the tag text and the second sample detection result according to a first loss function.

[0116] According to an embodiment of the present disclosure, the first processing module may include a first stitching sub-module, a first word segmentation sub-module, a first encoding sub-module and a first processing sub-module.

[0117] The first stitching sub-module stitches the sample text pairs and the sample tags to generate a sample target text. The first word segmentation sub-module segments the sample target text into words to generate a sample word sequence. The first encoding sub-module encodes each word in the sample word sequence according to its sequence position to generate a sample feature sequence. The first processing sub-module processes the sample feature sequence based on an attention mechanism to generate a second sample detection result.

[0118] According to an embodiment of the present disclosure, the training device further includes a construction module, a second processing module, and a first determination module.

[0119] The construction module constructs a presentation text based on the sample text pair, a first discrimination result tag corresponding to the sample text pair, and a reference text, where the reference text includes a text pair related to the target sample text, a second discrimination result tag corresponding to the text pair, and a discrimination cause tag corresponding to the text pair, and the first discrimination result tag and the second discrimination result tag are the same.

[0120] A second processing module processes the presented text using a large-scale language model to generate second discriminant causes corresponding to the sample text pairs.

[0121] The first determination module determines the second discrimination cause as a discrimination cause tag.

[0122] According to an embodiment of the present disclosure, the training device further includes a first detection module, a second loss calculation module, and a second adjustment module. The first detection module detects sample text pairs using a pre-trained model and generates sample detection results and confidence levels for the sample detection results. The second loss calculation module obtains a second loss value based on the sample detection results and sample classification result tags according to a second loss function. The second adjustment module adjusts model parameters of the pre-trained model based on the second loss value to obtain a first model.

[0123] FIG. 7 is a schematic block diagram of a text detection device according to an embodiment of the present disclosure.

[0124] As shown in FIG. 7, the text detection device 700 may include a second detection module 710, a second determination module 720, a third detection module 730 and a generation module 740.

[0125] The second detection module 710 detects a plurality of text pairs to be detected using a first model, and obtains a first detection result and a confidence level corresponding to the first detection result, where the first model is a discriminant model, and the plurality of text pairs to be detected include a plurality of question texts and a plurality of answer texts corresponding to each other.

[0126] The second determination module 720 determines a target text pair to be detected from the plurality of text pairs to be detected based on the confidence level, and the confidence level of the detection result corresponding to the target text pair to be detected is smaller than a predetermined threshold.

[0127] The third detection module 730 detects the text pair to be targeted using the second model and obtains a second detection result, where the second model is the trained target model obtained by the aforementioned training method.

[0128] The generation module 740 generates a target detection result based on the first detection result and the second detection result, where the target detection result represents an answer matching degree between the answer text and the corresponding question text.

[0129] According to an embodiment of the present disclosure, the third detection module may include a second word segmentation sub-module, a second encoding sub-module, and a second processing sub-module.

[0130] The second word segmentation sub-module segments the target text to generate a word sequence, the second encoding sub-module encodes each word in the word sequence according to its position in the sequence to generate a feature sequence, and the second processing sub-module processes the feature sequence based on an attention mechanism to generate a second detection result.

[0131] According to an embodiment of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium, and a computer program.

[0132] According to an embodiment of the present disclosure, an electronic device includes at least one processor and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, such that the at least one processor can perform the above-described method.

[0133] According to an embodiment of the present disclosure, there is provided a non-transitory computer-readable storage medium having computer instructions stored thereon, the computer instructions causing a computer to perform the above-described method.

[0134] According to an embodiment of the present disclosure, there is provided a computer program, which when executed by a processor, implements the above method.

[0135] 8 illustrates an exemplary block diagram for implementing an example electronic device 800 according to an embodiment of the present disclosure. The electronic device is intended to represent various types of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. The electronic device may also represent various types of mobile devices, such as personal digital assistants, mobile phones, smartphones, wearable devices, and other similar computing devices. The components, their connections and relationships, and their functions illustrated herein are merely exemplary and do not limit the implementation of the present disclosure as described and / or claimed herein.

[0136] 8, device 800 includes a computing unit 801, which can perform various appropriate operations and processes based on a computer program stored in a read-only memory (ROM) 802 or loaded from a storage unit 808 into a random access memory (RAM) 803. The RAM 803 can further store various programs and data necessary for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0137] The components of the device 800 are connected to an I / O interface 805, which includes an input unit 806, such as a keyboard, a mouse, etc., an output unit 807, such as various types of displays, speakers, etc., a storage unit 808, such as a magnetic disk, an optical disk, etc., and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 enables the device 800 to exchange information / data with other devices via a computer network, such as the Internet, and / or various telecommunication networks.

[0138] The computing unit 801 may be any of a variety of general-purpose and / or specialized processing modules having processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graph processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units that execute machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the methods and processes described above, such as the model training method or the text detection method. For example, in some embodiments, the model training method or the text detection method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 808. In some embodiments, some or all of the computer program may be loaded and / or installed into the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, it may perform one or more steps of the model training method or the text detection method described above. Alternatively, in another embodiment, the computing unit 801 may be configured to perform the model training method or the text detection method in any other suitable form (eg, via firmware).

[0139] Various embodiments of the systems and techniques described herein may be implemented in digital electronic circuitry systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may be embodied in one or more computer programs that can be executed and / or interpreted by a programmable system that includes at least one programmable processor, which may be a special purpose or general purpose programmable processor, and that can receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0140] Program codes for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, so that when the program code is executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are performed. The program code may be executed entirely on a device, partially on a device, partially on a device as a separate software package and partially on a remote device, or entirely on a remote device or server.

[0141] In the context of this disclosure, a machine-readable medium may be a tangible medium, and may contain or store a program for use in or in connection with an instruction execution system, device, or electronic device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or electronic device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include an electrical connection of one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0142] To provide interaction with a user, a computer may implement the systems and techniques described herein and include a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user, and a keyboard and pointing device (e.g., a mouse or trackball) through which a user can provide input to the computer. Other types of devices may also provide interaction with a user; for example, the feedback provided to the user may be any form of sensing feedback (e.g., visual feedback, auditory feedback, or tactile feedback), and may receive input from the user in any form (including voice input, speech input, or tactile input).

[0143] The systems and techniques described herein can be implemented in a computing system including background components (e.g., a data server), or middleware components (e.g., an application server), or front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with embodiments of the systems and techniques described herein), or any combination of such background, middleware, or front-end components. The components of the system can be connected to each other by any form or medium of digital data communication (e.g., a communications network). Examples of communications networks include, by way of example, a local area network (LAN), a wide area network (WAN), and the Internet.

[0144] The computer system may include a client and a server. The client and server are generally remote and typically interact via a communication network. The relationship between the client and the server is created by computer programs running on the corresponding computers and having a client-server relationship. The server may be a cloud server, a server in a distributed system, or a server coupled to a blockchain.

[0145] It should be understood that various types of flows shown above may be used, and steps may be rearranged, added, or deleted. For example, the steps described in the present invention may be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present disclosure can be achieved, and the present specification is not limited thereto.

[0146] The specific embodiments described above do not limit the scope of protection of the present disclosure. Those skilled in the art should understand that various modifications, combinations, subcombinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principle of the present disclosure should be included within the scope of protection of the present disclosure.

Claims

1. A method for model training by a processor, comprising: In response to the confidence of the first sample detection result being less than a predetermined threshold, processing the sample text pairs and the sample tags using the initial model to obtain a second sample detection result; Obtaining a first loss value based on the second sample detection result and the sample tag according to a first loss function; adjusting model parameters of the initial model based on the first loss value to obtain a trained target model; wherein the reliability of the first sample detection result is obtained by detecting a sample text pair using a first model, the sample text pair including a corresponding sample question text and a corresponding sample answer text; the second sample detection result includes a discrimination result for representing the answer matching degree between the sample answer text and the sample question text and a discrimination cause corresponding to the discrimination result; the sample tag includes a discrimination result tag and a discrimination cause tag; and the initial model is a type different from that of the first model. Model training method.

2. Obtaining a first loss value based on the second sample detection result and the sample tag based on a first loss function includes: stitching the determination result tag and the determination cause tag to generate a tag text; obtaining the first loss value based on the tag text and the second sample detection result based on the first loss function. The method of claim 1.

3. Processing the sample text pairs and the sample tags using the initial model to obtain second sample detection results includes: stitching the sample text pairs and the sample tags to generate a sample target text; word-segmenting the sample target text to generate a sample word sequence; encoding each word in the sample word sequence according to its sequence position to generate a sample feature sequence; and processing the sample feature sequence based on an attention mechanism to generate the second sample detection result. The method of claim 1.

4. constructing a presentation text based on the sample text pairs, the first determination result tags corresponding to the sample text pairs, and a reference text; processing the submitted text using a large-scale language model to generate second discriminant causes corresponding to the sample text pairs; determining the second discrimination cause as the discrimination cause tag; Here, the reference text includes a text pair related to the target sample text, a second determination result tag corresponding to the text pair, and a determination cause tag corresponding to the text pair, and the first determination result tag is the same as the second determination result tag. The method according to any one of claims 1 to 3.

5. Detecting the sample text pairs using a pre-trained model and generating a sample detection result and a confidence score for the sample detection result; Obtaining a second loss value based on the sample detection result and the sample discrimination result tag according to a second loss function; and adjusting model parameters of the pre-trained model based on the second loss value to obtain the first model. The method according to any one of claims 1 to 3.

6. A method for detecting text by a processor, comprising: Detecting a plurality of text pairs to be detected using a first model, obtaining a first detection result and a confidence level corresponding to the first detection result, wherein the first model is a discriminant model, and the plurality of text pairs to be detected include a plurality of question texts and a plurality of answer texts corresponding to each other; determining a target text pair to be detected from the plurality of text pairs to be detected based on the confidence level, wherein the confidence level of the detection result corresponding to the target text pair to be detected is less than a predetermined threshold; Detecting the target text pair to be detected using a second model to obtain a second detection result, wherein the second model is a trained target model obtained by the training method according to any one of claims 1 to 3; generating a target detection result representing an answer matching degree between the answer text and the corresponding question text based on the first detection result and the second detection result. Text detection methods.

7. Detecting the target text pair using a second model to obtain a second detection result includes: segmenting the text to be detected into words and generating word sequences; encoding each word in the word sequence according to its sequence position to generate a feature sequence; and processing the feature sequence based on an attention mechanism to generate the second detection result. The method of claim 6.

8. a first processing module that, in response to the confidence of the first sample detection result being less than a predetermined threshold, processes the sample text pairs and the sample tags using the initial model to obtain a second sample detection result; a first loss calculation module for obtaining a first loss value based on the second sample detection result and the sample tag according to a first loss function; a first adjustment module that adjusts model parameters of the initial model based on the first loss value to obtain a trained target model; wherein the reliability of the first sample detection result is obtained by detecting a sample text pair using a first model, the sample text pair including a corresponding sample question text and a corresponding sample answer text; the second sample detection result includes a discrimination result for representing the answer matching degree between the sample answer text and the sample question text and a discrimination cause corresponding to the discrimination result; the sample tag includes a discrimination result tag and a discrimination cause tag; and the initial model is a type different from that of the first model. Model training equipment.

9. The first loss calculation module: a first stitching sub-module for stitching the discrimination result tag and the discrimination cause tag to generate a tag text; a calculation submodule for obtaining the first loss value based on the first loss function, the tag text, and the second sample detection result.

9. The apparatus of claim 8.

10. The first processing module a first stitching sub-module for stitching the sample text pairs and the sample tags to generate a sample target text; a first word segmentation sub-module that segments the sample target text into words and generates a sample word sequence; a first encoding sub-module for encoding each word in the sample word sequence according to its sequence position to generate a sample feature sequence; a first processing sub-module that processes the sample feature sequence based on an attention mechanism and generates the second sample detection result.

9. The apparatus of claim 8.

11. a construction module that constructs a presentation text based on the sample text pairs, first determination result tags corresponding to the sample text pairs, and a reference text; a second processing module that processes the submitted text using a large-scale language model to generate second discriminant causes corresponding to the sample text pairs; a first determination module for determining the second discrimination cause as a discrimination cause tag; wherein the reference text includes a text pair related to the target sample text, a second determination result tag corresponding to the text pair, and a determination cause tag corresponding to the text pair, and the first determination result tag and the second determination result tag are the same. An apparatus according to any one of claims 8 to 10.

12. a first detection module that detects the sample text pairs using a pre-trained model and generates a sample detection result and a confidence level for the sample detection result; a second loss calculation module for obtaining a second loss value based on the sample detection result and the sample discrimination result tag according to a second loss function; and a second adjustment module that adjusts model parameters of the pre-trained model based on the second loss value to obtain the first model. An apparatus according to any one of claims 8 to 10.

13. a second detection module that detects a plurality of text pairs to be detected using a first model, obtains a first detection result and a confidence level corresponding to the first detection result, wherein the first model is a discriminant model, and the plurality of text pairs to be detected include a plurality of question texts and a plurality of answer texts that correspond to each other; a second determination module for determining a target text pair to be detected from the plurality of target text pairs based on the reliability, wherein the reliability of the detection result corresponding to the target text pair to be detected is smaller than a predetermined threshold; a third detection module for detecting the target text pair using a second model to obtain a second detection result, the second model being a trained target model obtained by the training method according to any one of claims 1 to 3; a generation module that generates a target detection result representing an answer matching degree between the answer text and the corresponding question text based on the first detection result and the second detection result. Text detector.

14. The third detection module is a second word segmentation sub-module for segmenting the target text into words and generating a word sequence; a second encoding sub-module for encoding each word in the word sequence according to its position in the sequence to generate a feature sequence; a second processing sub-module that processes the feature sequence based on an attention mechanism and generates the second detection result.

14. The apparatus of claim 13.

15. at least one processor; a memory communicatively coupled to the at least one processor; The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor such that the at least one processor can perform the method of any one of claims 1-3. electronic equipment.

16. A non-transitory computer-readable storage medium having computer instructions stored thereon, comprising: The computer instructions cause the computer to carry out the method of any one of claims 1-3.

17. A computer program which, when executed by a processor, implements the method according to any one of claims 1 to 3.

Images