Question and answer pair evaluation method and device, computer device and storage medium
By acquiring question-answer pairs that are judged as negative examples, and using a question generation model to generate question information with high similarity, the problem of neural network models misjudging positive examples as negative examples is solved, and the evaluation accuracy of question-answer pairs is improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- PING AN TECH (SHENZHEN) CO LTD
- Filing Date
- 2022-06-17
- Publication Date
- 2026-05-29
Smart Images
Figure CN115269789B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of natural language processing, and more particularly to a question-and-answer pair evaluation method, apparatus, computer device, and storage medium. Background Technology
[0002] With the rapid growth of internet services, users' demand for intelligent question answering has also increased. Intelligent question answering technology refers to the technology that allows smart devices to automatically match personalized information to users' questions and provide answers.
[0003] Currently, common intelligent question answering technologies are based on intelligent question answering models. The mainstream method for training these models is to use neural networks to label the question and answer information for each question-answer pair. If the question and answer information match, the corresponding question-answer pair is considered a positive example; otherwise, it is considered a negative example. However, in practical applications, the accuracy of neural network models in identifying negative examples is not high. It frequently results in question-answer pairs that are actually positive being identified as negative examples, i.e., false negatives. The occurrence of false negatives can bias the model's learning and affect its training performance. Therefore, there is an urgent need for a method that can effectively avoid false negatives and improve the accuracy of question-answer pair identification. Summary of the Invention
[0004] The main objective of this application is to provide a question-and-answer pair evaluation method, apparatus, device, and computer storage medium, which aims to improve the accuracy of question-and-answer pair evaluation and avoid incorrectly identifying positive question-and-answer pairs as negative question-and-answer pairs.
[0005] Firstly, this application provides a question-and-answer pair evaluation method, which includes the following steps:
[0006] Obtain negative example question-answer pairs that are determined to be negative examples, wherein the negative example question-answer pairs include first question information and negative example answer information;
[0007] Based on a preset question generation model, corresponding second question information is generated according to the negative example answer information;
[0008] Determine the similarity between the first question information and the second question information corresponding to the same negative example answer information;
[0009] If the similarity is greater than a preset threshold, the negative question-answer pair will be corrected to a positive question-answer pair.
[0010] Secondly, this application also provides a question-and-answer pair evaluation device, the question-and-answer pair evaluation device comprising:
[0011] The question-answer pair acquisition module is used to acquire negative question-answer pairs that are determined to be negative examples. The negative question-answer pairs include first question information and negative answer information.
[0012] The question information generation module is used to generate corresponding second question information based on the negative example answer information, according to a preset question generation model.
[0013] The similarity determination module is used to determine the similarity between the first question information and the second question information corresponding to the same negative example answer information;
[0014] The question-answer pair correction module is used to correct negative question-answer pairs to positive question-answer pairs if the similarity is greater than a preset threshold.
[0015] Thirdly, this application also provides a computer device, the computer device including a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, it implements the question-answer pair evaluation method as described above.
[0016] Fourthly, this application also provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, it implements the question-and-answer pair evaluation method as described above.
[0017] This application provides a question-and-answer pair evaluation method, apparatus, device, and computer storage medium. The application obtains negative question-and-answer pairs identified as negative examples, each pair including first question information and negative answer information. Based on a preset question generation model, it generates corresponding second question information according to the negative answer information. It determines the similarity between the first and second question information corresponding to the same negative answer information. If the similarity is greater than a preset threshold, the negative question-and-answer pair is corrected to a positive one. By processing negative question-and-answer pairs, positive question-and-answer pairs identified as negative are filtered out and corrected, thus improving the evaluation effect of question-and-answer pairs. Attached Figure Description
[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0019] Figure 1 A flowchart illustrating a question-and-answer pair determination method provided in an embodiment of this application;
[0020] Figure 2 This is a usage scenario diagram of a question-and-answer pair determination method provided in an embodiment of this application;
[0021] Figure 3 A schematic block diagram of a question-and-answer pair determination device provided in an embodiment of this application;
[0022] Figure 4 This is a schematic block diagram of the structure of a computer device according to an embodiment of this application. Detailed Implementation
[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0024] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.
[0025] This application provides a question-and-answer pair determination method, apparatus, computer device, and computer-readable storage medium.
[0026] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.
[0027] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a question-answer pair determination method provided in an embodiment of this application. This method can be used in a terminal or server to filter negative question-answer pairs that are determined to be negative. The terminal can be an electronic device such as a mobile phone, tablet, laptop, desktop computer, personal digital assistant, or wearable device; the server can be a standalone server, a server cluster, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.
[0028] Please refer to Figure 2 , Figure 2 This is a usage scenario diagram provided by an embodiment of this application. For example... Figure 2As shown, the server stores negative example question-answer pairs, each including first question information and negative example answer information. Inputting the negative example answer information from the negative example question-answer pair into a preset question generation model yields second question information. By comparing the first question information with the second question information, the negative example question-answer pair is determined. However, this is not a limitation; for example, the negative example question-answer pair can also be stored in a terminal device, which is not specified here.
[0029] like Figure 1 As shown, the question-and-answer pair determination method includes steps S101 to S104.
[0030] Step S101: Obtain the negative example question-answer pair that is determined to be a negative example. The negative example question-answer pair includes the first question information and the negative example answer information.
[0031] For example, a question-answer pair being judged as a negative question-answer pair means that the question information and answer information in the question-answer pair are judged to be mismatched. However, the automatic judgment of negative question-answer pairs in the prior art is not accurate. It is easy to mistakenly judge positive question-answer pairs where the question information and answer information match as negative question-answer pairs, resulting in false negative examples and low accuracy of question-answer pair judgment.
[0032] For example, to improve the accuracy of question-answer pair determination, negative question-answer pairs that have been determined to be negative are obtained, so as to filter out false negative examples from the negative question-answer pairs and correct the false negative examples to positive question-answer pairs, thereby improving the determination effect and accuracy of question-answer pair determination. The negative question-answer pairs include first question information and negative answer information.
[0033] Step S102: Based on the preset question generation model, generate corresponding second question information according to the negative example answer information.
[0034] For example, the question generation model can generate question information that matches the input answer information based on the input answer information, so as to compare the generated matching question information with the first question information in the negative example question-answer pair to determine whether the negative example question-answer pair is a false negative example.
[0035] In some implementations, prior to step S102, the question-answer pair evaluation method further includes: obtaining positive question-answer pairs that are determined to be positive examples, wherein the positive question-answer pairs include positive question information and positive answer information; and training the question generation model using the positive question-answer pairs so that the question generation model can generate the corresponding positive question information based on the input positive answer information.
[0036] For example, since the accuracy of judging positive question-answer pairs is relatively high, positive question-answer pairs are used as samples to train the question generation model, thereby improving the accuracy of the question generation model in generating questions.
[0037] For example, the question generation model is trained by matching the positive question information and the positive answer information in the positive question-answer pair, so that the question generation model can generate matching question information based on the input answer information. This facilitates the subsequent comparison of the question information generated by the question generation model with the first question information in the negative question-answer pair to filter out false negatives in the negative question-answer pair.
[0038] In some implementations, step S102 includes: inputting the negative example answer information into the question generation model; and obtaining second question information generated by the question generation model based on the negative example answer information.
[0039] For example, since the question generation model can generate matching question information based on the input answer information, when the negative example answer information in the negative example question-answer pair is input into the question generation model, the question generation model can generate second question information that matches the negative example answer information, so as to compare the second question information with the first question information in the negative example question-answer pair and filter out false negative examples.
[0040] Step S103: Determine the similarity between the first question information and the second question information corresponding to the same negative example answer information.
[0041] For example, by comparing the first question information in a negative question-answer pair determined to be a negative example with the second question information generated by the question generation model, it can be determined whether the negative question-answer pair is a false negative example. Specifically, if the first question information and the second question information are similar, it indicates that the first question information and the negative example answer information should be a matching question-answer pair; conversely, if the first question information and the second question information are not similar, it indicates that the first question information and the negative example answer information do indeed constitute a negative question-answer pair.
[0042] In some implementations, step S103 includes: encoding the first question information and the second question information corresponding to the same negative example answer information into a first question vector and a second question vector, respectively, based on a preset vector encoding model; and determining the cosine similarity between the first question vector and the second question vector.
[0043] For example, determining the similarity between the first question information and the second question information corresponding to the same negative example answer information may involve encoding the first question information and the second question information to obtain a first question vector corresponding to the first question information and a second question vector corresponding to the second question information, and calculating the cosine similarity between the first question vector and the second question vector.
[0044] For example, the cosine similarity S cos =cos(E q E gen ), where E q E represents the first question vector corresponding to the first question information in a negative question-answer pair. gen This represents the second question vector corresponding to the second question information generated by the question generation model.
[0045] For example, the vector encoding model can be implemented based on the Bidirectional Auto-Regressive Transformers (BART) model, and the maximum likelihood loss function is optimized based on the seq2seq structure of the BART model:
[0046]
[0047] Where W represents the weights of the fully connected layers in the bidirectional autoregressive transformer model. This represents the output of the decoder of the bidirectional autoregressive transformer model at time t.
[0048] Of course, it is not limited to this. The problem generation model can also be implemented based on other models, which are not limited here.
[0049] Step S104: If the similarity is greater than a preset threshold, the negative question-answer pair is corrected to a positive question-answer pair.
[0050] For example, if the similarity between the first question information and the second question information is greater than the preset threshold, it means that the first question information and the negative answer information in the negative question-answer pair that is determined to be a negative example actually match each other, and the negative question-answer pair is actually a false negative example and should be corrected to a positive question-answer pair.
[0051] In some implementations, the question-and-answer pair evaluation method further includes: determining the matching score between the target question information and each of the target answer information; and sorting the target answer information corresponding to the same target question information according to the matching score.
[0052] For example, the corrected positive question-answer pair includes target question information and target answer information. One of the target question information may correspond to multiple target answer information. In order to determine the target answer information that best matches the target question information, a matching score between the target question information and each target answer information can be determined, and the target answer information can be sorted according to the matching score.
[0053] In some implementations, determining the matching score between the target question information and each of the target answer information includes: analyzing the matching degree between each of the target answer information and the target question information based on a preset question-answer matching model; determining the perplexity of each of the target answer information based on a preset perplexity determination algorithm; analyzing each of the target answer information based on the question generation model to obtain corresponding third question information; determining the similarity between each of the third question information and the target question information; and determining the matching score between each of the target answer information and the target question information based on the matching degree, the perplexity, and the similarity.
[0054] For example, the question-answering matching model can be implemented based on the Bidirectional Encoder Representation from Transformers (BERT) model. The modified target question-answer pairs are then classified into matching and non-matching categories using the Sentence BERT structure, optimizing the cross-entropy loss function.
[0055]
[0056] Among them, y i This represents the category label of the i-th question-answer pair. i A value of 0 indicates that the i-th question-answer pair does not match. i A value of 1 indicates that the i-th question-answer pair is matched; s i =softmax(W t [u,v,|uv|]), where u is the vector representation of the question information, v is the vector representation of the answer information, and square brackets indicate vector concatenation. The BERT model is trained iteratively multiple times to obtain a question-answering matching model that can be used to determine the matching degree between question and answer information in a question-answer pair.
[0057] For example, based on the question-answering matching model, the matching degree between each of the target answer information and the target question information can be determined. Specifically, the higher the matching degree, the more the target answer information matches the target question information; the lower the matching degree, the less the target answer information matches the target question information.
[0058] For example, the perplexity level is used to represent the quality of each of the target answer information. Specifically, the lower the perplexity level, the higher the quality of the target answer information; the higher the perplexity level, the lower the quality of the target answer information.
[0059] For example, the perplexity determination algorithm can be to perform maximum likelihood estimation on the perplexity using the problem generation model to obtain an estimated value of the perplexity:
[0060]
[0061] For example, to determine whether each of the target answer pieces of information matches the target question information, a third question piece of information is generated based on each of the target answer pieces of information according to the question generation model, and the similarity between each of the third question pieces of information and the target question information is calculated. Specifically, the higher the similarity, the more closely the target answer information matches the target question information.
[0062] For example, determining the matching score between each target answer information and the target question information based on the matching degree, the confusion degree, and the similarity degree can be achieved by weighted summation of the matching degree, the confusion degree, and the similarity degree, or it is not limited to this.
[0063] In some implementations, determining the similarity between each of the third question information and the target question information includes: encoding the target question information and each of the third question information based on a preset vector encoding model to obtain a target question vector and a corresponding third question vector; and determining the similarity between the target question vector and each of the third question vectors.
[0064] The question-answer pair determination method provided in the above embodiments obtains negative question-answer pairs that are determined to be negative examples. Each negative question-answer pair includes first question information and negative answer information. Based on a preset question generation model, corresponding second question information is generated according to the negative answer information. The similarity between the first question information and the second question information corresponding to the same negative answer information is determined. If the similarity is greater than a preset threshold, the negative question-answer pair is corrected to a positive question-answer pair. This method can filter out positive question-answer pairs that are determined to be negative examples, improving the accuracy of question-answer pair evaluation.
[0065] Please see Figure 3 , Figure 3 This is a schematic diagram of a question-and-answer pair determination device provided in an embodiment of this application. The question-and-answer pair determination device can be configured in a server or terminal to execute the aforementioned question-and-answer pair determination method.
[0066] like Figure 3 As shown, the question-and-answer pair determination device includes: a question-and-answer pair acquisition module 110, a question information generation module 120, a similarity determination module 130, and a question-and-answer pair correction module 140.
[0067] The question-answer pair acquisition module 110 is used to acquire negative example question-answer pairs that are determined to be negative examples, wherein the negative example question-answer pair includes first question information and negative example answer information;
[0068] The question information generation module 120 is used to generate corresponding second question information based on the negative example answer information according to a preset question generation model;
[0069] Similarity determination module 130 is used to determine the similarity between the first question information and the second question information corresponding to the same negative example answer information;
[0070] The question-answer pair correction module 140 is used to correct the negative question-answer pair to a positive question-answer pair if the similarity is greater than a preset threshold.
[0071] For example, the similarity determination module 130 includes: a vector encoding module and a cosine similarity determination module.
[0072] The vector encoding module is used to encode the first question information and the second question information corresponding to the same negative example answer information into a first question vector and a second question vector, respectively, based on a preset vector encoding model.
[0073] The cosine similarity determination module is used to determine the cosine similarity between the first question vector and the second question vector.
[0074] For example, the question-answering pair evaluation device further includes: a positive example acquisition module and a question generation model training module.
[0075] The positive example acquisition module is used to acquire positive example question-answer pairs that are determined to be positive examples. The positive example question-answer pairs include positive example question information and positive example answer information.
[0076] The question generation model training module is used to train the question generation model using the positive question-answer pairs, so that the question generation model can generate the corresponding positive question information based on the input positive answer information.
[0077] For example, the question information generation module 120 includes: an answer input module and a question acquisition module.
[0078] The answer input module is used to input the negative example answer information into the question generation model;
[0079] The question acquisition module is used to acquire the second question information generated by the question generation model based on the negative example answer information.
[0080] For example, the question-and-answer pair evaluation device further includes: a matching score determination module and an answer information sorting module.
[0081] The matching score determination module is used to determine the matching score between the target question information and each of the target answer information;
[0082] The answer information sorting module is used to sort the target answer information corresponding to the same target question information according to the matching score.
[0083] For example, the matching score determination module includes: a third question information generation module, a target question similarity determination module, a target answer matching degree determination module, a target answer perplexity determination module, and a matching score determination submodule.
[0084] The third question information generation module is used to analyze the target answer information based on the question generation model to obtain the corresponding third question information;
[0085] The target question similarity determination module is used to determine the similarity between each of the third question information and the target question information;
[0086] The target answer matching degree determination module is used to analyze the matching degree between each target answer information and the target question information based on a preset question-answer matching model;
[0087] The target answer perplexity determination module is used to determine the perplexity of each target answer information based on a preset perplexity determination algorithm.
[0088] The matching score determination submodule is used to determine the matching score between each of the target answer information and the target question information based on the similarity, the matching degree, and the perplexity.
[0089] For example, the target question similarity determination module includes: a target question encoding module and a target question similarity determination submodule.
[0090] The target question encoding module is used to encode the target question information and each of the third question information based on a preset vector encoding model, so as to obtain the target question vector and the corresponding third question vector.
[0091] The target question similarity determination submodule is used to determine the similarity between the target question vector and each of the third question vectors.
[0092] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the above-described apparatus and its modules and units can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0093] The methods and apparatus of this application can be used in a wide variety of general-purpose or special-purpose computing system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.
[0094] For example, the above-described method and apparatus can be implemented as a computer program, which can be used in, for example... Figure 4 It runs on the computer device shown.
[0095] Please see Figure 4 , Figure 4 This is a schematic block diagram illustrating the structure of a computer device provided in an embodiment of this application. The computer device may be a server or a terminal.
[0096] like Figure 4 As shown, the computer device includes a processor, a memory, and a network interface connected via a system bus, wherein the memory may include a storage medium and internal memory.
[0097] The storage medium may store an operating system and a computer program. The computer program includes program instructions that, when executed, cause the processor to perform any question-and-answer pair determination method.
[0098] The processor provides computing and control capabilities, supporting the operation of the entire computer device.
[0099] Internal memory provides an environment for the execution of computer programs stored in storage media. When these computer programs are executed by a processor, the processor can perform any question-and-answer pair determination method.
[0100] This network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0101] It should be understood that the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.
[0102] In one embodiment, the processor is configured to run a computer program stored in memory to perform the following steps:
[0103] Obtain negative example question-answer pairs that are determined to be negative examples, wherein the negative example question-answer pairs include first question information and negative example answer information;
[0104] Based on a preset question generation model, corresponding second question information is generated according to the negative example answer information;
[0105] Determine the similarity between the first question information and the second question information corresponding to the same negative example answer information;
[0106] If the similarity is greater than a preset threshold, the negative question-answer pair will be corrected to a positive question-answer pair.
[0107] In one embodiment, when the processor performs the step of determining the similarity between the first question information and the second question information corresponding to the same negative example answer information, it is configured to:
[0108] Based on a preset vector encoding model, the first question information and the second question information corresponding to the same negative example answer information are respectively encoded into a first question vector and a second question vector;
[0109] Determine the cosine similarity between the first question vector and the second question vector.
[0110] In one embodiment, before the processor implements the second question information based on the negative example answer information and generates the corresponding question information according to the preset question generation model, it is used to perform the following:
[0111] Obtain positive example question-answer pairs that are determined to be positive examples, wherein the positive example question-answer pairs include positive example question information and positive example answer information;
[0112] The question generation model is trained using the positive example question-answer pairs so that it can generate corresponding positive example question information based on the input positive example answer information.
[0113] In one embodiment, when the processor implements the preset question generation model and generates corresponding second question information based on the negative example answer information, it is used to:
[0114] Input the negative example answer information into the question generation model;
[0115] Obtain the second question information generated by the question generation model based on the negative example answer information.
[0116] In one embodiment, after implementing the step of correcting the negative example question-answer pair to a positive example question-answer pair, the processor is configured to implement:
[0117] Determine the matching score between the target question information and each of the target answer information;
[0118] Based on the matching score, the target answer information corresponding to the same target question information is sorted.
[0119] In one embodiment, when the processor determines the matching scores between the target question information and each of the target answer information, it is configured to:
[0120] Based on the question generation model, the target answer information is analyzed to obtain the corresponding third question information;
[0121] Determine the similarity between each of the third problem information and the target problem information;
[0122] The matching degree between the target answer information and the target question information is analyzed based on a preset question-answer matching model.
[0123] Based on a preset perplexity determination algorithm, the perplexity of each target answer information is determined;
[0124] Based on the similarity, the matching degree, and the perplexity, a matching score is determined between each of the target answer information and the target question information.
[0125] In one embodiment, when the processor performs the step of determining the similarity between each of the third problem information and the target problem information, it is configured to:
[0126] Based on a preset vector encoding model, the target problem information and each of the third problem information are encoded respectively to obtain the target problem vector and the corresponding third problem vector;
[0127] Determine the similarity between the target question vector and each of the third question vectors.
[0128] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the question-and-answer pair determination described above can be referred to the corresponding process in the aforementioned question-and-answer pair determination control method embodiment, and will not be repeated here.
[0129] This application also provides a computer-readable storage medium storing a computer program, the computer program including program instructions, and the method implemented when the program instructions are executed can be referred to various embodiments of the question-and-answer pair determination method of this application.
[0130] The computer-readable storage medium can be an internal storage unit of the computer device described in the foregoing embodiments, such as the hard disk or memory of the computer device. The computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the computer device.
[0131] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.
[0132] It should also be understood that the term "and / or" as used in this specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. It should be noted that, herein, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.
[0133] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. The above descriptions are merely specific implementations of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A question-and-answer pair evaluation method, characterized in that, The method includes: Obtain negative example question-answer pairs that are determined to be negative examples, wherein the negative example question-answer pairs include first question information and negative example answer information; Based on a preset question generation model, corresponding second question information is generated according to the negative example answer information; Determine the similarity between the first question information and the second question information corresponding to the same negative example answer information; If the similarity is greater than a preset threshold, the negative question-answer pair will be corrected to a positive question-answer pair; The corrected positive question-answer pair includes target question information and multiple target answer information corresponding to the target question information; after correcting the negative question-answer pair into a positive question-answer pair, the process includes: determining the matching score between the target question information and each of the target answer information; and sorting the target answer information corresponding to the same target question information according to the matching score. Determining the matching score between the target question information and each of the target answer information includes: The matching degree between the target answer information and the target question information is analyzed based on a preset question-answer matching model. Based on a preset perplexity determination algorithm, the perplexity of each target answer information is determined; Based on the question generation model, the target answer information is analyzed to obtain the corresponding third question information; Determine the similarity between each of the third problem information and the target problem information; Based on the matching degree, the perplexity degree, and the similarity degree, a matching score is determined between each of the target answer information and the target question information.
2. The question-and-answer pair evaluation method according to claim 1, characterized in that, Determining the similarity between the first question information and the second question information corresponding to the same negative example answer information includes: Based on a preset vector encoding model, the first question information and the second question information corresponding to the same negative example answer information are respectively encoded into a first question vector and a second question vector; Determine the cosine similarity between the first question vector and the second question vector.
3. The question-and-answer pair evaluation method according to claim 1, characterized in that, Before generating the corresponding second question information based on the negative example answer information according to the preset question generation model, the method further includes: Obtain positive example question-answer pairs that are determined to be positive examples, wherein the positive example question-answer pairs include positive example question information and positive example answer information; The question generation model is trained using the positive example question-answer pairs so that it can generate corresponding positive example question information based on the input positive example answer information.
4. The question-and-answer pair evaluation method according to claim 1, characterized in that, The preset question generation model generates corresponding second question information based on the negative example answer information, including: Input the negative example answer information into the question generation model; Obtain the second question information generated by the question generation model based on the negative example answer information.
5. The question-and-answer pair evaluation method according to claim 2, characterized in that, The method further includes: Based on the seq2seq structure of the bidirectional autoregressive transformer model, the maximum likelihood loss function is optimized to obtain the vector encoding model.
6. The question-and-answer pair evaluation method according to claim 1, characterized in that, The perplexity level is used to represent the quality of each target answer information, wherein the lower the perplexity level, the higher the quality of the target answer information; and the higher the perplexity level, the lower the quality of the target answer information.
7. The question-and-answer pair evaluation method according to claim 1, characterized in that, The step of determining the similarity between each of the third question information and the target question information includes: Based on a preset vector encoding model, the target problem information and each of the third problem information are encoded respectively to obtain the target problem vector and the corresponding third problem vector; Determine the similarity between the target question vector and each of the third question vectors.
8. A question-and-answer pair quality assessment device, characterized in that, The question-and-answer pair quality assessment device includes: The question-answer pair acquisition module is used to acquire negative question-answer pairs that are determined to be negative examples. The negative question-answer pairs include first question information and negative answer information. The question information generation module is used to generate corresponding second question information based on the negative example answer information, according to a preset question generation model. The similarity determination module is used to determine the similarity between the first question information and the second question information corresponding to the same negative example answer information; The question-answer pair correction module is used to correct negative question-answer pairs to positive question-answer pairs if the similarity is greater than a preset threshold. The corrected positive question-answer pair includes target question information and multiple target answer information corresponding to the target question information; after correcting the negative question-answer pair into a positive question-answer pair, the process includes: determining the matching score between the target question information and each of the target answer information; and sorting the target answer information corresponding to the same target question information according to the matching score. Determining the matching score between the target question information and each of the target answer information includes: The matching degree between the target answer information and the target question information is analyzed based on a preset question-answer matching model. Based on a preset perplexity determination algorithm, the perplexity of each target answer information is determined; Based on the question generation model, the target answer information is analyzed to obtain the corresponding third question information; Determine the similarity between each of the third problem information and the target problem information; Based on the matching degree, the perplexity degree, and the similarity degree, a matching score is determined between each of the target answer information and the target question information.
9. A computer device, characterized in that, The computer device includes a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, it implements the steps of the question-answer pair evaluation method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, it implements the steps of the question-answer pair evaluation method as described in any one of claims 1 to 7.