Question and answer pair correlation evaluation method, electronic equipment and storage medium

Abstract

The invention provides a question and answer pair correlation evaluation method, electronic equipment and a storage medium, the method comprises the following steps: obtaining a to-be-evaluated question and answer pair, the to-be-evaluated question and answer pair comprising a to-be-evaluated question and a to-be-evaluated answer corresponding to the to-be-evaluated question, the to-be-evaluated answer is generated according to the answer of the question answering system to the to-be-evaluated question; based on the to-be-evaluated answers, a plurality of variant questions are generated, and the variant questions are questions corresponding to the to-be-evaluated answers and different from the to-be-evaluated questions; and determining the correlation of the question-answer pair to be evaluated based on each variant question and the question to be evaluated. The questions and answers can be accurately and efficiently evaluated, so that a foundation is laid for providing a questions and answers system with good questions and answers use experience for a user.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and particularly to a method for evaluating the relevance of question-and-answer pairs, an electronic device, and a storage medium. Background Art

[0002] In recent years, with the rapid progress of artificial intelligence technology, especially the wide application of large language models, question-and-answer systems have become powerful assistants for many enterprises and application systems to handle customer questions. In practical applications, the performance and accuracy of question-and-answer systems are directly related to the quality of the user experience.

[0003] In order to provide users with a good sense of use experience, evaluating the relevance of question-and-answer pairs in question-and-answer systems has become the focus of attention. Therefore, finding a method for evaluating the relevance of question-and-answer pairs that can accurately and efficiently evaluate question-and-answer pairs has become a current research hotspot. Summary of the Invention

[0004] The present invention provides a method for evaluating the relevance of question-and-answer pairs, an electronic device, and a storage medium, which can accurately and efficiently evaluate question-and-answer pairs, thereby laying a foundation for a question-and-answer system that can provide users with a good question-and-answer use experience.

[0005] The present invention provides a method for evaluating the relevance of question-and-answer pairs, the method comprising: obtaining a question-and-answer pair to be evaluated, wherein the question-and-answer pair to be evaluated includes a question to be evaluated and a corresponding answer to be evaluated for the question to be evaluated, and the answer to be evaluated is generated according to the answer of the question-and-answer system to the question to be evaluated; generating a plurality of variant questions based on the answer to be evaluated, wherein the variant questions are questions different from the question to be evaluated and that can be answered with the answer to be evaluated; and determining the relevance of the question-and-answer pair to be evaluated based on each of the variant questions and the question to be evaluated.

[0006] According to the method for evaluating the relevance of question-and-answer pairs provided by the present invention, the determining the relevance of the question-and-answer pair to be evaluated based on each of the variant questions and the question to be evaluated specifically includes: respectively obtaining various similarities between each of the variant questions and the question to be evaluated based on each of the variant questions and the question to be evaluated; and determining the relevance of the question-and-answer pair to be evaluated based on the various similarities.

[0007] According to the method for evaluating the relevance of question-and-answer pairs provided by the present invention, the similarity includes cosine similarity; the determining the relevance of the question-and-answer pair to be evaluated based on the various similarities specifically includes: performing a normalization transformation process on each of the cosine similarities to obtain a transformed similarity, wherein the similarity value of the transformed similarity is within a preset interval; performing an averaging process on the similarity values of each of the transformed similarities to obtain a target similarity value; and determining the relevance of the question-and-answer pair to be evaluated based on the target similarity value.

[0008] According to a method for evaluating the relevance of question-and-answer pairs provided by the present invention, the normalization transformation process is performed on each of the cosine similarities to obtain the transformed similarity, which specifically includes: performing a process of adding a preset value to each of the cosine similarities to obtain the increased cosine similarity; performing a logarithmic normalization transformation process on the increased cosine similarity to obtain the transformed similarity.

[0009] According to a method for evaluating the relevance of question-and-answer pairs provided by the present invention, based on the target similarity value, determining the relevance of the question-and-answer pair to be evaluated specifically includes: in the case where the target similarity value is greater than or equal to the similarity threshold, determining that the relevance of the question-and-answer pair to be evaluated meets the relevance requirement; in the case where the target similarity value is less than the similarity threshold, determining that the relevance of the question-and-answer pair to be evaluated does not meet the relevance requirement.

[0010] According to a method for evaluating the relevance of question-and-answer pairs provided by the present invention, after determining that the relevance of the question-and-answer pair to be evaluated does not meet the relevance requirement, the method further includes: sending an alarm reminder, where the alarm reminder is used to prompt the user that the relevance of the question-and-answer pair to be evaluated does not meet the relevance requirement.

[0011] According to a method for evaluating the relevance of question-and-answer pairs provided by the present invention, generating a plurality of variant questions based on the answer to be evaluated specifically includes: calling a pre-configured generative pre-training model; generating target text prompt information corresponding to the answer to be evaluated based on the answer to be evaluated, where the target text prompt information is used to guide the generative pre-training model to generate prompt information for variant questions with preset requirements; inputting the target text prompt information into the generative pre-training model to obtain a plurality of the variant questions corresponding to the answer to be evaluated output by the generative pre-training model.

[0012] The present invention also provides a device for evaluating the relevance of question-and-answer pairs, where the device includes: an acquisition module, configured to acquire a question-and-answer pair to be evaluated, where the question-and-answer pair to be evaluated includes a question to be evaluated and an answer to be evaluated corresponding to the question to be evaluated, and the answer to be evaluated is generated according to the answer of the question-and-answer system to the question to be evaluated; a processing module, configured to generate a plurality of variant questions based on the answer to be evaluated, where the variant questions are questions different from the question to be evaluated and that can be answered with the answer to be evaluated; a determination module, configured to determine the relevance of the question-and-answer pair to be evaluated based on each of the variant questions and the question to be evaluated.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the method for evaluating the relevance of a question-answer pair as described in any one of the above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method for evaluating the relevance of a question-answer pair as described in any one of the above is implemented.

[0015] The present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, the method for evaluating the relevance of a question-answer pair as described in any one of the above is implemented.

[0016] The method, device, electronic device, and storage medium for evaluating the relevance of a question-answer pair provided by the present invention obtain a question-answer pair to be evaluated. The question-answer pair to be evaluated includes a question to be evaluated and an answer to be evaluated corresponding to the question to be evaluated, and the answer to be evaluated is generated according to the answer of the question-answering system to the question to be evaluated. Based on the answer to be evaluated, a plurality of variant questions are generated, where the variant questions are questions different from the question to be evaluated and that can be answered by the answer to be evaluated. Based on each variant question and the question to be evaluated, the relevance of the question-answer pair to be evaluated is determined. It realizes the ability to accurately and efficiently evaluate the question-answer pair, thereby laying a foundation for a question-answering system that can provide users with a good question-answering experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0018] Figure 1 is a flowchart of the method for evaluating the relevance of a question-answer pair provided by the present invention.

[0019] Figure 2 is a flowchart of determining the relevance of the question-answer pair to be evaluated based on each of the variant questions and the question to be evaluated provided by the present invention.

[0020] Figure 3 is a flowchart of determining the relevance of the question-answer pair to be evaluated based on each similarity provided by the present invention.

[0021] Figure 4 is a flowchart of performing a normalization transformation process on each of the cosine similarities to obtain a transformed similarity provided by the present invention.

[0022] Figure 5 It is a schematic flowchart of a process for generating multiple variant questions based on the to-be-evaluated answer provided by the present invention.

[0023] Figure 6 It is a schematic structural diagram of a question-answer pair relevance evaluation device provided by the present invention.

[0024] Figure 7 It is a schematic structural diagram of an electronic device provided by the present invention. Specific Embodiments

[0025] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Obviously, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without any creative efforts shall fall within the scope of protection of the present invention.

[0026] The question-answer pair relevance evaluation method provided by the present invention can combine multi-dimensional relevance calculations to achieve accurate evaluation of the performance of a technical question-answer system (such as a RAG system). Furthermore, it can help evaluate the semantic and logical fit between the answer generated by the evaluation system and the original question, thereby effectively improving the performance of the question-answer system in actual application scenarios.

[0027] Figure 1 It is a schematic flowchart of the question-answer pair relevance evaluation method provided by the present invention.

[0028] The following will be combined with Figure 1 to illustrate the process of the question-answer pair relevance evaluation method provided by the present invention.

[0029] In an exemplary embodiment of the present invention, in combination with Figure 1 it can be seen that the question-answer pair relevance evaluation method may include steps 110 to 130, and each step will be introduced separately below.

[0030] In step 110, a to-be-evaluated question-answer pair is obtained, where the to-be-evaluated question-answer pair includes a to-be-evaluated question and a to-be-evaluated answer corresponding to the to-be-evaluated question, and the to-be-evaluated answer is generated according to the answer of the question-answer system to the to-be-evaluated question.

[0031] In one embodiment, a to-be-evaluated question-answer pair can be obtained. The to-be-evaluated question-answer pair may include a corresponding set of to-be-evaluated question and to-be-evaluated answer. It can be understood that the to-be-evaluated answer is obtained based on the answer of the question-answer system to the to-be-evaluated question. The relevance between the to-be-evaluated answer and the to-be-evaluated question can characterize the answer effectiveness of the question-answer system.

[0032] In step 120, based on the answer to be evaluated, multiple variant questions are generated, where the variant questions are questions that are different from the question to be evaluated and can be answered using the answer to be evaluated.

[0033] In step 130, based on each variant question and the question to be evaluated, the relevance of the question-and-answer pair to be evaluated is determined.

[0034] In yet another embodiment, multiple variant questions can be generated based on the answer to be evaluated. Among them, the variant questions can be questions corresponding to the answer to be evaluated and different from the question to be evaluated. In other words, the variant questions are questions that can be answered using the answer to be evaluated and are different from the question to be evaluated. Further, based on the relevance between each variant question and the question to be evaluated, the relevance of the question-and-answer pair to be evaluated is determined. It can be understood that the variant questions can be variant questions related to the question to be evaluated, that is, questions corresponding to the answer to be evaluated in multiple dimensions. Since the variant questions can not only capture the diversity of the questions corresponding to the answer to be evaluated, but also improve the system's understanding ability for complex questions, therefore, based on the relevance between each variant question and the question to be evaluated, determining the relevance of the question-and-answer pair to be evaluated can achieve a more accurate assessment of the answer quality of the question-and-answer system.

[0035] The method for evaluating the relevance of a question-and-answer pair provided by the present invention obtains a question-and-answer pair to be evaluated, where the question-and-answer pair to be evaluated includes a question to be evaluated and an answer to be evaluated corresponding to the question to be evaluated, and the answer to be evaluated is generated according to the answer of the question-and-answer system to the question to be evaluated; based on the answer to be evaluated, multiple variant questions are generated, where the variant questions are questions that are different from the question to be evaluated and can be answered using the answer to be evaluated; based on the relevance between each variant question and the question to be evaluated, the relevance of the question-and-answer pair to be evaluated is determined. It realizes the ability to accurately and efficiently evaluate the question-and-answer pair, thus laying a foundation for a question-and-answer system that can provide a good question-and-answer usage experience for users.

[0036] Figure 2 It is a schematic flowchart of determining the relevance of the question-and-answer pair to be evaluated based on each of the variant questions and the question to be evaluated provided by the present invention.

[0037] Next, it will be combined with Figure 2 to illustrate the process of determining the relevance of the question-and-answer pair to be evaluated based on each of the variant questions and the question to be evaluated.

[0038] In an exemplary embodiment of the present invention, combined with Figure 2 it can be seen that determining the relevance of the question-and-answer pair to be evaluated based on each of the variant questions and the question to be evaluated may include steps 210 to 220, and each step will be introduced separately below.

[0039] In step 210, based on each of the variant questions and the question to be evaluated, respective similarities between each of the variant questions and the question to be evaluated are obtained. In step 220, based on the respective similarities, the relevance of the question-and-answer pair to be evaluated is determined.

[0040] In one embodiment, based on each of the variant questions and the question to be evaluated, the similarities between each of the variant questions and the question to be evaluated can be obtained. It can be understood that since the variant questions can represent questions in different dimensions, then, based on the similarities between each of the variant questions and the question to be evaluated, the similarities in different dimensions can also be characterized. Furthermore, based on the respective similarities, the relevance of the question-and-answer pair to be evaluated can be evaluated from different dimensions, so that the comprehensiveness and rationality of the relevance of the question-and-answer pair to be evaluated obtained can be improved.

[0041] Figure 3 It is a schematic flowchart of the process for determining the relevance of the question-and-answer pair to be evaluated based on the respective similarities provided by the present invention.

[0042] Next, in conjunction with Figure 3 the process of determining the relevance of the question-and-answer pair to be evaluated based on the respective similarities will be described.

[0043] In an exemplary embodiment of the present invention, the similarity may include cosine similarity. In conjunction with Figure 3 it can be seen that determining the relevance of the question-and-answer pair to be evaluated based on the respective similarities may include steps 310 to 330, and each step will be introduced separately below.

[0044] In step 310, normalization transformation processing is performed on each of the cosine similarities to obtain the transformed similarity, wherein the similarity value of the transformed similarity is within a preset interval.

[0045] In step 320, an averaging process is performed on the similarity values of each of the transformed similarities to obtain a target similarity value.

[0046] In step 330, based on the target similarity value, the relevance of the question-and-answer pair to be evaluated is determined.

[0047] In one embodiment, each cosine similarity can be subjected to a normalization transformation process to obtain a transformed similarity, where the similarity value of the transformed similarity is within a preset range. It should be noted that the preset range can be adjusted according to the actual situation. For example, it can be a range greater than 0. Further, the similarity values of each transformed similarity are averaged to obtain a target similarity value, and based on the target similarity value, the relevance of the question-and-answer pair to be evaluated is determined. In this embodiment, normalizing the cosine similarities to the same preset range can ensure that each transformed similarity is within the same evaluation range, and thus can ensure the accuracy and rationality of evaluating the relevance of the question-and-answer pair to be evaluated based on the target similarity value determined from each transformed similarity.

[0048] Figure 4 FIG. is a schematic flowchart of the normalization transformation process for each of the cosine similarities provided by the present invention to obtain the transformed similarity.

[0049] The following will be combined with Figure 4 The process of normalizing each of the cosine similarities to obtain the transformed similarity will be described.

[0050] In an exemplary embodiment of the present invention, in combination with Figure 4 It can be seen that the process of normalizing each of the cosine similarities to obtain the transformed similarity may include step 410 and step 420, and each step will be introduced separately below.

[0051] In step 410, a preset value is added to each cosine similarity to obtain an incremented cosine similarity.

[0052] In step 420, a logarithmic normalization transformation is performed on the incremented cosine similarity to obtain the transformed similarity.

[0053] In one embodiment, a preset value can be added to each cosine similarity to obtain an incremented cosine similarity. In one example, the preset value can be 2. During application, the cosine similarity can be processed by adding 2, so as to obtain the incremented cosine similarity, that is, the cosine similarity + 2. Further, a logarithmic normalization transformation is performed on the incremented cosine similarity to obtain the transformed similarity. That is, the transformed similarity can be expressed as log(cosine similarity + 2). Through this embodiment, it can be ensured that the transformed similarity is within the same evaluation range, for example, within the range greater than 0, and thus can ensure the accuracy and rationality of evaluating the relevance of the question-and-answer pair to be evaluated based on the target similarity value determined from each transformed similarity, and avoid the situation where the target similarity value is offset due to the positive and negative values of the transformed similarity.

[0054] To address the limitation that the cosine similarity ranges from -1 to +1, in this embodiment, a smoothing transformation is innovatively introduced. Through the process of log(similarity + 2), the similarity value is projected onto a wider interval (from 1 to infinity), avoiding the influence of negative values and ensuring the stability and rationality of the calculation results.

[0055] In another exemplary embodiment of the present invention, continuing with the embodiment described above Figure 3 as an example for illustration, where, based on the target similarity value, determining the relevance of the question-and-answer pair to be evaluated (corresponding to step 330) can be achieved in the following manner: When the target similarity value is greater than or equal to the similarity threshold, it is determined that the relevance of the question-and-answer pair to be evaluated meets the relevance requirement; When the target similarity value is less than the similarity threshold, it is determined that the relevance of the question-and-answer pair to be evaluated does not meet the relevance requirement.

[0056] In one embodiment, when it is determined that the target similarity value is greater than or equal to the similarity threshold, it indicates that the similarity between each variant question determined based on the answer to be evaluated and the question to be evaluated is very high, and it also indicates that the relevance between the answer to be evaluated and the question to be evaluated is very high. Therefore, it can be determined that the relevance of the question-and-answer pair to be evaluated meets the relevance requirement.

[0057] In another embodiment, when it is determined that the target similarity value is less than the similarity threshold, it indicates that the similarity between each variant question determined based on the answer to be evaluated and the question to be evaluated is not high, and it also indicates that the relevance between the answer to be evaluated and the question to be evaluated is not high. Therefore, it can be determined that the relevance of the question-and-answer pair to be evaluated does not meet the relevance requirement. The similarity threshold can be adjusted according to the actual situation, and in this embodiment, no specific limitation is imposed on the similarity threshold.

[0058] In another exemplary embodiment of the present invention, continuing with the embodiment described above as an example for illustration, where, after it is determined that the relevance of the question-and-answer pair to be evaluated does not meet the relevance requirement, the method for evaluating the relevance of the question-and-answer pair may further include the following steps: Send an alarm reminder, where the alarm reminder is used to prompt the user that the relevance of the question-and-answer pair to be evaluated does not meet the relevance requirement.

[0059] In one embodiment, after it is determined that the relevance of the question-and-answer pair to be evaluated does not meet the relevance requirement, an alarm reminder can be sent to prompt the user that the relevance of the question-and-answer pair to be evaluated does not meet the relevance requirement. In this scenario, the user can be reminded to debug the accuracy of the question-and-answer system.

[0060] Figure 5 is a schematic flowchart of the process for generating multiple variant questions based on the answer to be evaluated provided by the present invention.

[0061] The following will be combined with Figure 5 to describe the process of generating multiple variant questions based on the answer to be evaluated.

[0062] In an exemplary embodiment of the present invention, combined with Figure 5 it can be seen that generating multiple variant questions based on the answer to be evaluated may include steps 510 to 530, and each step will be introduced separately below.

[0063] In step 510, a pre-configured generative pre-training model is called.

[0064] In step 520, based on the answer to be evaluated, target text prompt information corresponding to the answer to be evaluated is generated, where the target text prompt information is used to guide the generative pre-training model to generate prompt information for variant questions with preset requirements.

[0065] In step 530, the target text prompt information is input into the generative pre-training model to obtain multiple variant questions corresponding to the answer to be evaluated output by the generative pre-training model.

[0066] In one embodiment, a pre-configured generative pre-training model (also known as the GPT model) can be called; further, based on the answer to be evaluated, target text prompt information (also known as prompt) corresponding to the answer to be evaluated is generated. Among them, the target text prompt information can be used to guide the generative pre-training model to generate prompt information for variant questions with preset requirements. Then, the target text prompt information can be input into the generative pre-training model to obtain multiple variant questions corresponding to the answer to be evaluated output by the generative pre-training model.

[0067] It should be noted that the target text prompt information is used to guide the generative pre-training model to generate variant questions with preset requirements, where the preset requirements can be adjusted according to the actual situation and are not specifically limited in this embodiment. For example, the preset requirements may include the following: Direct relevance: The generated questions (corresponding to variant questions) must be directly answerable by the given answers (corresponding to the answer to be evaluated), and the focus of the questions should match the core content of the answers. Questions that are irrelevant or too broad to the answer content should not be generated; Diversity: At least 3 different forms of questions (corresponding to variant questions) should be generated for each answer (corresponding to the answer to be evaluated). The core semantics of the questions should remain the same, but the expression methods need to be changed to cover different ways of asking; Avoid redundancy: The questions (corresponding to variant questions) should avoid being completely repeated or too similar. Try to use different sentence patterns and wordings to form a diverse set of questions; Maintain the consistency of the answer: The generated questions (corresponding variant questions) should be highly consistent with the details provided in the answer (corresponding to the answer to be evaluated), ensuring that they do not contain information inconsistent with or deviating from the answer content; Be concise and clear: The questions (corresponding variant questions) should be concise and to the point, ensuring that the intention of the questions is clear and explicit.

[0068] In one example, the answer to be evaluated can be "In Java, HashMap is a collection class based on a hash table that allows storing key-value pairs and can quickly find the corresponding value by the key." Then, multiple variant questions corresponding to the answer to be evaluated output by the generative pre-trained model can be: "What is HashMap in Java"; "How to use HashMap to store key-value pairs in Java"; "In Java, how does HashMap find the value by the key", etc.

[0069] It should be noted that the target text prompt information (also known as prompt) can be in Chinese version, or can be translated into English version for use according to the model's sensitivity to language, so as to give full play to the model's performance and improve the accuracy of the entire evaluation.

[0070] According to the description above, the method for evaluating the relevance of question-answer pairs provided by the present invention obtains the question-answer pairs to be evaluated, where the question-answer pairs to be evaluated include the questions to be evaluated and the answers to be evaluated corresponding to the questions to be evaluated, and the answers to be evaluated are generated according to the answers of the question-answering system to the questions to be evaluated; based on the answers to be evaluated, multiple variant questions are generated, where the variant questions are questions different from the questions to be evaluated and can be answered by the answers to be evaluated; based on each variant question and the question to be evaluated, the relevance of the question-answer pairs to be evaluated is determined. It realizes the ability to accurately and efficiently evaluate question-answer pairs, thus laying a foundation for a question-answering system that can provide users with a good question-answering experience.

[0071] The following describes the device for evaluating the relevance of question-answer pairs provided by the present invention. The device for evaluating the relevance of question-answer pairs described below can be mutually corresponding and referred to with the method for evaluating the relevance of question-answer pairs described above.

[0072] Figure 6 It is a schematic structural diagram of the device for evaluating the relevance of question-answer pairs provided by the present invention.

[0073] In an exemplary embodiment of the present invention, in combination with Figure 6 it can be seen that the device for evaluating the relevance of question-answer pairs may include an acquisition module 610, a processing module 620, and a determination module 630. Each module will be introduced separately below.

[0074] An acquisition module 610 can be configured to acquire a question-and-answer pair to be evaluated, where the question-and-answer pair to be evaluated includes a question to be evaluated and a corresponding answer to be evaluated for the question to be evaluated, and the answer to be evaluated is generated according to the answer of the question-and-answer system to the question to be evaluated; A processing module 620 can be configured to generate a plurality of variant questions based on the answer to be evaluated, where the variant questions are questions different from the question to be evaluated and can be answered by the answer to be evaluated; A determination module 630 can be configured to determine the relevance of the question-and-answer pair to be evaluated based on each of the variant questions and the question to be evaluated.

[0075] In an exemplary embodiment of the present invention, the determination module 630 can implement determining the relevance of the question-and-answer pair to be evaluated based on each of the variant questions and the question to be evaluated in the following manner: Based on each of the variant questions and the question to be evaluated, respective similarities between each of the variant questions and the question to be evaluated are obtained; Based on the respective similarities, the relevance of the question-and-answer pair to be evaluated is determined.

[0076] In an exemplary embodiment of the present invention, the similarity includes cosine similarity; The determination module 630 can implement determining the relevance of the question-and-answer pair to be evaluated based on the respective similarities in the following manner: Perform a normalization transformation process on each of the cosine similarities to obtain transformed similarities, where the similarity values of the transformed similarities are within a preset range; Perform an averaging process on the similarity values of each of the transformed similarities to obtain a target similarity value; Based on the target similarity value, determine the relevance of the question-and-answer pair to be evaluated.

[0077] In an exemplary embodiment of the present invention, the determination module 630 can implement performing a normalization transformation process on each of the cosine similarities to obtain transformed similarities in the following manner: Perform a process of adding a preset value to each of the cosine similarities to obtain incremented cosine similarities; Perform a logarithmic normalization transformation process on the incremented cosine similarities to obtain the transformed similarities.

[0078] In an exemplary embodiment of the present invention, the determination module 630 can implement determining the relevance of the question-and-answer pair to be evaluated based on the target similarity value in the following manner: In the case where the target similarity value is greater than or equal to a similarity threshold, determine that the relevance of the question-and-answer pair to be evaluated meets the relevance requirement; In the case where the target similarity value is less than the similarity threshold, it is determined that the relevance of the question-and-answer pair to be evaluated does not meet the relevance requirement.

[0079] In one exemplary embodiment of the present invention, the determination module 630 may further be configured to: Issue an alarm reminder, where the alarm reminder is used to prompt the user that the relevance of the question-and-answer pair to be evaluated does not meet the relevance requirement.

[0080] In one exemplary embodiment of the present invention, the processing module 620 may implement generating a plurality of variant questions based on the answer to be evaluated in the following manner: Call a pre-configured generative pre-training model; Based on the answer to be evaluated, generate target text prompt information corresponding to the answer to be evaluated, where the target text prompt information is used to guide the generative pre-training model to generate prompt information for variant questions with preset requirements; Input the target text prompt information into the generative pre-training model to obtain a plurality of the variant questions corresponding to the answer to be evaluated output by the generative pre-training model.

[0081] Figure 7 An example of a schematic physical structure diagram of an electronic device is shown as Figure 7 shown. The electronic device may include: a processor 710, a communication interface 720, a memory 730, and a communication bus 740. Among them, the processor 710, the communication interface 720, and the memory 730 complete communication with each other through the communication bus 740. The processor 710 may call logic instructions in the memory 730 to execute a method for evaluating the relevance of a question-and-answer pair. The method includes: obtaining a question-and-answer pair to be evaluated, where the question-and-answer pair to be evaluated includes a question to be evaluated and an answer to be evaluated corresponding to the question to be evaluated, and the answer to be evaluated is generated according to the answer of the question-and-answer system to the question to be evaluated; generating a plurality of variant questions based on the answer to be evaluated, where the variant questions are questions different from the question to be evaluated and can be answered with the answer to be evaluated; and determining the relevance of the question-and-answer pair to be evaluated based on each of the variant questions and the question to be evaluated.

[0082] In addition, when the logical instructions in the above-mentioned memory 730 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0083] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the question-and-answer pair relevance evaluation method provided by the above-mentioned various methods. The method includes: obtaining a question-and-answer pair to be evaluated, where the question-and-answer pair to be evaluated includes a question to be evaluated and a corresponding answer to be evaluated for the question to be evaluated, and the answer to be evaluated is generated according to the answer of the question-and-answer system to the question to be evaluated; generating a plurality of variant questions based on the answer to be evaluated, where the variant questions are questions different from the question to be evaluated and can be answered with the answer to be evaluated; and determining the relevance of the question-and-answer pair to be evaluated based on each of the variant questions and the question to be evaluated.

[0084] On another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is implemented to execute the question-and-answer pair relevance evaluation method provided by the above-mentioned various methods. The method includes: obtaining a question-and-answer pair to be evaluated, where the question-and-answer pair to be evaluated includes a question to be evaluated and a corresponding answer to be evaluated for the question to be evaluated, and the answer to be evaluated is generated according to the answer of the question-and-answer system to the question to be evaluated; generating a plurality of variant questions based on the answer to be evaluated, where the variant questions are questions different from the question to be evaluated and can be answered with the answer to be evaluated; and determining the relevance of the question-and-answer pair to be evaluated based on each of the variant questions and the question to be evaluated.

[0085] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0086] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0087] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for evaluating the relevance of question-answer pairs, characterized in that, The method includes: Obtaining a question-and-answer pair to be evaluated, where the question-and-answer pair to be evaluated includes a question to be evaluated and a corresponding answer to be evaluated for the question to be evaluated, and the answer to be evaluated is generated according to the answer of the question-and-answer system to the question to be evaluated; Generating a plurality of variant questions based on the answer to be evaluated, where the variant questions are questions different from the question to be evaluated and can be answered with the answer to be evaluated; Determining the relevance of the question-and-answer pair to be evaluated based on each of the variant questions and the question to be evaluated.

2. The Q&A pair relevance evaluation method according to claim 1, wherein The determining the relevance of the question-and-answer pair to be evaluated based on each of the variant questions and the question to be evaluated specifically includes: Respectively obtaining various similarities between each of the variant questions and the question to be evaluated based on each of the variant questions and the question to be evaluated; Determining the relevance of the question-and-answer pair to be evaluated based on the various similarities.

3. The Q&A pair relevance evaluation method according to claim 2, wherein The similarity includes cosine similarity; The determining the relevance of the question-and-answer pair to be evaluated based on the various similarities specifically includes: Performing a normalization transformation process on each of the cosine similarities to obtain a transformed similarity, where the similarity value of the transformed similarity is within a preset range; Performing an averaging process on the similarity values of each of the transformed similarities to obtain a target similarity value; Determining the relevance of the question-and-answer pair to be evaluated based on the target similarity value.

4. The Q&A pair relevance evaluation method according to claim 3, characterized in that, The performing a normalization transformation process on each of the cosine similarities to obtain a transformed similarity specifically includes: Performing a process of adding a preset value to each of the cosine similarities to obtain an increased cosine similarity; Performing a logarithmic normalization transformation process on the increased cosine similarity to obtain the transformed similarity.

5. The Q&A pair relevance evaluation method according to claim 3 or 4, characterized in that The determining the relevance of the question-and-answer pair to be evaluated based on the target similarity value specifically includes: When the target similarity value is greater than or equal to a similarity threshold, determining that the relevance of the question-and-answer pair to be evaluated meets the relevance requirement; When the target similarity value is less than the similarity threshold, determining that the relevance of the question-and-answer pair to be evaluated does not meet the relevance requirement.

6. The Q&A pair relevance evaluation method according to claim 5, characterized in that After determining that the relevance of the question-and-answer pair to be evaluated does not meet the relevance requirement, the method further includes: Sending an alarm reminder, where the alarm reminder is used to prompt the user that the relevance of the question-and-answer pair to be evaluated does not meet the relevance requirement.

7. The Q&A pair relevance evaluation method according to claim 1, characterized in that The generating a plurality of variant questions based on the answer to be evaluated specifically includes: Invoking a pre-configured generative pre-training model; Generating target text prompt information corresponding to the answer to be evaluated based on the answer to be evaluated, where the target text prompt information is used to guide the generative pre-training model to generate prompt information for variant questions with preset requirements; Inputting the target text prompt information into the generative pre-training model to obtain a plurality of the variant questions corresponding to the answer to be evaluated output by the generative pre-training model.

8. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method for evaluating the relevance of the question-and-answer pair according to any one of claims 1 to 7.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the Q&A pair relevance evaluation method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the Q&A pair relevance evaluation method according to any one of claims 1 to 7.

Images