Question and answer pair set generation method and device, equipment, storage medium and program

Abstract

This application provides a method, apparatus, device, storage medium, and program for generating question-answer pair sets. The method includes: extracting information from a knowledge source document set in a target domain to generate an initial question-answer pair set; performing a first-round quality assessment on the initial question-answer pair set to select a set of selected question-answer pairs that meet preset assessment dimensions, the selected question-answer pair set including multiple selected question-answer pairs; determining the context information corresponding to each selected question-answer pair in the knowledge source document set; and performing a second-round quality assessment on the multiple selected question-answer pairs and their corresponding context information to generate a target question-answer pair set. The method of this application can improve the reliability of the question-answer pair set.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, device, storage medium, and program for generating question-answer pair sets. Background Technology

[0002] Enterprises or research institutions often need to extract professional knowledge from massive amounts of unstructured text (such as academic papers, technical reports, regulatory documents, product manuals, etc.) and transform it into a set of question-answer pairs that can be used to train large domain models.

[0003] In existing technologies, a general large model without domain fine-tuning can be used to directly generate a set of question-answer pairs based on a knowledge source document set. However, while this method is highly automated, it suffers from the problem of generating content out of thin air, which may result in the generated content not matching the original text, leading to low reliability of the question-answer pair set. Summary of the Invention

[0004] This application provides a method, apparatus, device, storage medium, and program for generating question-answer pair sets, in order to solve the technical problem of low reliability of question-answer pair sets.

[0005] Firstly, this application provides a method for generating a set of question-answer pairs, including:

[0006] Extract information from the knowledge source document set of the target domain to generate an initial question-answer pair set;

[0007] The initial question-and-answer pair set is subjected to a first round of quality assessment to select a set of selected question-and-answer pairs that meet the preset assessment dimensions. The set of selected question-and-answer pairs includes multiple selected question-and-answer pairs.

[0008] In the knowledge source document set, determine the context information corresponding to each selected question-answer pair among the plurality of selected question-answer pairs;

[0009] A second round of quality assessment is performed on the selected question-answer pairs and the context information corresponding to each selected question-answer pair to generate a target question-answer pair set.

[0010] This application ensures high data accuracy, effectively avoids the problem of data fabrication, and improves the reliability of question-answer pair sets.

[0011] Optionally, the second round of quality assessment includes multiple validation quality assessments; a second round of quality assessment is performed on the multiple selected question-answer pairs and the context information corresponding to each selected question-answer pair to generate a target question-answer pair set, including:

[0012] For any selected question-answer pair, multiple verification quality assessments are performed on the selected question-answer pair based on the context information corresponding to the selected question-answer pair, and multiple verification assessment results corresponding to the multiple verification quality assessments are obtained;

[0013] For any selected question-answer pair, the target evaluation result corresponding to the selected question-answer pair is determined based on the multiple verification evaluation results;

[0014] Based on the target evaluation results corresponding to each selected question-answer pair, the target question-answer pair set is generated.

[0015] In this application, a fine-grained evaluation system can be used to achieve quality verification, further filtering out low-quality samples with factual bias, missing information, and non-standard expressions, thereby improving the professionalism, accuracy, and reliability of the target question-and-answer pair set.

[0016] Optionally, for any given verification quality assessment, the verification quality assessment includes multiple sub-quality assessments; based on the context information corresponding to the selected question-answer pair, a verification quality assessment is performed on the selected question-answer pair to obtain the verification assessment result corresponding to the verification quality assessment, including:

[0017] For any sub-quality assessment, the selected question-answer pair is subjected to sub-quality assessment based on the context information corresponding to the selected question-answer pair, and the sub-assessment result corresponding to the sub-quality assessment is obtained;

[0018] Based on the sub-assessment results corresponding to each sub-quality assessment, the verification assessment result corresponding to the verification quality assessment is determined.

[0019] In this application, fine-grained quantitative evaluation can be used to achieve comprehensive, objective, and standardized verification of question-answer pairs and their corresponding contexts, further filtering out low-quality samples with factual biases, missing information, and non-standard expressions, thereby improving the accuracy and reliability of the verification evaluation results.

[0020] Optionally, based on the multiple verification evaluation results, the target evaluation result corresponding to the selected question-answer pair is determined, including:

[0021] Determine the verification and evaluation weights corresponding to each verification and evaluation result in the target domain;

[0022] The target evaluation result is obtained by multiplying each verification evaluation result by its corresponding verification evaluation weight and then summing the results.

[0023] In this application, a multi-dimensional weighted scoring mechanism can be used to achieve a quantitative, objective, and unified assessment of the quality of question-and-answer pairs, fully adapt to the assessment focus of different fields, improve the accuracy and credibility of the target assessment results, and provide a reliable basis for the final selection of high-quality question-and-answer pairs.

[0024] Optionally, based on the target evaluation results corresponding to each selected question-answer pair, the target question-answer pair set is generated, including:

[0025] Based on the target evaluation results corresponding to each selected question-answer pair, at least one target question-answer pair is determined from the plurality of selected question-answer pairs;

[0026] For any given target question-answer pair, generate multiple derived question-answer pairs corresponding to the target question-answer pair;

[0027] The at least one target question-and-answer pair and the multiple derived question-and-answer pairs corresponding to each target question-and-answer pair constitute the target question-and-answer pair set.

[0028] In this application, a multi-angle question-answer pair extension generation mechanism enriches the diversity and coverage of the question-answer pair set while ensuring the factual accuracy and semantic consistency of the question-answer pairs, thereby improving the generalization ability and understanding robustness of subsequent model training.

[0029] Optionally, the first round of quality assessment includes multiple initial quality assessments; the preset assessment dimensions include the initial assessment dimensions corresponding to each initial quality assessment; the first round of quality assessment is performed on the initial question-answer pair set to filter out the selected question-answer pair set that meets the preset assessment dimensions, including:

[0030] For any given initial question-answer pair, perform multiple initial quality assessments on the initial question-answer pair to obtain multiple initial assessment results;

[0031] The initial question-answer pairs that all the initial evaluation results meet their corresponding initial evaluation dimensions are determined as the selected question-answer pairs, so as to obtain multiple selected question-answer pairs.

[0032] In this application, multiple evaluation dimensions can be used to achieve macro-level quality screening of the initial question-and-answer pairs, filtering out samples that are irrelevant to the domain, contain superficial facts, or do not conform to professional standards, thereby improving the domain adaptability and overall reliability of the selected question-and-answer pair set.

[0033] Optionally, for any selected question-and-answer pair; also includes:

[0034] Initial question-and-answer pairs that do not conform to their corresponding initial evaluation dimensions are identified as question-and-answer pairs to be adjusted.

[0035] Based on the initial quality assessment of the question-and-answer pairs that do not meet the requirements, adjustment prompts are generated.

[0036] The question-and-answer pairs to be adjusted are processed according to the adjustment prompts to obtain adjusted question-and-answer pairs, and the adjusted question-and-answer pairs are subjected to a first round of quality evaluation to obtain the selected question-and-answer pairs.

[0037] In this application, by readjusting the defective question-and-answer pairs, logical deviations and professional defects in the initial generation can be effectively corrected, realizing automated iteration and optimization of question-and-answer pair quality, ensuring screening efficiency, and improving the pass rate and quality of the selected question-and-answer pair set.

[0038] Optionally, for any selected question-answer pair, in the knowledge source document set, the context information corresponding to the selected question-answer pair is determined, including:

[0039] In the knowledge source document set, determine the initial associated statement corresponding to the selected question-answer pair;

[0040] Verify the support of the initial associated statement for the selected question-and-answer pair;

[0041] Based on the support level of the initial associated statements, the context information corresponding to the selected question-answer pair is determined.

[0042] In this application, by using context binding and support assessment, we can ensure the logical relevance and factual consistency between the answer and the original text, avoid the problem of generated content deviating from the original text, and improve the reliability of question-answer pairs.

[0043] Optionally, based on the support of the initial associated statement, the context information corresponding to the selected question-answer pair is determined, including:

[0044] Determine whether the support of the initial associated statement is greater than the support threshold;

[0045] If so, the initial associated statement is determined as the context information corresponding to the selected question-answer pair;

[0046] If not, then determine the intermediate association statement corresponding to the selected question-answer pair, and determine the context information corresponding to the selected question-answer pair based on the initial association statement and the intermediate association statement;

[0047] The relevance between the initial associated statement and the selected question-and-answer pair is greater than that between the intermediate associated statement and the selected question-and-answer pair.

[0048] In this application, the selected question-answer pair can be supplemented and its context information can be completed and ensured by using quantitative thresholds to determine and retrieve supplementary context. This avoids the problem of insufficient context support due to the lack of information in a single related statement, improves the matching degree and factual consistency between context information and answer, and thus improves the quality foundation of the target question-answer pair set.

[0049] Secondly, embodiments of this application provide a question-answer pair set generation apparatus, including an extraction module, a first quality assessment module, a determination module, and a second quality assessment module:

[0050] The extraction module is used to extract information from the knowledge source document set of the target domain and generate an initial question-answer pair set.

[0051] The first quality assessment module is used to perform a first round of quality assessment on the initial question-answer pair set, and to select a set of selected question-answer pairs that meet the preset assessment dimensions. The set of selected question-answer pairs includes multiple selected question-answer pairs.

[0052] The determining module is used to determine, within the knowledge source document set, the context information corresponding to each selected question-answer pair among the plurality of selected question-answer pairs;

[0053] The second quality assessment module is used to perform a second round of quality assessment on the multiple selected question-answer pairs and the context information corresponding to each selected question-answer pair, and generate a target question-answer pair set.

[0054] Thirdly, embodiments of this application provide an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0055] The memory stores computer-executed instructions;

[0056] The processor executes computer execution instructions stored in the memory to implement the method as described in any of the first aspects.

[0057] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method described in the first aspect.

[0058] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method described in any of the first aspects.

[0059] The question-answer pair set generation method, apparatus, device, storage medium, and program provided in this application can screen out initial question-answer pairs that meet macro-quality standards and filter out samples that are irrelevant to the domain or contain superficial facts. Then, by binding context information, the logical consistency between the answer and the original text is ensured. Subsequently, a second round of quality assessment is carried out based on the matching degree between the question-answer pair and the context, ensuring high accuracy of data at the source, effectively avoiding the problem of creating data out of thin air, and improving the reliability of the question-answer pair set. Attached Figure Description

[0060] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0061] Figure 1 A schematic diagram illustrating the application scenarios provided in the embodiments of this application;

[0062] Figure 2 A flowchart illustrating a question-and-answer pair set generation method provided in an embodiment of this application;

[0063] Figure 3 This is a schematic diagram of the structure of a question-answer pair set generation method provided in an embodiment of this application;

[0064] Figure 4 A flowchart illustrating another question-and-answer pair set generation method provided in this application embodiment;

[0065] Figure 5 A schematic diagram of the architecture of another question-answer pair set generation method provided in an embodiment of this application;

[0066] Figure 6 A structural example diagram of a question-answer pair set generation device provided in an embodiment of this application;

[0067] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0068] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0069] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0070] Figure 1 This is a schematic diagram illustrating an application scenario provided in an embodiment of this application. Please refer to [link / reference]. Figure 1 This technology is suitable for industry scenarios that require the rapid construction of vertical domain knowledge bases, such as healthcare, finance and securities, legal consulting, scientific research (such as chemistry, biology, and materials science), and industrial manufacturing.

[0071] This scenario may include a database 101 and a processing device 102. Database 101 can be used to store a set of knowledge source documents in the target domain. Processing device 102 extracts information from the set of knowledge source documents in the target domain to generate an initial question-and-answer pair set. A first-round quality assessment is performed on the initial question-and-answer pair set to select a set of selected question-and-answer pairs that meet preset assessment dimensions. The selected question-and-answer pair set includes multiple selected question-and-answer pairs. Processing device 102 can determine the context information corresponding to each selected question-and-answer pair in the knowledge source document set, and perform a second-round quality assessment on the multiple selected question-and-answer pairs and their corresponding context information to generate a target question-and-answer pair set.

[0072] In related technologies, a general large model without domain fine-tuning can be used to directly generate question-answer pair sets based on a knowledge source document set. However, while this method is highly automated, it suffers from the problem of generating content out of thin air, which may result in the generated content not matching the original text, leading to low reliability of the question-answer pair set.

[0073] The method for generating question-answer pair sets provided in this application can extract information from a knowledge source document set in a target domain to generate an initial question-answer pair set; perform a first-round quality assessment on the initial question-answer pair set to select a set of selected question-answer pairs that meet preset assessment dimensions, and the selected question-answer pair set includes multiple selected question-answer pairs. Within the knowledge source document set, determine the context information corresponding to each selected question-answer pair, and perform a second-round quality assessment on the multiple selected question-answer pairs and their corresponding context information to generate a target question-answer pair set.

[0074] In the above process, initial question-answer pairs that meet the macro quality standards are selected, and samples that are irrelevant to the domain or contain superficial facts are filtered out. Then, the logical consistency between the answer and the original text is ensured by binding context information. A second round of quality assessment is then carried out based on the matching degree between the question-answer pairs and the context, which ensures the high accuracy of the data at the source, effectively avoids the problem of creating data out of thin air, and improves the reliability of the question-answer pair set.

[0075] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0076] Figure 2 This is a flowchart illustrating a question-and-answer pair set generation method provided in an embodiment of this application. Please refer to... Figure 2 The method may include:

[0077] S201. Extract information from the knowledge source document set of the target domain to generate an initial question-answer pair set.

[0078] The execution subject of this application embodiment can be a processing device or a question-and-answer pair set generation device installed on the processing device. The question-and-answer pair set generation device can be implemented by software or by a combination of software and hardware.

[0079] Target areas can include healthcare, finance and securities, legal consulting, scientific research (such as chemistry, biology, and materials science), and industrial manufacturing.

[0080] A knowledge source document set is a collection of authoritative texts in a target field, including but not limited to academic papers, technical reports, and regulatory documents. For example, in the medical field, a knowledge source document set could be literature on disease diagnostic standards in medical journals.

[0081] By extracting models, information can be extracted from the knowledge source document set of the target domain to generate an initial question-answer pair set.

[0082] During the extraction process, optical character recognition (OCR) technology can be used to identify chart information in the text, formula parsers (such as MathPix) can be used to extract mathematical expressions, and code parsers (such as Tree-sitter) can be used to identify the key logic of code segments, ensuring the complete extraction of unstructured information.

[0083] Among them, the extraction model can refer to the model used to initially generate or filter data, and open-source large models (such as Qwen, DeepSeek and other general language models based on deep learning) should be preferred.

[0084] For example, in the field of chemistry, the extraction model can be a large open-source model used to extract key information about the synthetic pathway of compounds from papers, generating initial question-and-answer pairs such as "What are the key reactions involved in the synthetic steps of compound A?"

[0085] S202. Conduct the first round of quality assessment on the initial question-answer pair set and select the question-answer pair set that meets the preset assessment dimensions.

[0086] The first round of quality assessment may include multiple initial quality assessments, and the preset assessment dimensions may include the initial assessment dimensions corresponding to each initial quality assessment.

[0087] Multiple initial quality assessments can be conducted on domain relevance and coverage, content accuracy and professionalism, adaptability to fine-tuning objectives, depth and hierarchy of questions and answers, and structure and format standardization.

[0088] The initial question-and-answer pair set can be evaluated in the first round using the first evaluation model, and selected question-and-answer pair sets that meet the preset evaluation dimensions can be selected.

[0089] The first evaluation model can refer to the model used for quality assessment, including but not limited to closed-source large models (such as business language models like GPT-4) or locally deployed models. For example, in the legal field, the first evaluation model can be a closed-source large model used to assess the accuracy of legal citations, domain relevance, and standardization of professional expressions in question-and-answer pairs. If a question-and-answer pair contains errors in citations or is irrelevant to the legal field, it is marked as low quality and removed.

[0090] The selected question-and-answer pair set can include multiple selected question-and-answer pairs.

[0091] In some embodiments, for any initial question-answer pair, multiple initial quality assessments are performed on the initial question-answer pair to obtain multiple initial assessment results; the initial question-answer pairs whose initial assessment results all meet their corresponding initial assessment dimensions are determined as selected question-answer pairs to obtain multiple selected question-answer pairs.

[0092] In this application, multiple evaluation dimensions can be used to achieve macro-level quality screening of the initial question-and-answer pairs, filtering out samples that are irrelevant to the domain, contain superficial facts, or do not conform to professional standards, thereby improving the domain adaptability and overall reliability of the selected question-and-answer pair set.

[0093] S203. In the knowledge source document set, determine the context information corresponding to each selected question and answer pair among multiple selected question and answer pairs.

[0094] Contextual information refers to the original text fragments extracted from the knowledge source that surround the answer. It is used to verify the matching degree between the answer and the original text and is a component of the QA-Context triple.

[0095] A QA-Context triple is a structured data unit that includes a question, an answer, and context.

[0096] For example, in the financial sector, contextual information could be a specific descriptive paragraph in a financial report about the company's revenue growth (such as "The company's revenue increased by 15% year-on-year in 2024, mainly due to the increase in market share of core products and the expansion of overseas business").

[0097] For any selected question-answer pair, determine the initial association statement corresponding to the selected question-answer pair in the knowledge source document set; verify the support of the initial association statement for the answer of the selected question-answer pair; and determine the context information corresponding to the selected question-answer pair based on the support of the initial association statement.

[0098] The initial related statements are original text fragments extracted from the knowledge source document set that are directly related to the question-and-answer pair, serving as the basic material for contextual information. For example, in the field of chemistry, paragraphs describing the steps of compound synthesis in papers can be extracted (e.g., "The preparation of compound B requires three steps: the first step is a nucleophilic substitution reaction, the second step is an elimination reaction, and the third step is an addition reaction"), for subsequent verification of the consistency between the answer and the original text.

[0099] By using Retrieval-Augmented Generation (RAG), we can achieve precise binding between question-answer pairs and the original context. During the binding process, we need to ensure that the contextual information fully covers the core factual basis of the answer.

[0100] For example, in the biomedical field, for questions and answers related to "the regulatory mechanism of gene A expression", it is necessary to extract complete paragraphs from the original text regarding regulatory factors, signaling pathways, and experimental conclusions to provide sufficient evidence for subsequent evaluation.

[0101] In this application, by using context binding and support assessment, we can ensure the logical relevance and factual consistency between the answer and the original text, avoid the problem of generated content deviating from the original text, and improve the reliability of question-answer pairs.

[0102] S204. Perform a second round of quality assessment on multiple selected question-answer pairs and the context information corresponding to each selected question-answer pair to generate a target question-answer pair set.

[0103] A second evaluation model can be used to conduct a second round of quality evaluation on the selected question-answer pairs by combining contextual information. The second round of quality evaluation is a fine-grained, multi-dimensional evaluation, which can specifically include three categories of verification quality evaluation: question quality evaluation, context quality evaluation, and answer quality evaluation.

[0104] The second evaluation model can be a large open-source model (such as Qwen, DeepSeek, and other general-purpose language models based on deep learning), which can achieve fine-grained evaluation through automated scoring (the score range can be set to 0-10 points) to ensure the consistency and objectivity of the evaluation criteria.

[0105] In some embodiments, for any selected question-answer pair, multiple verification quality assessments are performed on the selected question-answer pair based on the context information corresponding to the selected question-answer pair, resulting in multiple verification assessment results corresponding to the multiple verification quality assessments; for any selected question-answer pair, the target assessment result corresponding to the selected question-answer pair is determined based on the multiple verification assessment results; and a target question-answer pair set is generated based on the target assessment results corresponding to each selected question-answer pair.

[0106] The verification evaluation results can be assigned a specific score to the corresponding verification quality assessment.

[0107] For any given validation quality assessment, the validation quality assessment may include multiple sub-quality assessments.

[0108] Among them, the quality assessment of the problem focuses on the effectiveness of the problem itself, and its corresponding sub-quality assessments can include clarity (whether the expression is clear and unambiguous), professionalism (whether it conforms to the domain terminology standard), and domain relevance (whether it closely follows the core knowledge points of the target domain).

[0109] Context quality assessment can focus on the supporting value of the context. Its multiple sub-quality assessments can include information completeness (whether it fully covers the core basis of the answer), original text matching (whether it is an original fragment in the knowledge source document set), and answer support (whether it can provide direct factual support for the answer).

[0110] Answer quality assessment can focus on the accuracy and standardization of answers. Its corresponding sub-quality assessments can include accuracy (core facts are consistent with the original text without deviation), completeness (no key information is missing), and expression quality (logical clarity and conformity to domain expression habits).

[0111] In this application, a fine-grained evaluation system can be used to verify the quality of the QA-Context triplet, further filtering out low-quality samples with factual bias, missing information, and non-standard expression, thereby improving the professionalism, accuracy, and reliability of the target question-answer pair set.

[0112] The question-answer pair set generation method provided in this application can extract information from a knowledge source document set in a target domain to generate an initial question-answer pair set. A first-round quality assessment is performed on the initial question-answer pair set to select a set of question-answer pairs that meet preset assessment dimensions. Within the knowledge source document set, the context information corresponding to each selected question-answer pair is determined, and a second-round quality assessment is performed on the multiple selected question-answer pairs and their corresponding context information to generate a target question-answer pair set. First, initial question-answer pairs that meet macro-level quality standards are selected, filtering out samples irrelevant to the domain or containing superficial facts. Then, by binding context information, the logical consistency between the answer and the original text is ensured. Finally, a second-round quality assessment is conducted based on the matching degree between the question-answer pair and the context, ensuring high data accuracy from the source, effectively avoiding the problem of creating data out of thin air, and improving the reliability of the question-answer pair set.

[0113] Figure 3 This is a schematic diagram illustrating the structure of a question-answer pair set generation method provided in an embodiment of this application. Please refer to [link / reference]. Figure 3Information is extracted from the knowledge source document set through an open-source large model (i.e., the extraction model) to generate initial question-and-answer pairs (QA pairs). The initial QA pairs are input into a closed-source large model (i.e., the first evaluation model) for the first round of quality evaluation. The preset evaluation dimensions of this round of evaluation include domain relevance and coverage, content accuracy and professionalism, adaptability to fine-tuning goals, depth and hierarchy of questions and answers, and structure and format standardization.

[0114] The selected question-answer pairs from the first round of evaluation are matched and bound to corresponding contextual information in the knowledge source document set using RAG technology to form QA-Context triples. Subsequently, the QA-Context triples are input into the open-source large model (i.e., the second evaluation model) for a second round of quality evaluation. This round of fine-grained verification quality evaluation is divided into three dimensions: question quality evaluation (including clarity, professionalism, and domain relevance), context quality evaluation (including information completeness, original text matching degree, and answer support), and answer quality evaluation (including accuracy, completeness, and expression quality).

[0115] Finally, the high-quality samples that passed the two rounds of quality assessment were integrated to generate a training set (i.e., the target question-answer pair set) for model training.

[0116] Figure 4 This is a flowchart illustrating another question-and-answer pair set generation method provided in an embodiment of this application. Please refer to... Figure 4 The method may include:

[0117] S401. Extract information from the knowledge source document set of the target domain to generate an initial question-answer pair set.

[0118] The execution process of S401 can be found in the execution process of S201, and will not be repeated here.

[0119] S402. For any initial question-answer pair, perform multiple initial quality assessments on the initial question-answer pair to obtain multiple initial assessment results.

[0120] Specifically, the system judges whether the initial question-and-answer pair closely adheres to the core knowledge points of the target domain, whether the core facts of the answer are consistent with the knowledge source, whether the expression conforms to the domain terminology norms, whether it meets the knowledge depth requirements for model fine-tuning, and whether the format conforms to the preset structured standards, and finally outputs the initial evaluation results of each initial quality assessment quantitative score (e.g., 0-10 points).

[0121] S403. Selected question-answer pairs are those whose initial evaluation results all meet their corresponding initial evaluation dimensions.

[0122] In some embodiments, the characteristics of the target domain can be identified through a preset domain feature dictionary, and the weights of each dimension can be dynamically adjusted (e.g., in the chemical domain, the weight of "content accuracy" is enhanced, and in the legal domain, the weight of "professionalism" is increased).

[0123] Specifically, the dimensional weights of each initial quality assessment dimension can be determined based on the characteristics of the target domain. For any initial question-and-answer pair, the initial assessment results of each initial assessment dimension are multiplied by their corresponding dimensional weights and then summed to obtain the comprehensive quality score of the initial question-and-answer pair. Initial question-and-answer pairs with a comprehensive quality score greater than or equal to a preset score threshold are selected as meeting the requirements of all assessment dimensions; question-and-answer pairs that do not meet the threshold are filtered out.

[0124] The sum of the weights for each dimension is 1.

[0125] For example, the initial evaluation score is between 0 and 10, and the preset score threshold can be 8.

[0126] Optionally, to ensure that the core dimensions are absolutely met, a minimum score limit can be set for each dimension (e.g., the score for "content accuracy" must not be lower than 7 points). If any core dimension score fails to meet the standard, the question and answer pair will be removed even if the overall score meets the threshold.

[0127] In this application, dynamic weight allocation and comprehensive score screening can be used to filter the initial question-and-answer pairs, effectively eliminating samples that are irrelevant to the domain, lack superficial facts, or are insufficient in professional knowledge, thereby improving the domain adaptability, professionalism, and overall quality of the selected question-and-answer pair set.

[0128] In some embodiments, initial question-answer pairs that do not conform to their corresponding initial evaluation dimensions can be identified as question-answer pairs to be adjusted; adjustment prompts are generated based on the initial quality evaluations corresponding to the question-answer pairs to be adjusted; the question-answer pairs to be adjusted are adjusted according to the adjustment prompts to obtain adjusted question-answer pairs, and a first round of quality evaluation is performed on the adjusted question-answer pairs to obtain selected question-answer pairs.

[0129] For questions and answers that fail the initial quality assessment (e.g., insufficient domain relevance or lack of professionalism), targeted adjustment prompts are generated by combining the original content of the knowledge source document set.

[0130] The open-source large model can be invoked based on the adjustment prompt words to rewrite or complete the question-answer pair to be adjusted, thus obtaining the adjusted question-answer pair.

[0131] Subsequently, the adjusted question-and-answer pairs will be resubmitted into the closed-source large model for the first round of quality assessment. The assessment and adjustment process will be repeated until the generated question-and-answer pairs meet the preset assessment dimension requirements, and finally the selected question-and-answer pairs will be obtained.

[0132] In this application, by readjusting the defective question-and-answer pairs, logical deviations and professional defects in the initial generation can be effectively corrected, realizing automated iteration and optimization of question-and-answer pair quality, ensuring screening efficiency, and improving the pass rate and quality of the selected question-and-answer pair set.

[0133] S404. For any selected question-answer pair, determine the initial associated statement corresponding to the selected question-answer pair in the knowledge source document set.

[0134] Retrieval-Augmented Generation (RAG) technology can be used to perform segmented retrieval of knowledge source document sets, and to match text fragments that are semantically related to the question-answer pair step by step according to the sentence or paragraph level. The original text fragments that are highly relevant to the question topic, can directly support the core facts of the answer, and have the highest degree of matching with the answer content are identified as the initial associated statements corresponding to the selected question-answer pair.

[0135] For example, in the field of chemistry, for the selected question and answer pair "What are the key reactions involved in the synthesis of compound A?", the original paragraph describing the synthesis process of compound A is extracted from the corresponding paper through RAG search: "The synthesis of compound A is completed through three reactions: the first step is esterification, the second step is hydrogenation, and the third step is oxidation." This paragraph is then identified as the initial associated statement.

[0136] S405. For any selected question-answer pair, verify the support of the initial associated statement for the answer of the selected question-answer pair.

[0137] Verify whether the initial related statements meet the preset support conditions. The preset support conditions can be that the initial related statements can fully support all the core facts of the answer, have no contradiction with the answer content, can fully verify the authenticity and completeness of the answer from the original text level, and have no missing or unsupported key information.

[0138] For example, if the selected answer to the question is "The synthesis of compound A includes three steps: esterification, hydrogenation, and oxidation," but the initial associated statement only records two steps: esterification and hydrogenation, then the initial associated statement is deemed insufficient to support the answer, and the corresponding original text needs to be searched again or supplemented.

[0139] In addition, the initial related statement can be quantitatively scored from multiple dimensions such as coverage of core facts, semantic consistency, strength of support from the original text, and information completeness. The scores of each dimension are weighted and summed to obtain the support of the initial related statement for the answer.

[0140] S406. For any selected question-answer pair, determine the context information corresponding to the selected question-answer pair based on the support of the initial associated statement.

[0141] Specifically, it determines whether the support of the initial related statement is greater than the support threshold; if so, the initial related statement is determined as the context information corresponding to the selected question-answer pair; if not, the intermediate related statement corresponding to the selected question-answer pair is determined, and the context information corresponding to the selected question-answer pair is determined based on the initial related statement and the intermediate related statement.

[0142] Among them, the relevance between the initial associated statement and the selected question-and-answer pair is greater than that between the intermediate associated statements and the selected question-and-answer pair.

[0143] The purpose of intermediate join statements is to supplement key information missing from the initial join statement, rather than to replace the initial join statement.

[0144] RAG technology can be used to re-retrieve text fragments that are semantically related to the selected question-answer pair but have a lower priority than the initial associated statement in the knowledge source document set, and determine them as the intermediate associated statements corresponding to the selected question-answer pair.

[0145] For example, in the field of chemistry, the selected answer to the question-and-answer question is "The synthesis of compound A includes three steps: esterification, hydrogenation, and oxidation." The initial related statement only states, "The first step in the synthesis of compound A is esterification, and the second step is hydrogenation," with a comprehensive support score of 6 (below the threshold of 8). By re-searching using RAG technology, the intermediate related statement is obtained: "After the first two steps, compound A needs to be finally prepared through an oxidation reaction, which requires temperature control at 80-100℃." By merging the initial and intermediate related statements, the contextual information is formed: "The first step in the synthesis of compound A is esterification, the second step is hydrogenation, and after the first two steps, it needs to be finally prepared through an oxidation reaction, which requires temperature control at 80-100℃," ensuring complete support for the core facts of the answer.

[0146] In this application, the selected question-answer pair can be supplemented and its context information can be completed and ensured by using quantitative thresholds to determine and retrieve supplementary context. This avoids the problem of insufficient context support due to the lack of information in a single related statement, improves the matching degree and factual consistency between context information and answer, and thus improves the quality foundation of the target question-answer pair set.

[0147] S407. For any selected question-answer pair, perform multiple verification quality assessments on the selected question-answer pair based on the context information corresponding to the selected question-answer pair, and obtain multiple verification assessment results corresponding to the multiple verification quality assessments.

[0148] For any given validation quality assessment, the validation quality assessment may include multiple sub-quality assessments.

[0149] Specifically, for any sub-quality assessment, the selected question-answer pair can be sub-assessed based on the context information, resulting in a sub-assessment result. Based on the sub-assessment results, the validation assessment result corresponding to the validation quality assessment can be determined.

[0150] The verification assessment result corresponding to the verification quality assessment is determined by weighted summation of the sub-assessment results corresponding to each sub-quality assessment.

[0151] For example, for the question-and-answer question in the field of chemistry, "What are the key reactions involved in the synthesis of compound A?", a fine-grained evaluation was performed based on the corresponding context information: Question clarity: 9.2 points; Question professionalism: 9.0 points; Question domain relevance: 9.4 points; The weighted sum of the above three sub-evaluation results yielded a verification evaluation result of 9.2 points for the question quality assessment.

[0152] In this application, a fine-grained quantitative evaluation using three-dimensional nine indicators can be used to achieve comprehensive, objective, and standardized verification of question-answer pairs and their corresponding contexts. This further filters out low-quality samples with factual biases, missing information, and non-standard expressions, thereby improving the accuracy and reliability of the verification evaluation results.

[0153] S408. For any selected question-answer pair, determine the target evaluation result corresponding to the selected question-answer pair based on multiple verification evaluation results.

[0154] Specifically, determine the verification evaluation weight of each verification evaluation result corresponding to the target domain; multiply each verification evaluation result by its corresponding verification evaluation weight and sum them to obtain the target evaluation result.

[0155] For example, please refer to Table 1. Each verification evaluation dimension is calculated using a weighted percentage system: the score for each verification evaluation dimension = the average of the corresponding sub-evaluation results; the target evaluation result = question quality score × 30% + context quality score × 30% + answer quality score × 40%.

[0156] Table 1

[0157]

[0158] In this application, a multi-dimensional weighted scoring mechanism can be used to achieve a quantitative, objective, and unified assessment of the quality of question-and-answer pairs, fully adapt to the assessment focus of different fields, improve the accuracy and credibility of the target assessment results, and provide a reliable basis for the final selection of high-quality question-and-answer pairs.

[0159] S409. Generate a target question-answer pair set based on the target evaluation results corresponding to each selected question-answer pair.

[0160] Specifically, based on the target evaluation results corresponding to each selected question-answer pair, at least one target question-answer pair can be determined from multiple selected question-answer pairs; for any target question-answer pair, multiple derived question-answer pairs corresponding to the target question-answer pair can be generated; and at least one target question-answer pair and multiple derived question-answer pairs corresponding to each target question-answer pair can be combined to form a target question-answer pair set.

[0161] Selected question-answer pairs whose target evaluation results are greater than or equal to a preset evaluation threshold can be identified as target question-answer pairs, so as to obtain at least one target question-answer pair.

[0162] Derivative question-and-answer pairs are question-and-answer pairs generated from different questioning angles and using different forms of expression, while keeping the core facts of the answer and the basis of the original text unchanged. The core facts of the answer of the same derivative question-and-answer pair are consistent with the answer of the original target question-and-answer pair.

[0163] It can call open-source large models to perform semantic parsing on the questions and answers of the target question-answer pair, extract core entities (such as compound A), key actions (such as synthesis), constraints (such as steps) and other elements, and construct a structured element map;

[0164] Based on a pre-defined domain template library (such as "step sequence", "key reagents", "reaction conditions"), and combined with extracted elements, diverse question templates are generated, such as "In the key actions of the core entity, what is step N?" and "In the key actions of the core entity, which step involves a key substance?".

[0165] The specific information from the elements is filled into the template to generate candidate derivative questions. The large model is then called to verify the semantic rationality of the candidate questions and their consistency with the original text, eliminating questions that are semantically contradictory or deviate from the core facts. Based on the questioning angle of the derivative questions, the corresponding answers are mapped from the original answers to ensure that the core facts of the answers remain unchanged from the basis of the original text.

[0166] For the same answer, multiple questions from different angles can be generated, such as focusing on the order of steps, key reagents, reaction conditions, mechanism of action, or applicable scenarios.

[0167] For example, in response to the answer "The synthesis steps of compound A include esterification, hydrogenation, and oxidation reactions," derivative question-and-answer pairs can be generated, such as "What is the second reaction in the synthesis steps of compound A?", "Which reaction in the synthesis of compound A usually involves hydrogen gas?", and "What are the key reactions involved in the synthesis of compound A?"

[0168] In this application, the multi-angle question-answer pair extension generation mechanism, while ensuring the factual accuracy and semantic consistency of the question-answer pairs, can enrich the diversity and coverage of the question-answer pair set, and improve the generalization ability and understanding robustness of subsequent model training.

[0169] The question-answer pair set generation method provided in this application can extract information from a knowledge source document set in a target domain to generate an initial question-answer pair set. A first-round quality assessment with multi-dimensional dynamic weights is then performed on the initial question-answer pair set to select question-answer pairs that meet preset assessment criteria. Within the knowledge source document set, the context information corresponding to each selected question-answer pair is determined through retrieval and quantitative verification. Then, a second-round quality assessment with fine-grained, multi-dimensional methods is performed on the selected question-answer pairs based on the context information, ultimately generating a target question-answer pair set containing multi-angle derived question-answer pairs. First, a macro-level quality screening of the initial question-answer pairs is achieved through dynamic weighted assessment, filtering out samples that are domain-irrelevant, factually superficial, or lack professional depth. Then, context binding and quantitative support verification ensure that the answers are highly consistent with the original text and are based on verifiable evidence. Furthermore, a second round of fine-grained assessment further improves the quality of the question-answer pairs and enriches the question angles and expression diversity through derivative extensions, ensuring the accuracy, standardization, and richness of the data from the source. This effectively avoids problems such as fabrication, missing information, or simplistic expression, improving the reliability, completeness, and generalization ability of the question-answer pair set.

[0170] Figure 5 This is a schematic diagram illustrating the architecture of another question-answer pair set generation method provided in an embodiment of this application. Please refer to... Figure 5 It takes domain knowledge sources and new knowledge sources as inputs, and uses them to construct training sets and test sets respectively, providing original knowledge basis for subsequent processes.

[0171] During the data generation phase, open-source large models can be invoked to extract information from knowledge sources and generate initial question-answer pairs (QA pairs), forming test set QA pairs and training set QA pairs.

[0172] During the quality assessment phase, the initial training set data samples are used as input to perform the first round of quality assessment using a closed-source large model. If the assessment passes, the corresponding paper context is added to the QA pair, forming a QA-Context triplet; if the assessment fails, low-quality samples are discarded. An open-source large model is then used to perform a fine-grained second round of assessment on the triplets, generating a standardized quality report and outputting high-quality QA-Context triplets.

[0173] During the data augmentation phase, the high-quality samples that have been evaluated are expanded by generating 5–10 different question formulations while keeping the answers and context unchanged, thus forming an augmented and diversified dataset.

[0174] In the data assembly and output phase, the best candidate samples are selected from both the test set QA pairs and the training set QA pairs, and then assembled into the final test set and training set. High-quality training set and heterogeneous test set are output in standard JSON format, completing the entire dataset construction process.

[0175] Figure 6 This is a structural example diagram of a question-and-answer pair set generation device provided in an embodiment of this application. Please refer to... Figure 6 The question-and-answer set generation device 600 may include an extraction module 601, a first quality assessment module 602, a determination module 603, and a second quality assessment module 604.

[0176] The extraction module 601 is used to extract information from the knowledge source document set of the target domain and generate an initial question-answer pair set.

[0177] The first quality assessment module 602 is used to perform a first round of quality assessment on the initial question-and-answer pair set, and to select a set of selected question-and-answer pairs that meet the preset assessment dimensions. The set of selected question-and-answer pairs includes multiple selected question-and-answer pairs.

[0178] The determination module 603 is used to determine the context information corresponding to each selected question-answer pair among multiple selected question-answer pairs in the knowledge source document set;

[0179] The second quality assessment module 604 is used to perform a second round of quality assessment on multiple selected question-answer pairs and the context information corresponding to each selected question-answer pair, and generate a target question-answer pair set.

[0180] Optionally, the second round of quality assessment includes multiple validation quality assessments; the second quality assessment module 604 is specifically used for:

[0181] For any selected question-answer pair, multiple validation quality assessments are performed on the selected question-answer pair based on the context information corresponding to the selected question-answer pair, resulting in multiple validation assessment results corresponding to the multiple validation quality assessments;

[0182] For any selected question-answer pair, determine the target evaluation result corresponding to the selected question-answer pair based on multiple validation evaluation results;

[0183] Based on the target evaluation results corresponding to each selected question-answer pair, a target question-answer pair set is generated.

[0184] Optionally, for any given verification quality assessment, the verification quality assessment includes multiple sub-quality assessments; the second quality assessment module 604 is specifically used for:

[0185] For any sub-quality assessment, the sub-quality assessment is performed on the selected question-answer pair based on the context information corresponding to the selected question-answer pair, and the sub-assessment result corresponding to the sub-quality assessment is obtained;

[0186] Based on the sub-assessment results corresponding to each sub-quality assessment, determine the verification assessment results corresponding to the verification quality assessment.

[0187] Optionally, the second quality assessment module 604 is specifically used for:

[0188] Determine the verification and evaluation weights corresponding to each verification and evaluation result in the target domain;

[0189] The target evaluation result is obtained by multiplying each verification evaluation result by its corresponding verification evaluation weight and then summing the results.

[0190] Optionally, the second quality assessment module 604 is specifically used for:

[0191] Based on the target evaluation results corresponding to each selected question-answer pair, at least one target question-answer pair is determined from multiple selected question-answer pairs;

[0192] For any given target question-answer pair, generate multiple derivative question-answer pairs corresponding to the target question-answer pair;

[0193] A target question-answer pair set is formed by combining at least one target question-answer pair with multiple derived question-answer pairs corresponding to each target question-answer pair.

[0194] Optionally, the first round of quality assessment includes multiple initial quality assessments; the preset assessment dimensions include the initial assessment dimensions corresponding to each initial quality assessment; the first quality assessment module 602 is specifically used for:

[0195] For any given initial question-answer pair, perform multiple initial quality assessments on the initial question-answer pair to obtain multiple initial assessment results;

[0196] Initial question-answer pairs whose initial evaluation results all conform to their corresponding initial evaluation dimensions are identified as selected question-answer pairs, thus obtaining multiple selected question-answer pairs.

[0197] Optionally, for any selected question-answer pair; the second quality assessment module 604 is also used for:

[0198] Initial question-and-answer pairs that do not conform to their corresponding initial assessment dimensions are identified as question-and-answer pairs that need to be adjusted.

[0199] Based on the initial quality assessment of the non-compliance of the question-and-answer pairs to be adjusted, adjustment prompts are generated.

[0200] The question-and-answer pairs to be adjusted are processed according to the adjustment prompts to obtain the adjusted question-and-answer pairs. The first round of quality evaluation is then carried out on the adjusted question-and-answer pairs to obtain the selected question-and-answer pairs.

[0201] Optionally, for any selected question-answer pair; module 603 is specifically used for:

[0202] In the knowledge source document set, determine the initial associated statement corresponding to the selected question-answer pair;

[0203] Verify the support of the initial associated statement for the selected question-and-answer pair;

[0204] Based on the support of the initial associated statements, determine the context information corresponding to the selected question-answer pair.

[0205] Optionally, module 603 is specifically used for:

[0206] Determine whether the support of the initial associated statement is greater than the support threshold;

[0207] If so, the initial association statement will be determined as the context information corresponding to the selected question-and-answer pair;

[0208] If not, determine the intermediate linking statement corresponding to the selected question-answer pair, and determine the context information corresponding to the selected question-answer pair based on the initial linking statement and the intermediate linking statement;

[0209] Among them, the relevance between the initial associated statement and the selected question-and-answer pair is greater than that between the intermediate associated statements and the selected question-and-answer pair.

[0210] The question-answer pair set generation device provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be described again here.

[0211] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Please refer to... Figure 7 The electronic device 700 may include a processor 701 and a memory 702 communicatively connected to the processor 701. Exemplarily, the processor 701 and the memory 702 are interconnected via a bus 703.

[0212] Memory 702 stores instructions executed by the computer;

[0213] The processor 701 executes computer execution instructions stored in the memory 702, causing the processor 701 to execute the question-answer pair set generation method as shown in the above method embodiment.

[0214] Accordingly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the question-answer pair set generation method of the above method embodiments.

[0215] Accordingly, embodiments of this application may also provide a computer program product, including a computer program, which, when executed by a processor, can implement the question-answer pair set generation method shown in the above method embodiments.

[0216] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0217] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0218] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0219] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0220] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0221] When integrated units / modules are implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage unit can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc.

[0222] If the integrated unit / module is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0223] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0224] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0225] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for generating a set of question-answer pairs, characterized in that, include: Extract information from the knowledge source document set of the target domain to generate an initial question-answer pair set; The initial question-and-answer pair set is subjected to a first round of quality assessment to select a set of selected question-and-answer pairs that meet the preset assessment dimensions. The set of selected question-and-answer pairs includes multiple selected question-and-answer pairs. In the knowledge source document set, determine the context information corresponding to each selected question-answer pair among the plurality of selected question-answer pairs; A second round of quality assessment is performed on the selected question-answer pairs and the context information corresponding to each selected question-answer pair to generate a target question-answer pair set.

2. The method according to claim 1, characterized in that, The second round of quality assessment includes multiple validation quality assessments; a second round of quality assessment is performed on the multiple selected question-answer pairs and the context information corresponding to each selected question-answer pair to generate a target question-answer pair set, including: For any selected question-answer pair, multiple verification quality assessments are performed on the selected question-answer pair based on the context information corresponding to the selected question-answer pair, and multiple verification assessment results corresponding to the multiple verification quality assessments are obtained; For any selected question-answer pair, the target evaluation result corresponding to the selected question-answer pair is determined based on the multiple verification evaluation results; Based on the target evaluation results corresponding to each selected question-answer pair, the target question-answer pair set is generated.

3. The method according to claim 2, characterized in that, For any given validation quality assessment, the validation quality assessment includes multiple sub-quality assessments; based on the context information corresponding to the selected question-answer pair, a validation quality assessment is performed on the selected question-answer pair to obtain the validation assessment result corresponding to the validation quality assessment, including: For any sub-quality assessment, the selected question-answer pair is subjected to sub-quality assessment based on the context information corresponding to the selected question-answer pair, and the sub-assessment result corresponding to the sub-quality assessment is obtained; Based on the sub-assessment results corresponding to each sub-quality assessment, the verification assessment result corresponding to the verification quality assessment is determined.

4. The method according to claim 2, characterized in that, Based on the multiple verification and evaluation results, the target evaluation result corresponding to the selected question-answer pair is determined, including: Determine the verification and evaluation weights corresponding to each verification and evaluation result in the target domain; The target evaluation result is obtained by multiplying each verification evaluation result by its corresponding verification evaluation weight and then summing the results.

5. The method according to claim 2, characterized in that, Based on the target evaluation results corresponding to each selected question-answer pair, the target question-answer pair set is generated, including: Based on the target evaluation results corresponding to each selected question-answer pair, at least one target question-answer pair is determined from the plurality of selected question-answer pairs; For any given target question-answer pair, generate multiple derived question-answer pairs corresponding to the target question-answer pair; The at least one target question-and-answer pair and the multiple derived question-and-answer pairs corresponding to each target question-and-answer pair constitute the target question-and-answer pair set.

6. The method according to claim 1, characterized in that, The first round of quality assessment includes multiple initial quality assessments; the preset assessment dimensions include the initial assessment dimensions corresponding to each initial quality assessment; the first round of quality assessment is performed on the initial question-answer pair set to filter out the selected question-answer pair set that meets the preset assessment dimensions, including: For any given initial question-answer pair, perform multiple initial quality assessments on the initial question-answer pair to obtain multiple initial assessment results; The initial question-answer pairs that all the initial evaluation results meet their corresponding initial evaluation dimensions are determined as the selected question-answer pairs, so as to obtain multiple selected question-answer pairs.

7. The method according to claim 6, characterized in that, For any selected question-and-answer pair; also includes: Initial question-and-answer pairs that do not conform to their corresponding initial evaluation dimensions are identified as question-and-answer pairs to be adjusted. Based on the initial quality assessment of the question-and-answer pairs that do not meet the requirements, adjustment prompts are generated. The question-and-answer pairs to be adjusted are processed according to the adjustment prompts to obtain adjusted question-and-answer pairs, and the adjusted question-and-answer pairs are subjected to a first round of quality evaluation to obtain the selected question-and-answer pairs.

8. The method according to claim 1, characterized in that, For any selected question-answer pair; in the knowledge source document set, determine the context information corresponding to the selected question-answer pair, including: In the knowledge source document set, determine the initial associated statement corresponding to the selected question-answer pair; Verify the support of the initial associated statement for the selected question-and-answer pair; Based on the support level of the initial associated statements, the context information corresponding to the selected question-answer pair is determined.

9. The method according to claim 8, characterized in that, Based on the support level of the initial associated statements, the context information corresponding to the selected question-answer pair is determined, including: Determine whether the support of the initial associated statement is greater than the support threshold; If so, the initial associated statement is determined as the context information corresponding to the selected question-answer pair; If not, then determine the intermediate association statement corresponding to the selected question-answer pair, and determine the context information corresponding to the selected question-answer pair based on the initial association statement and the intermediate association statement; The relevance between the initial associated statement and the selected question-and-answer pair is greater than that between the intermediate associated statement and the selected question-and-answer pair.

10. A question-answer pair set generation device, characterized in that, It includes an extraction module, a first quality assessment module, a determination module, and a second quality assessment module. The extraction module is used to extract information from the knowledge source document set of the target domain and generate an initial question-answer pair set. The first quality assessment module is used to perform a first round of quality assessment on the initial question-answer pair set, and to select a set of selected question-answer pairs that meet the preset assessment dimensions. The set of selected question-answer pairs includes multiple selected question-answer pairs. The determining module is used to determine, within the knowledge source document set, the context information corresponding to each selected question-answer pair among the plurality of selected question-answer pairs; The second quality assessment module is used to perform a second round of quality assessment on the multiple selected question-answer pairs and the context information corresponding to each selected question-answer pair, and generate a target question-answer pair set.

11. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions that, when executed by a processor, are used to implement the method described in any one of claims 1-9.

13. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method described in any one of claims 1-9.

Images