Question and answer pair automatic generation method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of natural language processing and information retrieval, and discloses a question and answer pair automatic generation method and device, computer equipment and a storage medium. The method comprises the following steps: firstly, extracting an initial question and answer pair from an input text through a large language model (LLM); slicing and vectorizing the original text, and constructing an original text vector database; vectorizing answers in the initial question and answer pair, matching the vectorized answers with an original text vector database, and positioning one or more original text fragments for each answer to form a traceable triple; performing quality verification on the triple by adopting a comprehensive verification mechanism based on integrity and accuracy evaluation and key information coverage calculation; and finally, de-duplication is carried out in combination with editing distance calculation and semantic similarity calculation, and high-quality question and answer pairs are output. According to the method, the problems of unstable generation quality, lack of answer basis and the like are solved, and automatic construction of the high-credibility knowledge base is realized.

Description

Technical Field

[0001] This invention relates to the field of natural language processing and information retrieval technology, specifically to a method, apparatus, computer device, and storage medium for automatically generating question-answer pairs. Background Technology

[0002] While automated question-answer pair generation techniques based on Large Language Models (LLMs) are widely used in knowledge base construction, they still have significant drawbacks. Firstly, existing solutions either rely solely on LLM self-evaluation or employ single similarity matching, making it difficult to systematically and comprehensively assess question-answer quality. This results in unstable generation quality, a lack of effective verification mechanisms, and the inclusion of factual errors or irrelevant answers. Secondly, deduplication relies on surface text similarity, making it difficult to identify semantically identical but differently worded questions, leading to data redundancy. Most critically, the generated answers fail to accurately anchor to the original text, lacking credibility and exhibiting a limited format. Therefore, how to systematically and automatically generate question-answer pairs with high credibility and traceability is an urgent problem to solve. Thus, an integrated solution is needed that combines automated quality verification, semantic-level deduplication, and accurate original text matching to reliably generate high-quality question-answer datasets. Summary of the Invention

[0003] In view of this, the present invention provides a method, system and storage medium for automatically generating question-answer pairs, in order to solve the problems of uncontrollable quality and lack of traceability of answers when automatically generating question-answer pairs using large language models.

[0004] On one hand, the present invention provides a method for automatically generating question-answer pairs, the steps of which include: Question-answer pair extraction steps: Use a large language model to extract an initial set of question-answer pairs from the input text.

[0005] Original text processing steps: The input text is sliced ​​to obtain a set of original text slices. Regardless of whether the input text is a structured document, table, or unstructured plain text, it is transformed into a unified slice representation through this step. The original text slice set is then vectorized using a semantic coding model to construct an original text vector database.

[0006] Original text matching steps: Vectorize the answers in the initial question-answer pair set and calculate the similarity with the original text vector database. For each answer, filter out the original text segments with similarity higher than the preset similarity threshold, and select the top K segments with the highest similarity scores that are higher than the similarity threshold as the matching results, forming a set of triples consisting of questions, answers and original text segments, where K is a preset integer greater than or equal to 1, and the triple is the smallest data unit used to associate questions, answers and their original text source basis.

[0007] Quality verification steps: Perform quality verification on the set of triples, including: The set of triples is evaluated based on completeness and accuracy to obtain a first evaluation score; The triplet set is evaluated based on the coverage of key information to obtain a second evaluation score; Based on the first and second evaluation scores, a set of triplets that meet the criteria is selected.

[0008] Deduplication steps: For the questions in the triplet set, calculate the edit distance and semantic similarity between each pair of questions, and perform deduplication based on the calculation results of the edit distance and semantic similarity. If the edit distance between two questions is lower than the first threshold, or the semantic similarity is higher than the second threshold, they are determined to be duplicate questions and are removed, retaining only the one with the higher comprehensive score.

[0009] Data storage steps: Store the set of triples that have passed quality verification and have been deduplicated into the database.

[0010] As a preferred embodiment of the present invention, the method further includes: in the quality verification step, based on the factual nature of the question, comparing the entity information in the answer of the same triple with the entity information in the original text fragment; if a factual error is found, the triple is discarded, otherwise it is retained.

[0011] As a preferred embodiment of the present invention, the step of evaluating the set of triples based on completeness and accuracy to obtain a first evaluation score includes: using a large language model to perform a quality assessment of the completeness and accuracy of the triples to obtain a first evaluation score.

[0012] As a preferred embodiment of the present invention, the step of evaluating the triple set based on the coverage of key information to obtain a second evaluation score includes: extracting key information blocks from the matching original text fragments based on the questions in the triples to calculate the coverage of the key information in the answers of the triples, thereby obtaining a second evaluation score.

[0013] As a preferred embodiment of the present invention, the step of selecting a set of triplets that meet the conditions based on the first evaluation score and the second evaluation score includes: making a comprehensive judgment based on the first evaluation score and the second evaluation score, wherein the comprehensive judgment includes: weighting and summing the first evaluation score and the coverage rate according to a predefined weight coefficient to obtain a comprehensive score, and comparing the comprehensive score with a preset quality threshold.

[0014] Secondly, the present invention provides an automatic question-and-answer pair generation device, comprising: The question-answer pair extraction module is configured to generate initial question-answer pairs from the input text using a large language model; The text processing module is configured to slice and vectorize the original text, and build a vector database of the original text. The original text matching module is configured to calculate the similarity between the answer vector and the original text vector, providing the original text basis for accurate answer location; The quality verification module is configured to perform question classification, large language model evaluation, key information coverage calculation, and entity consistency verification to filter low-quality question-answer pairs. The question deduplication module is configured to combine surface text and semantic similarity to deduplicate questions; The data storage module is configured to store and return the final structured, high-quality question-and-answer data.

[0015] Thirdly, the present invention provides a computer device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the question-answer pair automatic generation method described in the first aspect.

[0016] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the question-and-answer pair automatic generation method described in the first aspect above.

[0017] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to perform the method described in the first aspect or any corresponding embodiment thereof.

[0018] By introducing a question classification and differentiated verification process, entity consistency verification is prioritized for factual questions, fundamentally eliminating factual errors. For non-factual questions, a weighted comprehensive score based on LLM evaluation and key information coverage effectively avoids issues such as irrelevance between answers and questions, and answers deviating from the question's main point, thus systematically ensuring the high quality and accuracy of generated question-answer pairs. A two-level deduplication strategy combining surface text and deep semantics accurately identifies and eliminates questions with the same semantics but different expressions, fundamentally solving the semantic redundancy problem that traditional methods struggle to handle. Precise matching of answer vectors with the original text vector database anchors each generated answer to the most relevant original text fragment as its basis, greatly improving the credibility and interpretability of the answers and laying the foundation for building a highly credible knowledge base. Attached Figure Description

[0019] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the modules of an automatic question-and-answer pair generation device according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating an automatic question-and-answer pair generation method according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the deduplication step in a question-and-answer pair generation method according to an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] According to an embodiment of the present invention, an embodiment of a question-and-answer pair automatic generation method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0023] Example 1:

[0024] This embodiment provides a method for automatically generating question-answer pairs, which can be used in the aforementioned computer equipment. Figure 2 This is a flowchart of a question-and-answer pair automatic generation method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps: Step S101, Question-Answer Pair Extraction Step, uses a large language model optimized by prompt word engineering to extract an initial question-answer pair set from the input text, including: constructing prompt words containing roles, tasks, and requirements, and inputting them together with the input text into the large language model to generate the initial question-answer pair set.

[0025] Step S102, original text processing step. The input text is sliced ​​to obtain a set of original text slices. The slicing strategy of the input text is based on paragraphs and punctuation marks as boundaries. In other optional embodiments of the present invention, fixed-length overlapping slicing, adaptive slicing based on semantic coherence (e.g., using the TextTiling algorithm), or structured slicing based on chapter titles can be used, depending on the text characteristics and application requirements. The slicing unit can be a sentence, a group of sentences, or a paragraph. The set of original text slices is vectorized using a semantic coding model to construct an original text vector database. Specifically, slicing is performed based on paragraphs and punctuation marks, and a semantic coding model such as BGE is used for vectorization.

[0026] Step S103, the original text matching step, involves vectorizing the answers in the initial question-answer pair set and calculating their similarity with the original text vector database. For each answer, one or more most relevant original text segments are matched. Specifically, the cosine similarity between the answer vector and the original text segment vector is calculated. For each answer, original text segments with similarity scores higher than a preset similarity threshold are selected, and the top K segments with the highest similarity scores are chosen as the matching results, thus forming a set of triplets consisting of the question, answer, and matched original text.

[0027] Step S104, quality verification step, involves verifying the quality of the triplet set. It should be noted that the core of this step is performing the evaluation and screening as described in claim 1. The following sub-steps S1041 to S1046 illustrate a comprehensive verification process integrating the preferred solutions described in dependent claims 2-5, aiming to more comprehensively ensure quality. This step includes: Step S1041, Problem Classification Sub-step: Use a problem classification model to classify the problems in the triplet set into factual problems and non-factual problems. Preferably, a pre-trained language model can be trained and fine-tuned using a problem data set that is manually labeled as factual problems and non-factual problems to obtain the problem classification model. The criterion for judging factual problems is whether the answer completely depends on the objective facts (such as time, place, people, data, specific steps) explicitly stated in the original text, while non-factual problems involve summarizing, reasoning, inductive reasoning, or intention judgment of the original text.

[0028] Step S1042: For triples classified as factual questions, perform the factual question verification sub-step, first using named entity recognition technology to verify the consistency between the answer and the entities in the original text.

[0029] Step S1043: After passing the entity consistency check, a non-factual question verification sub-step is executed. This sub-step is applicable to both factual and non-factual question triples and includes: using a large language model to perform quality assessment on the question-answer pair to obtain a first assessment score; extracting key information blocks from the matching original text based on the questions in the triple; the extraction of key information blocks is achieved through a large language model (such as Qwen3), which can dynamically extract based on semantic understanding to achieve the best implementation effect; subsequently, calculating the coverage rate of the extracted key information blocks in the answer, and making a comprehensive judgment based on the first assessment score and the coverage rate.

[0030] In step S1044, factual issues that fail the entity consistency check will be filtered out.

[0031] Step S1045: The comprehensive determination is a logical judgment or calculation score based on the first evaluation score and the key information coverage rate. A basic implementation could be that when the first evaluation score is higher than a certain scoring threshold and the key information coverage rate is higher than a certain coverage rate threshold, the verification is deemed successful. In some preferred embodiments, the comprehensive score is calculated using the formula: Comprehensive Score = α * First Evaluation Score + β * Coverage Rate, where α and β are preset weighting coefficients. The weighting coefficient α ranges from 0.5 to 0.7, and β ranges from 0.3 to 0.5, satisfying α + β = 1. The preset quality threshold ranges from 0.6 to 0.9. The parameter K in the original text matching step is usually taken as 1 to 5. Experiments have shown that within this parameter range, the system can achieve a good balance between answer accuracy and information completeness. Those skilled in the art can make adaptive adjustments to the above parameters according to different application scenarios. Such adaptive adjustments based on the core concept of this invention should fall within the protection scope of this invention. When the comprehensive score is greater than or equal to the preset quality threshold, the triplet is determined to have passed the verification. Those skilled in the art should understand that the above weighted summation method is an efficient preferred solution, but the specific algorithm for comprehensive judgment is not limited to this. Other mathematical models or decision rules that can integrate the above two indicators can also be used.

[0032] In step S1046, triplets with a comprehensive score lower than a preset quality threshold will be filtered out.

[0033] Step S105, the question deduplication step, involves deduplicating questions in the verified triplet set by combining edit distance and semantic similarity. If the edit distance is lower than a preset first threshold, or the semantic similarity is higher than a second threshold, the question is considered duplicated and removed, retaining only one of them to obtain the final question-answer pair set. The first threshold can be preset based on experience; for example, when the edit distance uses a normalized edit distance score, it can be set to 0.4. The preset value of the second threshold can also be adjusted according to the model and scenario; for example, when using cosine similarity, it can be set to 0.9.

[0034] Step S106, Data storage step. The final question-answer pair set and its matching original text fragments are stored in the database in a structured form.

[0035] The question-and-answer pair automatic generation method provided in this embodiment achieves the automatic generation of high-quality question-and-answer pairs through a systematic process design.

[0036] Example 2: This embodiment uses a quantum computing-related scientific article of about 5,000 words as unstructured input text to demonstrate the application value of this invention in building a high-precision, traceable domain knowledge base.

[0037] Step S101, Question-Answer Pair Extraction Step. Using a large language model, an initial set of question-answer pairs is extracted from the input text. This step generates a total of 65 initial question-answer pairs.

[0038] Step S102, Original Text Processing Step. The input text is sliced, with slice boundaries determined by punctuation marks such as periods, question marks, and exclamation marks, as well as paragraph structure, resulting in 320 original text slices. Subsequently, the bge-large-zh-v1.5 semantic coding model is used to vectorize each slice in the original text slice set, and a Chroma vector database is constructed based on this vector, forming the original text vector database of this embodiment.

[0039] Step S103, Original Text Matching Step. The 65 answer texts in the initial question-answer pair set are vectorized using the same bge-large-zh-v1.5 model. Then, the cosine similarity between each answer vector and all slice vectors in the original text vector database is calculated. A similarity threshold Wsim = 0.7 is set. This threshold is determined by performing parameter scanning on a reserved validation set, ultimately selecting 0.7 as a balance point that ensures matching accuracy while maintaining a high recall rate. First, all original text slices with similarity below this threshold are filtered out. Then, the top K original text segments with the highest similarity scores (in this embodiment, the preset parameter K = 1) are selected from the remaining slices as their source tracing basis, thus forming a triplet set consisting of the question, answer, and matched original text segment.

[0040] Step S104, Quality Verification Step. This step performs multi-level quality verification on the set of triples.

[0041] Step S1041, Question Classification Sub-step. A BERT pre-trained model, fine-tuned on a manually labeled question type dataset, is used as the question classifier to categorize the questions in the triples into factual and non-factual questions. For example, the question "What are the physical implementation methods of qubits?" is classified as a factual question because its answer must directly correspond to objective facts enumerated in the text (such as superconducting circuits, ion traps, etc.). The question "What are the advantages of quantum entanglement for quantum computing?" is classified as a non-factual question because its answer involves synthesizing and inferring information from multiple parts of the original text.

[0042] Steps S1042 and S1043 are differential verification sub-steps.

[0043] For triples classified as factual questions, such as "What is a quantum bit?", the HanLP toolkit is first used for named entity recognition to verify whether the key entities in the answer are consistent with the entities in the matching original text. After passing the entity consistency check, the subsequent general evaluation process is then executed.

[0044] For all triples that pass the pre-validation (for factual questions) or proceed directly to this step for non-factual questions, perform a uniform deep evaluation: LLM Quality Assessment (First Assessment Score): The question-and-answer pair and its matching original text fragment are input into the Qwen3-7B model. The model is then evaluated using the prompt, "Please rate the question-and-answer pair comprehensively from three dimensions: accuracy of the answer relative to the question, completeness of information, and fluency of language, with a maximum score of 10." For example, a high-quality question-and-answer pair receives 9 points.

[0045] Dynamic Key Information Block Extraction and Coverage Calculation Based on Large Language Model (Second Evaluation Score): One of the core innovations of this invention lies in its use of a large language model to achieve a dynamic, deep semantic understanding-based, and precisely defined key information block extraction method. It is important to note that the "key information blocks" here are not extracted using traditional methods based on word frequency statistics (such as TF-IDF), graph algorithms (such as TextRank), or fixed rules. These traditional methods extract static "keywords" independent of the question, failing to understand the question context and thus unable to determine which information is "key" to answering a specific question. The purpose of this invention is to overcome this inherent deficiency by using LLM to achieve a dynamic, question-relevant key information semantic extraction method, with its computational basis strictly limited to the same triplet. The specific implementation process is as follows: Constructing dynamic extraction instructions: For a specific question within the current triplet, a dedicated prompt word instruction is constructed and input into the large language model (Qwen3-7B is used in this embodiment). This instruction is carefully designed to guide the model to perform a specific task and explicitly limits its information extraction scope to only the matched original text fragments within the triplet, as shown in the following example:

instruction

[0046] Extraction requirements: (1). List them as short noun phrases or verb-object structures that express the core facts.

[0047] (2). Ensure that the extracted information blocks are necessary to answer the questions and are strictly based on the content of the original text.

[0048] (3) Do not add any explanations, just list the information blocks starting with "-".

[0049]

question

[0050] Innovation is evident in the fact that, in this process, the large language model does not perform simple keyword matching, but rather executes deep semantic analysis and reasoning. Dynamism: The extracted key information blocks are determined entirely by the current "question" and "original text fragment", rather than by a predefined dictionary or fixed rules.

[0051] Semantic understanding: The model can understand abstract concepts such as "effect" and "function" and identify the core semantic units describing the function of quantum entanglement from the original text, rather than superficial word frequency statistics.

[0052] Information condensation: The model can condense potentially scattered, descriptive statements in the original text into concise phrases that express the core facts.

[0053] Calculate key information coverage: The core of this step is that the baseline and the object of the coverage calculation are strictly limited to the same triplet.

[0054] Using key information blocks dynamically extracted from the matched original text fragments of the triple as a benchmark, each block is checked to see if it appears semantically equivalently in the answer of the same triple. Semantic equivalence, as described in this invention, means that when determining whether a key information block is covered by an answer, the text does not need to be completely identical; as long as the answer of the same triple contains text or expressions that express the same or very similar core meaning as the key information block, it is considered a cover. For example: Key information block (from the matching original text): "Achieving quantum parallel computing" If the answer within a triple contains "...parallel computation is achieved through quantum entanglement...", it is considered a cover.

[0055] Key information block (from the matching original text): "Building correlations between qubits" If the answer within a triple contains "...entangled states enable connections between qubits...", it is considered a cover.

[0056] Each key information block extracted from the matched original text is examined sequentially, and the coverage rate is calculated as: number of covered key information blocks / total number of key information blocks. For example, if 5 key information blocks are extracted and 4 are covered in the answer, the coverage rate is 80%.

[0057] Comprehensive Judgment: A predefined weighted summation formula is used for comprehensive scoring. Weighting coefficients α=0.7 and β=0.3 are set. This weighting configuration, verified through experiments, optimally balances the subjective judgment of the large language model assessment with the objective measurement of key information coverage, maximizing the comprehensive score's discriminatory power for question-answer pair quality. The quality threshold is set at 7.5 (equivalent to 75 points on a percentage scale). For a question-answer pair that receives a first assessment score of 9 and a key information coverage rate of 85%, its comprehensive score = 0.7 * 9 + 0.3 * (0.85 * 10) = 6.3 + 2.55 = 8.85. Since 8.85 > 7.5, the triplet is deemed to have passed quality verification.

[0058] Step S105, the question deduplication step, involves performing a two-stage deduplication process on the 52 question-answer pairs that passed the quality verification.

[0059] Edit distance calculation is based on the surface similarity of character sequences between normalized edit distance calculation problems, with a preset first threshold of 0.3. This stage can effectively eliminate text-level repetition issues.

[0060] Semantic similarity calculation: Cosine similarity is calculated based on the semantic vector of the question (generated using the bge-large-zh-v1.5 model), with a preset second threshold of 0.9. This stage can identify and merge questions with the same semantics but different expressions.

[0061] If the edit distance between two questions is less than 0.3, or the semantic similarity is greater than 0.9, they are considered duplicate questions, and the duplicates are removed, keeping only one of them. After this step, the number of question-answer pairs is reduced from 52 to 48, resulting in the final question-answer pair set.

[0062] Step S106, Data Storage Step. The final question-answer pair set (48 pairs) and their respective matching original text fragments (including the original text fragment ID and its position information in the original text) are stored in an Elasticsearch database in structured JSON format. The storage scheme creates a document for each question-answer pair, with document fields including question, answer, source_fragments (an array of original text fragments), verification_score (a comprehensive verification score), etc., to facilitate efficient subsequent retrieval and application.

[0063] The professional knowledge question-and-answer base generated by this embodiment improves the accuracy of answers from 80% (relying on a large language model) to 98%, effectively avoiding conceptual errors and content "illusions." By combining a dual deduplication strategy of surface text and deep semantics, the semantic redundancy of questions is reduced from 15% to below 3%, ensuring the simplicity and efficiency of the knowledge base. Moreover, each answer is anchored to the most relevant original text fragment, achieving 100% traceability, which greatly enhances the credibility and interpretability of the knowledge base.

[0064] Example 3: To demonstrate the superiority of the key information extraction method of this invention, the traditional ROUGE index and the LLM dynamic extraction method of this invention were compared for the same question-answer pair:

[0065] Experimental conclusions: Comparative experiments fully demonstrate that the method of "dynamically extracting key information blocks based on a large language model" adopted in this invention effectively solves the inherent technical defects of traditional evaluation methods in terms of semantic flexibility and evaluation accuracy, and provides a reliable technical guarantee for achieving high-quality automated question-answering pair filtering.

[0066] Example 4: This embodiment provides a comparative experiment between the present invention and the traditional method. The difference between this embodiment and Embodiment 2 is that the text is a structured text.

[0067] Experiment setup: Use the same input text (a structured text).

[0068] Experimental group: The entire process method described in this invention is adopted.

[0069] Control group: Only LLM is used to generate question-answer pairs, and simple edit distance-based deduplication is performed, which does not include the original text matching and source tracing and multi-level quality verification mechanism in this invention.

[0070] Evaluation results:

[0071] Experimental conclusions: Comparative experiments fully demonstrate that this invention, by introducing original text matching, classification quality verification, and a dual deduplication mechanism, systematically solves the deficiencies of existing technical solutions in terms of quality, redundancy, and credibility, achieving a qualitative leap in the generation of high-quality question answers.

[0072] Example 5: This embodiment provides a question-and-answer pair automatic generation device, such as... Figure 1 As shown, it includes: The question-answer pair extraction module is used to extract an initial set of question-answer pairs from the input text using a large language model.

[0073] The original text processing module is used to slice and quantize the input text to construct an original text vector database.

[0074] The original text matching module is used to vectorize the answers in the initial question-answer pair set and perform similarity matching with the original text vector database to form a set of triplets consisting of questions, answers and original text fragments.

[0075] The quality verification module is used to perform quality verification on the triple set according to the classification of factual issues and non-factual issues.

[0076] The question deduplication module is used to deduplicat questions in question-answer pairs that have passed quality verification by combining edit distance calculation and semantic similarity calculation to obtain the final question-answer pair set.

[0077] The data storage module is used to store the final question-answer pair set and its matching original text fragments into the database.

[0078] In some alternative embodiments, The question-answer pair extraction module includes an API interface configured with the DeepSeek-R1 model and a prompt word management unit, which is responsible for receiving text uploaded by users and outputting initial question-answer pairs.

[0079] The text processing module includes a built-in sentence segmenter unit and a bge-large-zh-v1.5 model unit, which are responsible for slicing the original text and quantizing it, and storing the vectors into the integrated Chroma vector database.

[0080] The original text matching module includes: a vector database connection unit, which is used to vectorize the answers in the initial question-answer pair set and perform similarity search, returning the top-K most similar matching original text fragments (K=1 in this device example).

[0081] The multi-level quality verification module includes: A fine-tuned BERT classification submodule unit for question classification.

[0082] A Qwen3 call submodule unit for Large Language Model (LLM) evaluation and key information extraction.

[0083] A HanLP NER processing submodule unit is used for entity consistency verification.

[0084] A calculation unit is used to execute the comprehensive scoring formula (Score_final = 0.7 * Score_llm + 0.3 * Score_coverage) and apply a threshold for filtering.

[0085] The question deduplication module includes an edit distance calculator unit and a semantic similarity calculator unit (based on bge-large-zh-v1.5), which are responsible for two-level deduplication of the question set.

[0086] The data storage module includes an Elasticsearch client unit, which receives the final data processed by the aforementioned modules, indexes and stores it in the specified database, and provides a user query interface.

[0087] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0088] In this embodiment, the question-and-answer pair automatic generation device is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0089] Example 6: This invention also provides a computer device having the above-described features. Figure 2 The above describes an automatic question-and-answer pair generation device.

[0090] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 3 As shown, the computer device includes an input device 30 and an output device 40. The processor 10, memory 20, input device 30, and output device 20 can be connected via a bus or other means. Figure 3Taking a bus connection as an example, input device 30 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the computer device, such as a touch screen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 40 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The aforementioned display devices include, but are not limited to, liquid crystal displays, light-emitting diodes, displays, and plasma displays. In some alternative embodiments, the display device may be a touch screen. The various components communicate with each other using different buses and may be mounted on a common motherboard or otherwise installed as needed. The processor can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of the GUI on external input / output devices (such as display devices coupled to an interface). In some alternative embodiments, multiple processors and / or multiple buses may be used with multiple memories and multiple memory sets, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 3 Take a processor 10 as an example.

[0091] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0092] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0093] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0094] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0095] Example 7: This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0096] Example 8: A portion of this application can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to this application through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0097] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for automatically generating question-answer pairs, characterized in that, include: Using a large language model, an initial set of question-answer pairs is extracted from the input text; The input text is sliced ​​to obtain a set of original text slices; The similarity between the answers in the initial question-and-answer pair set and the original text segments is calculated. For each answer, the original text segments with similarity higher than a preset similarity threshold are selected, and the top K segments with the highest similarity scores and scores higher than the similarity threshold are selected as the matching results, forming a set of triplets consisting of questions, answers and original text segments, where K is a preset integer greater than or equal to 1. The set of triples is evaluated based on completeness and accuracy to obtain a first evaluation score; The triplet set is evaluated based on the coverage of key information to obtain a second evaluation score; Based on the first evaluation score and the second evaluation score, a set of triplets that meet the criteria is selected.

2. The method according to claim 1, characterized in that, Also includes: Since the question is a factual question, the entity information in the answer of the same triple is compared with the entity information in the original text. If a factual error is found, the triple is discarded; otherwise, it is retained.

3. The method according to claim 1, characterized in that, The evaluation of the triple set based on completeness and accuracy to obtain a first evaluation score includes: using a large language model to perform a quality assessment of the completeness and accuracy of the triple set to obtain a first evaluation score.

4. The method according to claim 1, characterized in that, The evaluation of the triple set based on key information coverage to obtain a second evaluation score includes: extracting key information blocks from the matching original text fragments based on the questions in the triple to calculate the coverage of the key information in the answer of the triple, and obtaining a second evaluation score.

5. The method according to claim 1, characterized in that, The step of selecting a set of triplets that meet the criteria based on the first evaluation score and the second evaluation score includes: making a comprehensive judgment based on the first evaluation score and the second evaluation score, wherein the comprehensive judgment includes: weighting and summing the first evaluation score and the second evaluation score according to a predefined weight coefficient to obtain a comprehensive score, and comparing the comprehensive score with a preset quality threshold to select triplets whose comprehensive score is greater than the preset quality threshold.

6. The method according to claim 1, characterized in that, Also includes: For the questions in the triplet set, the edit distance and semantic similarity between each pair of questions are calculated, and deduplication is performed based on the calculation results of the edit distance and semantic similarity. If the edit distance between two questions is lower than the first threshold, or the semantic similarity is higher than the second threshold, they are determined to be duplicate questions and are removed, retaining only the one with the higher comprehensive score.

7. A computer device, characterized in that, include: A memory and a processor are communicatively connected, the memory storing computer instructions, and the processor executing the computer instructions to perform the question-answer pair automatic generation method according to any one of claims 1-6.

8. An automatic question-and-answer pair generation device, characterized in that, include: The question-answer pair extraction module is used to extract an initial set of question-answer pairs from the input text using a large language model. The original text processing module is used to slice and quantize the input text to construct an original text vector database; The original text matching module is used to vectorize the answers in the initial question-answer pair set and perform similarity matching with the original text vector database to form a set of triplets consisting of questions, answers and original text fragments; The quality verification module is used to classify the triple set according to factual questions and non-factual questions for quality verification, and output the triple set consisting of question-answer pairs that pass the quality verification and their corresponding original text fragments. The question deduplication module is used to deduplicatize the questions in the question-answer pairs that have passed the quality verification by combining edit distance calculation and semantic similarity calculation, and outputs a set of triples consisting of the deduplicated question-answer pairs and their corresponding original text fragments. The data storage module is used to store the set of triples that have passed quality verification and have been deduplicated into the database.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the question-answer pair automatic generation method as described in any one of claims 1-6.

10. A computer program product, characterized in that, It includes computer instructions for causing a computer to perform the question-and-answer pair automatic generation method according to any one of claims 1-6.

Images