Knowledge base question and answer pair generation method and system combining large model analysis and text clustering
By combining large-scale model analysis and text clustering methods, high-quality knowledge base question-answer pairs are generated, which solves the problems of data clutter and information conflict in knowledge base construction and improves the accuracy and efficiency of answer generation using RAG technology.
Patent Information
- Application Number
- CN202510675677.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-23
- Publication Date
- 2025-09-16
AI Technical Summary
Existing RAG technology has problems in knowledge base construction, such as messy data sources, information conflicts, inaccurate answer generation, and time-consuming and labor-intensive processes. There is also a lack of effective research on knowledge base data preparation.
Combining large-scale model analysis and text clustering, by preprocessing and sentence-by-sentence segmentation of documents, specific questions are generated and text clustering is performed. Question-answer pairs are reviewed, information conflicts are eliminated, and high-quality knowledge base question-answer pairs are generated.
It improves the quality of knowledge base question-answer pairs and the application effect of RAG technology, ensures the accuracy and consistency of answers, and reduces the workload of manual review.
Smart Images
Figure CN120653729A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of retrieval enhancement generation technology, and in particular to a method and system for generating knowledge base question-answer pairs by combining large model analysis and text clustering. Background Art
[0002] Retrieval-augmented generation (RAG) is a key research direction in the application of large AI models. By combining the generative capabilities of large models with the retrieval functionality of external knowledge bases, it effectively overcomes the domain-specific knowledge limitations of general-purpose large models and improves the accuracy and robustness of generated content. However, the current practical application of RAG technology in the industry is less than satisfactory, primarily due to issues with knowledge base construction. As the operational foundation of RAG technology, the quality of the knowledge base plays a decisive role in its performance. The data sources that constitute the knowledge base primarily come from two formats: documents (in formats such as PDF, doc, and txt) and question-answer pairs (one-question-one-answer pairs, typically in Excel spreadsheet format). While providing document data is simple, due to the large volume of document content and the disparate data sources, content across multiple documents may contain conflicting information. Directly using these documents for RAG processing can yield inaccurate answers, particularly for statistical data that varies over time and location. Knowledge Q&A pairs can be generated using fixed templates, but the format is relatively rigid, and the generated Q&A content does not conform well to natural language expression habits. Manually generated knowledge Q&A pairs are subject to significant subjective influences, lack a wide variety of question formats and quantity, and are time-consuming and labor-intensive. Currently, however, the vast majority of RAG technology research focuses on text vectorization and information retrieval, with little research on the aforementioned knowledge base data preparation issues. Summary of the Invention
[0003] The purpose of the present invention is to propose a knowledge base question-answer pair generation method and system that combines large model analysis and text clustering, which can generate high-quality question-answer pair data based on documents and help improve the application effect of RAG technology.
[0004] To achieve the above object, the technical solution of the present invention is as follows:
[0005] The present invention proposes a method for generating knowledge base question-answer pairs by combining large model analysis and text clustering, which specifically includes the following steps:
[0006] S1. Preprocess each document and segment it sentence by sentence, using the resulting sentence set as the reference answer. Based on the reference answer, generate corresponding specific questions using multiple large model platforms. Perform text clustering on the specific questions generated by the same reference answer, selecting one question from each cluster as the representative question. Review and improve each "representative question-reference answer" pair, and generate candidate question-answer pairs.
[0007] S2. Comprehensively compare the content of all candidate question-answer pairs in the document, cluster question-answer pairs that may have information conflicts, analyze and judge the clustering results, and generate reliable knowledge base question-answer pairs;
[0008] S3. Compare the credible knowledge base question-answer pairs with the existing question-answer pairs in the knowledge base one by one. If there is no information conflict, update them into the database.
[0009] Preferably, the preprocessing and sentence-by-sentence segmentation of each document specifically includes: cleaning the document content, including removing irrelevant characters, special symbols, and HTML tags; and segmenting the document content sentence by sentence based on punctuation marks to obtain a sentence set.
[0010] Preferably, the multiple large model platforms include Wenxin Yiyan, Tongyi Qianwen, DeepSeek, KIMI, Tencent Yuanbao and Doubao.
[0011] Preferably, in step S1, the text is clustered using the nearest neighbor propagation algorithm. After the clustering is completed, the cluster center of each cluster is selected as the representative question.
[0012] Preferably, the method of comprehensively comparing the contents of all candidate question-answer pairs in the documents and clustering question-answer pairs that may have information conflicts specifically includes the following steps:
[0013] S11. Calculate any two candidate question texts QueC ija 、QueC ijb The similarity QSim(QueC ija ,QueC ijb );
[0014] S12. Calculate any two candidate answer texts AnsC ijc 、AnsC ijd The similarity ASim(AnsC ijc ,AnsC ijd );
[0015] S13. Calculate any two candidate question-answer pairs QueC ija -AnsC ijc and QueC ijb -AnsC ijd The similarity Sim(QueC ija -AnsC ijc ,QueC ijb -AnsC ijd );
[0016] S14. Convert the similarity of the candidate question-answer pairs into distance, and use the OPTICS algorithm to perform clustering to generate multiple clusters.
[0017] Preferably, the Cosine similarity is used to calculate the similarity between any two candidate question texts QueC ija 、QueC ijb Similarity:
[0018]
[0019] Among them, A and B are QueC ija and QueC ijb The text vector is generated based on the TF-IDF model. ||A|| and ||B|| represent the modulo lengths of vectors A and B, respectively, and A·B represents the dot product of the two vectors.
[0020] Preferably, the difference between the text content and the quantifier is considered to calculate the answer text AnsC of any two candidate answers. ijc 、AnsC ijd Similarity:
[0021]
[0022]
[0023] Among them, C and D are AnsC ijc and AnsC ijd The text vector generated based on the TF-IDF model, ||C|| and ||D|| represent the modulus of vectors C and D respectively, and C·D represents the dot product of the two vectors; Q c and Q d From AnsC ijc and AnsC ijd A collection of quantifiers extracted from the text.
[0024] Preferably, any two candidate question-answer pairs QueC ija -AnsC ijc and QueC ijb -AnsC ijd The similarity calculation is as follows:
[0025]
[0026] Among them, S th represents the similarity threshold, and w represents the weight coefficient.
[0027] Preferably, the credible knowledge base question-answer pairs are compared with the existing question-answer pairs in the knowledge base one by one, which is expressed as:
[0028]
[0029] Among them, QueD′ ija -AnsD′ ijcRepresents a credible knowledge base question-answer pair, QueE ijb -AnsE ijd Represents the existing question-answer pairs in the knowledge base, Sim(QueD ija -AnsD ijc ,QueE ijb -AnsE ijd ) represents the similarity between the credible knowledge base question-answer pair and the existing question-answer pairs in the knowledge base; D th Indicates the information conflict threshold.
[0030] The present invention also proposes a knowledge base question and answer pair generation system that combines large model analysis and text clustering, which is characterized by including a processor, a memory, and a computer program stored on the memory. When the processor executes the computer program, it specifically executes any step in the above-mentioned knowledge base question and answer pair generation method.
[0031] Compared with the prior art, the present invention has the following beneficial effects:
[0032] The present invention combines large model analysis and text clustering to generate knowledge base question-answer pairs, which can generate high-quality question-answer pair data based on documents, and help improve the application effect of RAG technology. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1 Flow chart of the method of the present invention. DETAILED DESCRIPTION
[0034] The following is combined with Figure 1 , the technical solution of the present invention is described in detail.
[0035] The present invention proposes a method for generating knowledge base question-answer pairs by combining large model analysis and text clustering, which specifically includes the following steps:
[0036] S1. Preprocess each document and segment it sentence by sentence, using the resulting sentence set as the reference answer. Based on the reference answer, generate corresponding specific questions using multiple large model platforms. Perform text clustering on the specific questions generated by the same reference answer, selecting one question from each cluster as a representative question. Review and improve the "each representative question-reference answer" pair, and generate candidate question-answer pairs.
[0037] S2. Comprehensively compare the content of all candidate question-answer pairs in the document, cluster question-answer pairs that may have information conflicts, analyze and judge the clustering results, and generate reliable knowledge base question-answer pairs;
[0038] S3. Compare the credible knowledge base question-answer pairs with the existing question-answer pairs in the knowledge base one by one. If there is no information conflict, update them into the database.
[0039] In this embodiment, the pre-processing and sentence-by-sentence segmentation of each document specifically includes: cleaning the document content, including removing irrelevant characters, special symbols, and HTML tags to ensure the text quality; and segmenting the document content sentence by sentence based on punctuation marks such as periods and exclamation marks, and obtaining a sentence set {AnsT ij ,j=1,2,…} as the reference answer, where AnsT ij Represents the document Text i The j-th sentence obtained after sentence-by-sentence segmentation.
[0040] In this embodiment, the multiple large model platforms include Wenxin Yiyan, Tongyi Qianwen, DeepSeek, KIMI, Tencent Yuanbao and Doubao.
[0041] Based on the reference answer {AnsT ij ,j=1,2,…}, generate specific question sets {QueT ijkt ,j=1,2,…,k=1,2,…,t=1,2,…,m}, where QueT ijkt Represents the large model platform t according to the reference answer AnsT ij The kth specific question sentence generated; the prompt words input into the large model platform are as follows:
[0042] {Please parse each declarative sentence and generate specific questions ending with question words such as "what", "how much", and "where" based on the sentence content and the identified subject. Output result format requirements: Present the results (specific questions / declarative sentences) in a table. Each row in the table corresponds to a specific question. If a declarative sentence generates multiple specific questions, each specific question needs to be split into a separate line. For example: the declarative sentence: "By the end of 2023, the total mileage of expressways open to traffic in the province will reach 6,168 kilometers." The subject is "the total mileage of expressways open to traffic in the province", and the generated specific question is: "By the end of 2023, what is the total mileage of expressways open to traffic in the province?"}
[0043] In this embodiment, according to the reference answer AnsT ij , find out the specific question set C generated by each large model platform ij ={QueT ij11 ,QueT ij21 ,…QueT ijkm}, perform text clustering on it, the purpose is to find questions with similar content for subsequent merging processing (for multiple specific questions with the same reference answers and similar content, only one representative question needs to be retained) to improve the quality of question-answer pairs.
[0044] Among them, commonly used algorithms for measuring text similarity include Cosine similarity and Jaccard coefficient. Commonly used algorithms for text clustering include partition-based clustering algorithms (k-means, k-medoids, etc.), hierarchical clustering algorithms (splitting method, merging method, etc.), density-based clustering algorithms (DBSCAN, OPTICS, etc.), grid-based clustering algorithms (STING, CLIQUE, etc.), and model-based clustering algorithms (GMM, SOM, etc.). These algorithms each have their own advantages and disadvantages.
[0045] The present invention uses the Affinity Propagation (AP) algorithm to cluster texts. After clustering, each cluster selects a question as a representative question (the cluster center of the AP algorithm is the representative question), which is recorded as QueR ijk .
[0046] The present invention uses the nearest neighbor propagation algorithm for text clustering, which has three main advantages: first, the algorithm does not need to determine the number of cluster categories and can adaptively process text data of unknown structure; second, the algorithm automatically determines which data points serve as cluster centers (representative questions) through the message passing process, which can effectively reduce the workload of subsequent manual review; third, for large model platforms that are good at generating specific questions, better clustering effects can be obtained by setting the corresponding bias parameter value (the larger the value, the greater the possibility that the point will become a representative point).
[0047] Each “representative question (QueR ijk )—Reference answer (AnsT ij )” question and answer pairs are processed, with the focus on reviewing the rationality (whether it conforms to daily expression habits) and practicality (whether it has practical significance) of the question content, as well as the completeness of the reference answers, and modifying and improving them, and finally generating candidate question and answer pairs “Candidate Question (QueC ijk )—Candidate answer (AnsC ijk )”.
[0048] In this embodiment, for the candidate question-answer pairs generated after manual review, the candidate question (QueC ijk )—Candidate answer (AnsC ijk )” to find question-answer pairs that may have information conflicts (similar questions but very different answers); specifically, the following steps are involved:
[0049] S11. Calculate any two candidate question texts QueC ija 、QueC ijb The similarity QSim(QueC ija ,QueC ijb );
[0050] S12. Calculate any two candidate answer texts AnsC ijc 、AnsC ijd The similarity ASim(AnsC ijc ,AnsC ijd );
[0051] S13. Calculate any two candidate question-answer pairs QueC ija -AnsC ijc and QueC ijb -AnsC ijd The similarity Sim(QueC ija -AnsC ijc ,QueC ijb -AnsC ijd );
[0052] S14. Convert the similarity of the candidate question-answer pairs into distance and cluster them using the OPTICS algorithm (the minimum number of samples required to form a core point is 2; a noise point with a sample number of 1 indicates that there is no other question-answer pair information conflicting with it and no additional processing is required) to generate multiple clusters Can t ={QueC ija -AnsC ijb ...}, t = 1, 2...;
[0053] Manually classify each clustering result t The analysis and judgment are focused on ensuring the consistency of answers to similar questions in different documents, eliminating contradictions and ambiguities, and generating credible knowledge base question-answer pairs after processing.
[0054] In this embodiment, Cosine similarity is used to calculate the similarity between any two candidate question texts QueC ija 、QueC ijb Similarity:
[0055]
[0056] Among them, A and B are QueC ija and QueC ijb The text vector is generated based on the TF-IDF model. ||A|| and ||B|| represent the modulo lengths of vectors A and B, respectively, and A·B represents the dot product of the two vectors.
[0057] In this embodiment, the difference between the text content and the quantifiers (or the place names and the like can also be considered according to the business scenario) is comprehensively considered to calculate any two candidate answer texts AnsC. ijc 、AnsC ijd Similarity:
[0058]
[0059] Among them, C and D are AnsC ijc and AnsC ijd The text vector generated based on the TF-IDF model, ||C|| and ||D|| represent the modulus of vectors C and D respectively, and C·D represents the dot product of the two vectors; Q c and Q d From AnsC ijc and AnsC ijd A collection of quantifiers extracted from the text.
[0060] In this embodiment, any two candidate question-answer pairs QueC ija -AnsC ijc and QueC ijb -AnsC ijd The similarity calculation is as follows:
[0061]
[0062] Among them, S th It represents the similarity threshold, that is, it specifically refers to the situation where the question and answer pairs that are basically different within the question sentence do not consider the information conflict, and w represents the weight coefficient.
[0063] In this embodiment, the credible knowledge base question-answer pairs are compared with the existing question-answer pairs in the knowledge base one by one to find the question-answer pairs that may have information conflicts, which is expressed as:
[0064]
[0065] Among them, QueD′ ija -AnsD′ ijc Represents a credible knowledge base question-answer pair, QueE ijb -AnsE ijd Represents the existing question-answer pairs in the knowledge base, Sim(QueD ija -AnsD ijc ,QueE ijb -AnsE ijd ) represents the similarity between the credible knowledge base question-answer pair and the existing question-answer pair in the knowledge base, Sim(QueD ija -AnsD ijc ,QueE ijb -AnsE ijd ) is calculated in the same way as the formula in candidate question-answer pair clustering; D th Indicates the information conflict threshold.
[0066] The present invention also proposes a knowledge base question and answer pair generation system that combines large model analysis and text clustering, which is characterized by including a processor, a memory, and a computer program stored on the memory. When the processor executes the computer program, it specifically executes any step in the above-mentioned knowledge base question and answer pair generation method.
[0067] The above are preferred embodiments of the present invention. Any changes made according to the technical solution of the present invention, as long as the resulting functions and effects do not exceed the scope of the technical solution of the present invention, shall fall within the scope of protection of the present invention.
Claims
1. A knowledge base question-answer pair generation method combining large model analysis and text clustering, characterized by: The specific steps include: S1. Preprocess each document and segment it sentence by sentence, and use the obtained sentence set as the reference answer; Based on the reference answers, corresponding specific questions are generated through multiple large model platforms. The specific questions generated by the same reference answer are clustered, and one question is selected as the representative question in each cluster. Review and improve each "representative question-reference answer" and generate candidate question-answer pairs; S2. Comprehensively compare the content of all candidate question-answer pairs in the document, cluster question-answer pairs that may have information conflicts, analyze and judge the clustering results, and generate reliable knowledge base question-answer pairs; S3. Compare the credible knowledge base question-answer pairs with the existing question-answer pairs in the knowledge base one by one. If there is no information conflict, update them into the database.
2. The method for generating knowledge base question-answer pairs by combining large model analysis and text clustering according to claim 1, characterized in that: The pre-processing and sentence-by-sentence segmentation of each document specifically includes: cleaning the document content, including removing irrelevant characters, special symbols, and HTML tags; and segmenting the document content sentence by sentence based on punctuation marks to obtain a sentence set.
3. The method for generating knowledge base question-answer pairs by combining large model analysis and text clustering according to claim 1, characterized in that: The multiple large model platforms include Wenxin Yiyan, Tongyi Qianwen, DeepSeek, KIMI, Tencent Yuanbao and Doubao.
4. The method for generating knowledge base question-answer pairs by combining large model analysis and text clustering according to claim 1, characterized in that: In step S1, the text is clustered using the nearest neighbor propagation algorithm. After the clustering is completed, the cluster center of each cluster is selected as the representative question.
5. The method for generating knowledge base question-answer pairs by combining large model analysis and text clustering according to claim 1, characterized in that: The method comprehensively compares the contents of all candidate question-answer pairs in the documents and clusters the question-answer pairs that may have information conflicts. Specifically, the method includes the following steps: S11. Calculate any two candidate question texts QueC ija 、QueC ijb The similarity QSim(QueC ija ,QueC ijb ); S12. Calculate any two candidate answer texts AnsC ijc 、AnsC ijd The similarity ASim(AnsC ijc ,AnsC ijd ); S13. Calculate any two candidate question-answer pairs QueC ija -AnsC ijc and QueC ijb -AnsC ijd The similarity Sim(QueC ija -AnsC ijc ,QueC ijb -AnsC ijd ); S14. Convert the similarity of the candidate question-answer pairs into distance, and use the OPTICS algorithm to perform clustering to generate multiple clusters.
6. The method for generating knowledge base question-answer pairs by combining large model analysis and text clustering according to claim 5, characterized in that: Use Cosine similarity to calculate the similarity between any two candidate question texts QueC ija 、QueC ijb Similarity: Among them, A and B are QueC ija and QueC ijb The text vector is generated based on the TF-IDF model. ||A|| and ||B|| represent the modulo lengths of vectors A and B, respectively, and A·B represents the dot product of the two vectors.
7. The method for generating knowledge base question-answer pairs by combining large model analysis and text clustering according to claim 5, characterized in that: Calculate the answer AnsC of any two candidate answers by comprehensively considering the differences between the text content and the quantifiers ijc 、AnsC ijd Similarity: Among them, C and D are AnsC ijc and AnsC ijd The text vector generated based on the TF-IDF model, ||C|| and ||D|| represent the modulus of vectors C and D respectively, and C·D represents the dot product of the two vectors; Q c and Q d From AnsC ijc and AnsC ijd A collection of quantifiers extracted from the text.
8. The method for generating knowledge base question-answer pairs by combining large model analysis and text clustering according to claim 5, characterized in that: Any two candidate question-answer pairs QueC ija -AnsC ijc and QueC ijb -AnsC ijd The similarity calculation is as follows: Among them, S th represents the similarity threshold, and w represents the weight coefficient.
9. The method for generating knowledge base question-answer pairs by combining large model analysis and text clustering according to claim 1, characterized in that: The credible knowledge base question-answer pairs are compared with the existing question-answer pairs in the knowledge base one by one, which is expressed as: Among them, QueD′ ija -AnsD′ ijc Represents a credible knowledge base question-answer pair, QueE ijb -AnsE ijd Represents the existing question-answer pairs in the knowledge base, Sim(QueD ija -AnsD ijc ,QueE ijb -AnsE ijd ) represents the similarity between the credible knowledge base question-answer pair and the existing question-answer pairs in the knowledge base; D th Indicates the information conflict threshold.
10. The knowledge base question-answer pair generation system combining large model analysis and text clustering according to claim 1, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory. When the processor executes the computer program, the method specifically performs the steps in the method for generating knowledge base question-answer pairs as described in any one of claims 1 to 9.