An intelligent question and answer generation method based on triple search query optimization

Abstract

The application discloses an intelligent question and answer generation method based on triple retrieval query optimization, comprising: constructing a multi-level tree retrieval structure for each document in the enterprise corpus, forming a retrieval result of the multi-level tree retrieval structure of each document according to the cosine similarity; performing keyword retrieval on each document in the enterprise corpus based on the original question, obtaining the keyword retrieval result of each document based on the score of BM25; respectively performing vector embedding on each document in the enterprise corpus and the original question, obtaining the vector retrieval result of each document based on Embedding; calculating the comprehensive score of the retrieval result of the multi-level tree structure of each document, the result of the keyword retrieval and the result of the vector retrieval, rewriting the original question by using a large language model; splicing the rewritten question and the document in the enterprise corpus, inputting into the large language model, and generating an answer. Through multi-level retrieval optimization, the retrieval accuracy and comprehensiveness are improved to generate high-quality answers.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence question-answering technology, specifically to an intelligent question-answering generation method based on triple retrieval query optimization. Background Technology

[0002] Traditional document question answering systems primarily rely on retrieval techniques to convert user questions into vectors and search for matching documents in a document repository. However, they often lack a deep understanding of the document context, leading to insufficient accuracy and reliability of the retrieval results. Although Large Language Models (LLMs) perform well on various tasks, they still suffer from the "illusion" problem of generating incorrect information.

[0003] Retrieval-Augmented Generation (RAG) technology enhances the answering capabilities of LLM by combining information retrieval and content generation. However, embedding-based vector retrieval techniques struggle to meet the accuracy requirements of generated content, while knowledge graph technology, although highly accurate, demands extensive data annotation and processing. Therefore, how to quickly, efficiently, and accurately construct an enterprise-level intelligent question-answering database based on LLM to improve the accuracy of intelligent question answering is a pressing technical problem that needs to be solved. Summary of the Invention

[0004] To address the problems existing in the prior art, this invention provides an intelligent question-answering generation method based on triple retrieval query optimization. Through multi-level corpus management and optimized query processing, it improves the accuracy and comprehensiveness of enterprise corpus retrieval, thereby generating high-quality answers.

[0005] To achieve the above technical objectives, the present invention adopts the following technical solution: an intelligent question-answering generation method based on triple retrieval query optimization, comprising the following steps:

[0006] Vector embedding and clustering are performed on each document in the enterprise corpus to construct a multi-level tree retrieval structure. The multi-level tree retrieval structure is traversed according to the original question. The vectors of the selected nodes are concatenated according to the cosine similarity to form the retrieval results of the multi-level tree retrieval structure for each document.

[0007] The original question was searched for keywords in each document in the enterprise corpus, and the keyword search results for each document were obtained based on the BM25 score.

[0008] Vector embedding is performed on each document and the original question in the enterprise corpus, and vector retrieval results for each document are obtained based on the embedding.

[0009] Calculate the combined score for each document's multi-level tree structure search results, keyword search results, and vector search results;

[0010] The documents in the enterprise corpus were sorted from high to low according to their comprehensive scores, and the original question was rewritten using a large language model (LLM).

[0011] The rewritten question is concatenated with documents from the enterprise corpus and input into a large language model (LLM) to generate an answer.

[0012] Furthermore, the construction process of the multi-level tree retrieval structure is as follows:

[0013] Step S1: Segment each document in the enterprise corpus into text according to a fixed token length, and use a semantic vector model to represent the segmented text using vector embedding;

[0014] Step S2: Randomly select several vectors as the center of the nodes of the Gaussian mixture model;

[0015] Step S3: Calculate the prior probability that each vector belongs to the k-th node, and combine it with the Gaussian distribution likelihood function to determine the current optimal number of nodes;

[0016] Step S4: For each vector, calculate its posterior probability of belonging to the current best node, and control the generation tendency of new nodes based on the posterior probability;

[0017] Step S5: Repeat steps S3-S4 until the nodes no longer change, embed the corresponding vectors into the corresponding nodes to form a multi-level tree retrieval structure for the document.

[0018] Furthermore, the current optimal number of nodes K * The calculation process is as follows:

[0019]

[0020] Where K represents the initial number of nodes, N represents the number of vectors in the document, p represents the prior probability that a vector belongs to the k-th node, and L represents the Gaussian likelihood function.

[0021] Furthermore, the process of forming the retrieval results of the multi-level tree retrieval structure for each document is as follows:

[0022] i. Start searching from the root node of the multi-level tree retrieval structure of each document, calculate the cosine similarity between the vector representation of the original question and the vector of each root node, sort the cosine similarity in descending order, and select the first n root nodes to form the root set;

[0023] ii. Recursively search for child nodes in the root set, calculate the cosine similarity between the vector representation of the original problem and the vector of each child node, sort the cosine similarity in descending order, select the first m child nodes to form a subset, and continue until the leaf node of the multi-level tree retrieval structure is reached.

[0024] iii. Assemble the root set and all subsets according to the relationships between nodes to form the retrieval context of the document, which serves as the retrieval result of the multi-level tree retrieval structure of the document.

[0025] Furthermore, the calculation process for the overall score of each document is as follows:

[0026] H = α·S tree (q,D h )+β·BM25(q,D h )+γ·Embedding(q,D h )

[0027] Where H represents the document's overall score, and D... h S represents the h-th document in the enterprise corpus. tree (q,D h ) represents the original problem q and D h The cosine similarity of search results from a multi-level tree retrieval structure, where α represents S tree (q,D h The weighting coefficients of BM25(q,D); h ) represents the original problem q and D h For keyword retrieval relevance, β represents BM25(q,D) h The weighting coefficients of ); Embedding(q,D h ) represents the original problem q and D h For vector retrieval relevance, γ represents Embedding(q,D) h The weighting coefficients are α+β+γ=1.

[0028] Furthermore, the original problem q and D h Cosine similarity S of search results from a multi-level tree search structure tree (q,D h The calculation process for ) is as follows:

[0029]

[0030] Where S represents D h The number of nodes in S, where s represents the index of S, and w s v represents the weight coefficient of the s-th node. s Let cos(q, v) be the vector representing the s-th node. s) represents the original problem q and v s The cosine similarity.

[0031] Furthermore, the original problem q and D h Relevance BM25(q,D) for keyword retrieval h The calculation process for ) is as follows:

[0032]

[0033] Where D represents the number of documents in the enterprise corpus, I represents the number of terms in the original question, i represents the index of I, and f i D represents the frequency of term i in the h-th document. i avgdl represents the number of documents containing term i, avgdl represents the average length of all documents in the enterprise corpus, k represents an adjustable parameter that controls word frequency, and b represents an adjustable parameter that controls document length.

[0034] Furthermore, the original problem q and D h Relevance embedding (q,D) for vector retrieval h The calculation process for ) is as follows:

[0035]

[0036] Among them, Q q Represents the dense vector in the original problem. D represents h In the dense vector, δ represents the weight coefficient of the dense vector; S q Represents the sparse vectors in the original problem. D represents h In the sparse vector, η represents the weight coefficient of the sparse vector; δ+η=1.

[0037] Furthermore, the prompt for rewriting the original problem using LLM is as follows: the goal is to rewrite the original problem using the top l documents with the highest overall scores, requiring that the rewritten problem be different from the original problem and as short as possible; the input is the original problem and the top l documents with the highest overall scores; the output is the rewritten problem.

[0038] Furthermore, the process of combining the rewritten questions with the documents in the enterprise corpus is as follows: the document with the highest overall score is placed at the beginning, the rewritten questions are placed at the end, and the remaining documents are placed between the beginning and the end in descending order of their overall scores.

[0039] Compared with existing technologies, the present invention has the following beneficial effects: The present invention introduces a multi-level tree retrieval structure based on a triple retrieval query optimization intelligent question answering generation method. By clustering texts in the enterprise corpus, the hierarchical information of the corresponding documents is obtained, and the content of the documents can be obtained from multiple levels. Then, the document relevance is comprehensively scored by combining keyword retrieval results and vector retrieval results, thereby ensuring that the retrieved documents meet the requirements in terms of structure, semantic understanding, and keyword matching, improving the accuracy, comprehensiveness, and flexibility of the retrieval. This allows for the rewriting of questions, improving the accuracy of question rewriting and generating high-quality answers. Attached Figure Description

[0040] Figure 1 This is a flowchart of the intelligent question-answering generation method based on triple retrieval query optimization of the present invention;

[0041] Figure 2 This is a schematic diagram of the intelligent question-answering generation system based on triple retrieval query optimization of the present invention. Detailed Implementation

[0042] The technical solution of the present invention will be further explained and described below with reference to the accompanying drawings.

[0043] like Figure 1 The present invention provides an intelligent question-answering generation method based on triple retrieval query optimization, comprising the following steps:

[0044] (1) Perform vector embedding and clustering on each document in the enterprise corpus, construct a multi-level tree retrieval structure, traverse the multi-level tree retrieval structure according to the original question, and concatenate the vectors of the selected nodes according to the cosine similarity to form the retrieval results of the multi-level tree retrieval structure of each document. By clustering the text in the enterprise corpus, the hierarchical information of the corresponding document can be obtained, and the content of the document can be obtained from multiple levels.

[0045] (2) The original question is searched for keywords in each document in the enterprise corpus. The keyword search results for each document are obtained based on the BM25 score. Based on word frequency statistics, the calculation speed is fast.

[0046] (3) Vector embedding is performed on each document and the original question in the enterprise corpus respectively. Vector retrieval results of each document are obtained based on embedding. Through vector embedding, deep semantics are captured and the understanding ability is stronger.

[0047] (4) Calculate the comprehensive score of the multi-level tree structure retrieval results, keyword retrieval results and vector retrieval results for each document, so as to ensure that the retrieved documents meet the requirements in terms of structure, semantic understanding and keyword matching, and improve the accuracy, comprehensiveness and flexibility of retrieval.

[0048] (5) Sort the documents in the enterprise corpus in order of comprehensive score from high to low, and use large language model LLM to rewrite the original question, transforming the original question into a more accurate expression that is more in line with the user's intent, thereby significantly improving the search effect and user experience.

[0049] (6) The rewritten question and the document in the enterprise corpus are concatenated and input into the large language model LLM to generate a high-quality answer.

[0050] The construction process of the multi-level tree retrieval structure in this invention is as follows:

[0051] Step S1: Segment each document in the enterprise corpus according to a fixed token length, and use the semantic vector model BGE-M3 to represent the segmented text using vector embedding;

[0052] Step S2: Randomly select several vectors as the center of the nodes of the Gaussian mixture model;

[0053] Step S3: Calculate the prior probability that each vector belongs to the k-th node. Combine the Gaussian distribution likelihood function with the Bayesian information criterion to determine the current optimal number of nodes as an initial approximation for variational inference. After completing variational inference, the Gaussian distribution parameters and mixing weights of each cluster are obtained. These parameters can be used to update the nodes of the cluster.

[0054] The current optimal number of nodes K in this invention * The calculation process:

[0055]

[0056] Where K represents the initial number of nodes, N represents the number of vectors in the document, p represents the prior probability that a vector belongs to the k-th node, and L represents the Gaussian likelihood function.

[0057] Step S4: The process Gaussian mixture model allows for dynamic clustering of complex data without pre-determining the number of clusters. For each vector, the posterior probability of it belonging to the current best node is calculated. The tendency to generate new nodes is controlled based on the posterior probability. The lower the posterior probability, the more encouraged to generate new nodes; otherwise, the tendency is to generate the current node.

[0058] Step S5: Repeat steps S3-S4 until the nodes no longer change, embed the corresponding vectors into the corresponding nodes to form a multi-level tree retrieval structure for the document.

[0059] This invention constructs a multi-level tree retrieval structure for documents using a process Gaussian mixture model, which can accurately capture multimodal features in document data, thereby dividing documents into multiple levels and expressing the semantic relationship between nodes through prior probabilities. At the same time, it can dynamically adjust the multi-level tree retrieval structure based on posterior probabilities.

[0060] The process of forming the retrieval results for each document using a multi-level tree retrieval structure is as follows:

[0061] i. Start searching from the root node of the multi-level tree retrieval structure of each document, calculate the cosine similarity between the vector representation of the original question and the vector of each root node, sort the cosine similarity in descending order, and select the first n root nodes to form the root set;

[0062] ii. Recursively search for child nodes in the root set, calculate the cosine similarity between the vector representation of the original problem and the vector of each child node, sort the cosine similarity in descending order, select the first m child nodes to form a subset, and continue until the leaf node of the multi-level tree retrieval structure is reached.

[0063] iii. Assemble the root set and all subsets according to the relationships between nodes to form the retrieval context of the document, which serves as the retrieval result of the document's multi-level tree retrieval structure.

[0064] In this invention, the original problem q and D h Cosine similarity S of search results from a multi-level tree search structure tree (q,D h The calculation process for ) is as follows:

[0065]

[0066] Where S represents D h The number of nodes in S, where s represents the index of S, and w s This represents the weight coefficient of the s-th node. v s Let cos(q, v) be the vector representing the s-th node. s ) represents the original problem q and v s The cosine similarity.

[0067] In this invention, the original problem q and D h Relevance BM25(q,D) for keyword retrieval h The calculation process for ) is as follows:

[0068]

[0069] Where D represents the number of documents in the enterprise corpus, I represents the number of terms in the original question, i represents the index of I, and f iD represents the frequency of term i in the h-th document. i avgdl represents the number of documents containing term i, avgdl represents the average length of all documents in the enterprise corpus, k represents an adjustable parameter that controls word frequency, and b represents an adjustable parameter that controls document length.

[0070] In this invention, the original problem q and D h Relevance embedding (q,D) for vector retrieval h The calculation process for ) is as follows:

[0071]

[0072] Among them, Q q Represents the dense vector in the original problem. D represents h In the dense vector, δ represents the weight coefficient of the dense vector; S q Represents the sparse vectors in the original problem. D represents h In the sparse vector, η represents the weight coefficient of the sparse vector; δ+η=1, and according to the experimental results, δ=0.7 and η=0.3.

[0073] In summary, the calculation process for the overall score of each document in this invention is as follows:

[0074] H = α·S tree (q,D h )+β·BM25(q,D h )+γ·Embedding(q,D h )

[0075] Where H represents the document's overall score, and D... h S represents the h-th document in the enterprise corpus. tree (q,D h ) represents the original problem q and D h The cosine similarity of search results from a multi-level tree retrieval structure, where α represents S tree (q,D h The weighting coefficients of )

[0076] BM25(q,D h ) represents the original problem q and D h For keyword retrieval relevance, β represents BM25(q,D) h The weighting coefficients of Embedding(q,D); h ) represents the original problem q and D h For vector retrieval relevance, γ represents Embedding(q,D) h) weight coefficient, α+β+γ=1, α=0.6, β=0.2, γ=0.2.

[0077] In one technical solution of this invention, the prompt for rewriting the original question using LLM specifically involves: the goal is to rewrite the original question using the top l-ranked documents based on their overall scores, requiring the rewritten question to be different from the original question and as short as possible; the input is the original question and the top l-ranked documents based on their overall scores; the output is the rewritten question. By rewriting the original question, user intent can be better understood, improving the accuracy and relevance of subsequent answers.

[0078] In one technical solution of this invention, the process of concatenating the rewritten question and documents from the enterprise corpus is as follows: Considering that large language models tend to select blocks at the beginning and end of the page, while ignoring blocks in the middle, the document with the highest overall score is placed at the beginning. This ensures that the large language model prioritizes capturing core conclusions, key data, or definitions, avoiding interference from low-quality information. The rewritten question is placed at the end as a semantic anchor, helping the model to always focus on the user's intent when generating answers, preventing it from being misled by irrelevant information. The remaining documents are placed between the beginning and end in descending order of their overall scores, ensuring that the large language model can gradually introduce secondary but complementary information when it needs to expand the answer. Through the above concatenation method, the accuracy of the large language model's answers is greatly improved.

[0079] like Figure 2 The present invention also provides an intelligent question-answering generation system based on triple retrieval query optimization, including: a tree-level retrieval module, a keyword retrieval module, a vector retrieval module and a scoring module set in an enterprise corpus, and a question rewriting module and a question splicing module set in a large language model;

[0080] The tree-level retrieval module is used to construct a multi-level tree retrieval structure for each document in the enterprise corpus. It traverses the multi-level tree retrieval structure according to the original question, and concatenates the vectors of the selected nodes according to the cosine similarity to form the retrieval results of the multi-level tree retrieval structure for each document.

[0081] The keyword retrieval module is used to perform keyword retrieval on each document in the enterprise corpus for the original question, and obtain the keyword retrieval results for each document based on the BM25 score;

[0082] The vector retrieval module is used to perform vector embedding on each document in the enterprise corpus and obtain the vector retrieval results for each document based on the embedding.

[0083] The scoring module is used to calculate a comprehensive score for each document, based on the multi-level tree structure search results, keyword search results, and vector search results.

[0084] The question rewriting module, based on a large language model (LLM), rewrites the original question using retrieved documents.

[0085] The question concatenation module is used to concatenate rewritten questions and documents from the enterprise corpus, input them into a large language model (LLM), and generate answers.

[0086] In one technical solution of the present invention, a computer-readable storage medium is also provided, storing a computer program that enables a computer to execute the intelligent question-answering generation method based on triple retrieval query optimization of the present invention.

[0087] In one technical solution of the present invention, an electronic device is also provided, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the intelligent question-answering generation method based on triple retrieval query optimization of the present invention.

[0088] In the embodiments disclosed in this application, a computer storage medium may be a tangible medium that may contain or store programs for use by or in conjunction with an instruction execution system, apparatus, or device. The computer storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of computer storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0089] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0090] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. An intelligent question-answering generation method based on triple retrieval query optimization, characterized in that, Includes the following steps: Vector embedding and clustering are performed on each document in the enterprise corpus to construct a multi-level tree retrieval structure. The multi-level tree retrieval structure is traversed according to the original question. The vectors of the selected nodes are concatenated according to the cosine similarity to form the retrieval results of the multi-level tree retrieval structure for each document. The original question was searched for keywords in each document in the enterprise corpus, and the keyword search results for each document were obtained based on the BM25 score. Vector embedding is performed on each document and the original question in the enterprise corpus, and vector retrieval results for each document are obtained based on the embedding. Calculate the combined score for each document's multi-level tree structure search results, keyword search results, and vector search results; The documents in the enterprise corpus were sorted from high to low according to their comprehensive scores, and the original question was rewritten using a large language model (LLM). The rewritten question is concatenated with documents from the enterprise corpus and input into a large language model (LLM) to generate an answer. The construction process of the multi-level tree retrieval structure is as follows: Step S1: Segment each document in the enterprise corpus into text according to a fixed token length, and use a semantic vector model to represent the segmented text using vector embedding; Step S2: Randomly select several vectors as the center of the nodes of the Gaussian mixture model; Step S3: Calculate the vector belonging to the first... k The prior probability of each node is used to determine the optimal number of nodes at the current time, combined with the Gaussian distribution likelihood function. The current optimal number of nodes The calculation process is as follows: in, Indicates the initial number of nodes. This indicates the number of vectors in the document. Indicates that the vector belongs to the first... k The prior probability of each node Represents the likelihood function of a Gaussian distribution; Step S4: For each vector, calculate its posterior probability of belonging to the current best node, and control the generation tendency of new nodes based on the posterior probability; Step S5: Repeat steps S3-S4 until the nodes no longer change, and embed the corresponding vectors into the corresponding nodes to form a multi-level tree retrieval structure for the document. The process of combining the rewritten questions with documents from the enterprise corpus is as follows: place the document with the highest overall score at the beginning, place the rewritten questions at the end, and place the remaining documents in descending order of their overall scores between the beginning and the end.

2. The intelligent question-answering generation method based on triple retrieval query optimization according to claim 1, characterized in that, The process of forming the retrieval results of the multi-level tree retrieval structure for each document is as follows: i. Starting from the root node of the multi-level tree retrieval structure for each document, calculate the cosine similarity between the vector representation of the original question and the vector of each root node. Sort the cosine similarities in descending order and select the top... n There are root nodes, forming a root set; ii. Recursively retrieve child nodes in the root set, calculate the cosine similarity between the vector representation of the original problem and the vector of each child node, sort the cosine similarities in descending order, and select the top... m Each child node forms a subset, until the leaf node of the multi-level tree retrieval structure is reached; iii. Assemble the root set and all subsets according to the relationships between nodes to form the retrieval context of the document, which serves as the retrieval result of the multi-level tree retrieval structure of the document.

3. The intelligent question-answering generation method based on triple retrieval query optimization according to claim 1, characterized in that, The calculation process for the overall score of each document is as follows: in, This indicates the document's overall score. Indicates the first in the enterprise corpus h One document, Representing the original problem and The cosine similarity of search results from a multi-level tree retrieval structure. express Weighting coefficients; Representing the original problem and To assess the relevance of keyword searches, express Weighting coefficients; Representing the original problem and Relevance of vector retrieval express The weighting coefficients, .

4. The intelligent question-answering generation method based on triple retrieval query optimization according to claim 3, characterized in that, Original question and Cosine similarity of search results from a multi-level tree search structure The calculation process is as follows: in, S express The number of nodes in s express S index, Indicates the first s The weight coefficients of each node, Indicates the first s A vector of nodes, Representing the original problem and The cosine similarity.

5. The intelligent question-answering generation method based on triple retrieval query optimization according to claim 3, characterized in that, Original question and Relevance of keyword search The calculation process is as follows: in, This indicates the number of documents in the enterprise corpus. I This indicates the number of terms in the original question. i express I index, Indicates terms i In the h Frequency in a document Indicates included terms i The number of documents, This represents the average length of all documents in the enterprise corpus. This represents an adjustable parameter that controls word frequency. This represents an adjustable parameter that controls the document length.

6. The intelligent question-answering generation method based on triple retrieval query optimization according to claim 3, characterized in that, Original question and Relevance of vector retrieval The calculation process is as follows: in, Represents the dense vector in the original problem. express Dense vectors in Represents the weight coefficients of a dense vector; Represents the sparse vectors in the original problem. express sparse vectors in Represents the weight coefficients of a sparse vector; .

Images