Method for retrieving a chain of domain knowledge based on coding specifications
By constructing a trie and encoding standards, and combining knowledge encoding and reordering models, the retrieval process of the large language model is optimized, solving the problems of recalling irrelevant text blocks and missing key information, thereby improving retrieval efficiency and the quality of generated text.
Patent Information
- Application Number
- CN202511029400.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-25
- Publication Date
- 2025-11-07
- Estimated Expiration
- 2045-07-25
AI Technical Summary
Existing large language models tend to recall irrelevant text blocks and miss key information when generating text, and are also affected by irrelevant text blocks during cross-document retrieval, resulting in low computational efficiency.
By constructing a trie and adopting coding standards, and by calculating the similarity between the embedding vector of the query text and the summary of the document tree, combined with knowledge encoding and re-ranking models, the retrieval process is optimized. Multiple ranking and scoring calculation methods are used to reduce interference from irrelevant text blocks and improve the accuracy and efficiency of the recall results.
It effectively avoids recalling irrelevant text blocks, improves the quality and retrieval efficiency of large language model responses, reduces the interference of similar text blocks from irrelevant documents on recall results, and improves the accuracy and efficiency of generated text.
Smart Images
Figure CN120523928B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to a retrieval method, more particularly to a retrieval method based on a domain knowledge chain of coding specifications. BACKGROUND
[0002] A large language model refers to a neural network model with a large number of parameters and certain language capabilities. The large language model can generate relevant text according to human instructions. However, the text directly generated by the large language model may contain content inconsistent with facts, i.e. the hallucination phenomenon of the large language model. The retrieval enhanced generation technology first queries in the knowledge base according to the human instruction, inputs the recalled text and the human instruction into the large language model together, and provides correct, professional and timely knowledge for the large language model, so as to provide the quality of the text generated by the large language model.
[0003] The naive retrieval enhanced generation method first divides the document into text blocks, and then uses an embedding model to generate an embedding vector for each text block. For each inquiry of the user, the naive retrieval enhanced generation method first uses the embedding model to generate an embedding vector of the inquiry text, then calculates the cosine similarity of the inquiry text embedding vector and the embedding vector of each text block, and recalls a batch of text blocks with the highest similarity.
[0004] The naive retrieval enhanced generation method still has many defects. First, the naive retrieval enhanced generation method is limited by the capability of the embedding model, and it is very easy to recall text blocks irrelevant to the inquiry and miss text blocks with key information. Second, the naive retrieval enhanced generation method is easy to be disturbed by similar text blocks of irrelevant documents when cross-document retrieval, thereby affecting the recall result. Finally, the naive retrieval enhanced generation method needs to calculate the cosine similarity of the inquiry text embedding vector and the embedding vector of each text block, which has a certain disadvantage in time. SUMMARY
[0005] In view of the deficiencies of the prior art, the purpose of the present application is to provide a retrieval method based on a domain knowledge chain of coding specifications, which can effectively avoid the above-mentioned disadvantages.
[0006] To achieve the above-mentioned purpose, the present application provides the following technical scheme: a retrieval method based on a domain knowledge chain of coding specifications, comprising the following steps:
[0007] Step one, define the corresponding coding specification for different domains, and then build a dictionary tree The dictionary tree has an abstract The abstract is the parent node of all text blocks at the top level;
[0008] Step two, knowledge chain retrieval based on the constructed dictionary tree in step one by calculating the embedding vector of the query text , then calculating the similarity between the embedding vector of the query text and the summary of all document trees .
[0009] As a further improvement of the present application, the specific steps of knowledge chain retrieval in step two are as follows:
[0010] Step two one, calculating the embedding vector of the query text , then generating the code of the knowledge needed to answer the query ,
[0011] Step two two, according to the path given by the knowledge code get documents with the longest common prefix , and calculate the similarity between the embedding vector of the query text and the summary of all document trees , and select the tree with the highest similarity from the candidate text block set ,
[0012] Step two three, calculate the cosine similarity ranking of the embedding vector and the embedding vector of all candidate text blocks, and BM25 ranking of all candidate text blocks;
[0013] Step two four, calculate the comprehensive score using the rrf algorithm, and update the candidate text block set to the top text blocks in the current set by score;
[0014] Step two five, merge the obtained candidate text block set, and calculate the similarity between the query text and all text blocks in the merged candidate text block set, and return the top text blocks with the highest similarity as the retrieval result.
[0015] As a further improvement of the present application, the specific steps of constructing the dictionary tree in step one are as follows:
[0016] Step one one, maintain a dictionary tree ;
[0017] Step one two, for each document to be constructed , split it into several text blocks according to length without destroying the sentence , represents the 0th layer of the document tree , and the 0th text block of the 0th layer of the document tree ;
[0018] Step 13, calculate the embedding vectors of all text blocks using the embedding model, wherein the embedding vector of the 0th text block of the 0th document is ; ;
[0019] Step 14, for the 0th layer of the document , use the Gaussian Mixture Clustering algorithm to cluster the embedding vectors ;
[0020] Step 15, according to the clustering results, group the text blocks, and the text blocks in each group belong to the same class;
[0021] Step 16, then, traverse each group, and use the large language model to create a summary for the 0th text block in the 0th group, and the obtained summary is used as the text block of the next layer , which is the parent node of all text blocks in the 0th group; Step 17, repeat steps 13 to 15 for a predetermined number of times, and then use the large language model to create a summary for all text blocks in the top layer at this time, which is the summary of the document tree ,
[0022] , which is the parent node of all text blocks in the top layer; Step 18, use the summary of the document tree and the description of the coding specification under the current scenario to create the knowledge coding of the document tree using the large language model
[0023] , and add the coding to the dictionary tree . As a further improvement of the present application, the specific way of calculating the embedding vector of the query text in step 21 and then generating the coding of the knowledge required to answer the query is as follows: first, calculate the embedding vector
[0024] of the query text using the embedding model , and then use the large language model to generate the coding of the knowledge required to answer the query according to and the description of the coding specification under the current scenario .
[0025] As a further improvement of the present invention, in step two, the embedding vector of the query text is calculated. and a summary of all document trees The similarity is calculated, and the one with the highest similarity is selected. The tree obtains a set of candidate text blocks as The specific method is as follows: First, along the knowledge encoding... The given path, in the trie The search continues downwards until a leaf node is found or the next element of the knowledge code does not exist in the child nodes of the current node in the trie. This search path is called the knowledge chain. By tracing back along the knowledge chain, the sum can be obtained. Longest common prefix Each document is analyzed, and then the embedding vector of the query text is computed using a reordering model. and a summary of all document trees The similarity is calculated, and the one with the highest similarity is selected. For each selected document tree, initialize the candidate text block set as follows: .
[0026] As a further improvement of the present invention, in step two, the initialization candidate text block set is obtained as follows: Then, the candidate text block set is updated to the set of child nodes of all nodes in the current set.
[0027] As a further improvement of the present invention, in step two and four, the candidate text block set is updated to the one with the highest score in the current set. After each text block, step two is repeated to calculate the embedding vector of the query text using the reordering model. and a summary of all document trees The similarity is calculated, and the one with the highest similarity is selected. For each selected document tree, initialize the candidate text block set as follows: The process continues until a node in the candidate text block set is a leaf node.
[0028] As a further improvement of the present invention, in step two and four, the candidate text block set is updated to the one with the highest score in the current set. After each text block, step two is repeated to calculate the embedding vector of the query text using the reordering model. and a summary of all document trees The similarity is calculated, and the one with the highest similarity is selected. For each selected document tree, initialize the candidate text block set as follows: The process continues until a node in the candidate text block set is a leaf node.
[0029] Compared with the naive retrieval enhancement generation method mentioned in the background art, the present application effectively avoids the problems of easily recalling irrelevant text blocks and missing key information text blocks due to the limitation of the embedding model's ability by constructing a dictionary tree and a series of operations such as encoding specifications, etc. In the cross-document retrieval aspect, through knowledge encoding and dictionary tree searching and other steps, the interference of similar text blocks in irrelevant documents is reduced, and the accuracy of the recall result is improved. And in the process of calculating the similarity, multiple ranking and comprehensive score calculation methods are adopted, not just calculating the cosine similarity, which to some extent optimizes the time cost, improves the retrieval efficiency and the quality of the generated text, overcomes the many defects of the naive retrieval enhancement generation method, and compared with the existing technology, the method of the present application reduces the interference of irrelevant text blocks in relevant documents on the recall result, thereby improving the quality of the large model answer, and reduces the interference of similar text blocks in irrelevant documents on the recall result, thereby improving the quality of the large model answer. BRIEF DESCRIPTION OF DRAWINGS
[0030] Fig. 1 is a schematic diagram for defining corresponding encoding specifications according to different fields;
[0031] Fig. 2 is a schematic diagram of the construction process of the dictionary tree ;
[0032] Fig. 3 is a schematic diagram of the retrieval method based on the field knowledge chain of the encoding specification. DETAILED DESCRIPTION
[0033] The present application will be further described in detail below with reference to the embodiments given in the accompanying drawings.
[0034] Referring to Figs. 1 to 3 , a retrieval method based on the field knowledge chain of the encoding specification of the present embodiment mainly includes encoding specification definition, construction and retrieval.
[0035] The encoding specification in the present embodiment is named knowledge encoding, which is defined as follows:
[0036] The knowledge encoding should be composed of two parts appearing in sequence:
[0037] Six general encodings, each group of two bits.
[0038] Optional indefinite length extension encoding, each group of two bits.
[0039] The value range of each group of encoding is 00~99.
[0040] The first group of general encoding is the field major category encoding, for example, 06 represents technology.
[0041] The second group of the general coding is the domain subclass coding, which is further subdivided under the domain represented by the first group. For example, if the first group is 06, the second group 01 represents medicine.
[0042] The third group of the general coding is the subdivision direction. For example, if the first two groups are 06-01, the third group 16 represents the disease.
[0043] Extended coding is used to further subdivide knowledge. For example, in the case of the general coding 06-01-16, the first group of 01 in the extended coding represents cardiovascular disease.
[0044] Extended coding can continuously subdivide knowledge according to actual needs. For example, in the case of medical knowledge Q&A, subdividing the document into cardiovascular disease is still too broad. We can continue to add the next extended code, 02, to represent thrombosis.
[0045] In summary, the knowledge code for a document related to thrombosis prevention should be 06-01-16-01-02.
[0046] Based on this coding standard, the more relevant the content of a document, the longer its knowledge code will have a common prefix.
[0047] For general applications, this coding standard is compatible with both the Dewey Decimal Classification and the Chinese Library Classification. It can be used directly for comparison.
[0048] For specific professional fields, experts in the relevant fields can be organized to define the semantics of each code, thereby achieving the classification of subdivided documents.
[0049] The build process includes the following steps:
[0050] Maintaining a trie .
[0051] For each document to be built Without breaking the sentence, the text is split into several blocks according to its length. , Indicates the first The document tree at level 0 A text block.
[0052] The embedding vectors of all text blocks are computed using an embedding model, where the first... The first document The embedding vectors of the text blocks are .
[0053] For documents The Layers, using Gaussian mixture clustering algorithm for embedding vectors Perform clustering.
[0054] The text blocks are grouped according to the clustering results, and the text blocks in each group belong to the same class.
[0055] Then, iterate through each group and use the large language model to process the first group. Group text blocks create summaries, and the resulting summaries are used as text blocks in the next level. This text block is the first The parent node of all text blocks in the group.
[0056] Repeat steps 3-5 a specified number of times, and then use the large language model to create a summary of all text blocks at the top level, which will serve as the summary of the document tree. , It is the parent node of all text blocks at the top level.
[0057] Using the document tree summary This includes an explanation of coding standards for the current scenario, and the knowledge encoding of the document tree created using a large language model. And add the encoding to the trie. middle.
[0058] The search process includes the following steps:
[0059] Calculate query text using an embedding model. Embedded vector .
[0060] Using a large language model, based on Including a description of the coding standards for the current scenario, generate the coding of the knowledge required to answer the query. .
[0061] Along knowledge encoding The given path, in the trie The search continues downwards until a leaf node is found or the next element of the knowledge code does not exist in the child nodes of the current node in the trie. This search path is called the knowledge chain. Tracing back along the knowledge chain yields the results. Longest common prefix One document.
[0062] Calculate the embedding vector of the query text using a reordering model. and a summary of all document trees The similarity is calculated, and the one with the highest similarity is selected. A tree.
[0063] For each selected document tree, initialize the candidate text block set as follows: .
[0064] Update the candidate text block set to the set of children of all nodes in the current set.
[0065] Calculate the cosine similarity of the embedding vectors of all candidate text blocks, and the BM25 ranking of all candidate text blocks, and then use the rrf algorithm to calculate the comprehensive score.
[0066] Update the candidate text block set to the top text blocks in the current set by score.
[0067] Repeat steps (4)-(5) until the nodes in the candidate text block set are leaf nodes.
[0068] Merge the candidate text block sets obtained in all trees.
[0069] Calculate the similarity of the query text to all text blocks in the merged candidate text block set using the re-ranking model.
[0070] Return the top text blocks with the highest similarity as the retrieval results.
[0071] In summary, the retrieval method based on the domain knowledge chain of the coding specification in the present embodiment, relative to the naive retrieval enhancement generation method in the prior art, the method of the present embodiment and the naive retrieval enhancement generation method were tested on the UltraDomain-CS dataset, and the testing method was as follows:
[0072] Extract all references in the UltraDomain-CS dataset.
[0073] For the naive retrieval enhancement generation method, without destroying the sentence, all references were split into several text blocks according to the length, and the cosine similarity of the top 100 text blocks was found using the bce-embdding_v1 model as the embedding model, and then the bce-rerank_v1 model was used as the re-ranking model to reorder the top 100 text blocks, and the top three text blocks with the highest similarity were taken as the retrieval results
[0074] For the method proposed in the present embodiment, the Qwen2.5-72B-Instruct-AWQ model was used as the large language model, the bce-embdding_v1 model was used as the embedding model, the bce-rerank_v1 model was used as the re-ranking model, the number of layers of the tree was specified as 2, the maximum number of categories was 20, and the parameters during retrieval .
[0075] After obtaining the retrieval results by using the two methods, the Qwen2.5-72B-Instruct-AWQ model is used as a large language model to answer the questions in the data set according to the retrieval results.
[0076] The Qwen2.5-72B-Instruct-AWQ model is used as a large language model to judge the pros and cons of the answers obtained by the two retrieval results. If the retrieval results of the two methods are the same, it is considered as a draw.
[0077] The test results show that the method proposed in this embodiment wins 40 and loses 3, which indicates that the method proposed in this embodiment has obvious advantages.
[0078] The above only describes the preferred embodiments of the present application, and the protection scope of the present application is not limited to the above-mentioned embodiments. Any technical solutions falling within the concept of the present application shall fall within the protection scope of the present application. It should be noted that for ordinary technical personnel in the technical field, some improvements and refinements without departing from the principles of the present application shall also be considered as the protection scope of the present application.
Claims
1. A method for retrieving a domain knowledge chain based on coding specification, characterized in that: The method comprises the following steps: Step one, define the corresponding coding specification for different fields, and then build a dictionary tree The dictionary tree Has an abstract The abstract Is the parent node of all text blocks of the top layer; Step 2: Based on the trie constructed in Step 1 By calculating the query text Embedded vector Then calculate the embedding vector of the query text. and a summary of all document trees Knowledge chain retrieval is performed using similarity as a method; the specific steps for knowledge chain retrieval in step two are as follows: Step 21: Calculate the query text Embedded vector Then, it generates the encoding of the knowledge needed to answer the query. , Step two, encode knowledge Given path get and With the longest common prefix Documents, and calculate the embedding vector of the query text And the summary of all document trees Similarity, and select the highest similarity The tree obtains the candidate text block set as , Step two three, compute embedding vectors and cosine similarity ranking of embedding vectors of all candidate text blocks, and and BM25 ranking of all candidate text blocks; Step two four, use rrf algorithm to calculate the comprehensive score, update the candidate text block set to the highest score in the current set of text blocks one text block; Step two, the obtained candidate text block set is merged, and the similarity of all text blocks in the merged candidate text block set is calculated, and the text block with the highest similarity is returned as the search result. and the similarity of all text blocks in the merged candidate text block set, and the text block with the highest similarity is returned as the search result. 2. The method of claim 1, wherein: The step one constructs the dictionary tree The specific steps are as follows: Step one, maintain a dictionary tree ; Steps one and two: For each document to be built Without breaking the sentence, the text is split into several blocks according to its length. , Indicates the first The document tree at level 0 A text block; Step one three, calculate the embedding vector of all text blocks using the embedding model, where the embedding vector of the i-th text block of the document is . ; Step 14, for the document The Layers, using Gaussian mixture clustering algorithm for embedding vectors Perform clustering; Step one five, grouping the text blocks according to the clustering result, and the text blocks in each group belong to the same class; Step one six, then, iterate through each group, use large language model to create summary of first group of text blocks, resulting summary as text block for next level , this text block parent node for all text blocks of first group Step one seven, repeat step one three to step one five a predetermined number of times, then use a large language model to create a summary of all the text blocks at this time as the summary of this document tree , is the parent node of all the text blocks of the top layer; Step one eight, using the summary of the document tree and the specification of the encoding norm under the current scene, using a large language model to create the knowledge encoding of the document tree and adding the encoding to the dictionary tree .
3. The method of claim 2, wherein: In step two, the query text is calculated. Embedded vector Then, it generates the encoding of the knowledge needed to answer the query. The specific method is as follows: First, use an embedding model to calculate the query text. Embedded vector Then, using a large language model, based on Including a description of the coding standards for the current scenario, generate the coding of the knowledge required to answer the query. .
4. The method according to any one of claims 1 to 3, wherein the method is characterized by: In step two, the embedding vector of the query text is calculated. and a summary of all document trees The similarity is calculated, and the one with the highest similarity is selected. The tree obtains a set of candidate text blocks as The specific method is as follows: First, along the knowledge encoding... The given path, in the trie The search continues downwards until a leaf node is found or the next element of the knowledge code does not exist in the child nodes of the current node in the trie. This search path is called the knowledge chain. By tracing back along the knowledge chain, the sum can be obtained. Longest common prefix Each document is analyzed, and then the embedding vector of the query text is computed using a reordering model. and a summary of all document trees The similarity is calculated, and the one with the highest similarity is selected. For each selected document tree, initialize the candidate text block set as follows: .
5. The method of claim 4, wherein: The step two in the method is to obtain an initialization candidate text block set as After that, the candidate text block set is updated to be the set of child nodes of all nodes in the current set.
6. The method of claim 4, wherein: In step two and four, the candidate text block set is updated to the one with the highest score in the current set. After each text block, the embedding vector of the query text using the reordering model is calculated again in step two. and a summary of all document trees The similarity is calculated, and the one with the highest similarity is selected. For each selected document tree, initialize the candidate text block set as follows: The process continues until a node in the candidate text block set is a leaf node.
7. The method of claim 1 to 3, wherein the method is based on a domain knowledge chain of coding conventions. The candidate text blocks obtained in step two are merged and the similarity of all text blocks in the merged candidate text block set is calculated The text block with the highest similarity is returned as the retrieval result. The specific manner of returning the retrieval result is as follows: the candidate text blocks obtained in all trees are merged, the similarity of the query text and all text blocks in the merged candidate text block set is calculated, and the text block with the highest similarity is returned as the retrieval result.
Citation Information
Patent Citations
Engineering project digital archive management method and system based on BIM
CN118820563A
Short text query expansion enhancement retrieval method based on knowledge base hierarchical tree structure
CN118861088A