A RAG optimization method for large language models based on tree neighbor context

Abstract

The present invention discloses a large language model RAG optimization method based on tree neighbor context, including: reading the directory structure and content of the document, generating a corresponding directory tree and converting it into a Markdown format containing titles and levels, and automatically extracting titles according to the document template if there is no title; dividing the document into knowledge blocks according to priority; after recalling the knowledge blocks, expanding the context using the tree neighbor context method to form a new knowledge block list; grouping and sorting the expanded knowledge blocks according to document identifiers, splicing reference materials based on different fields and confidence levels, and generating a string set from the original document; using the constructed reference materials and the new knowledge block list, splicing prompt words and sending them to the large model for processing. The scheme of the present invention comprehensively enhances the performance and practicality of the large model in the process of information retrieval and answer generation by optimizing context information, emphasizing key content, adapting to a variety of document formats, and improving user experience.

Description

Technical Field

[0001] The present invention relates to the field of large model knowledge question answering applications, and in particular to a large language model RAG optimization method based on tree neighbor context. Background Art

[0002] Currently, in the application of RAG (retrieval-augmented generation) technology, documents are usually split into multiple knowledge blocks and stored in the database. Based on the user's question, the system recalls relevant knowledge blocks from the database, splices them into reference information, forms prompt words, and then passes them to the large language model. Since knowledge blocks are only fragments of knowledge and lack complete context information, the information in the prompt words often lacks context.

[0003] As RAG technology is widely used in various fields, this problem has gradually emerged. Large language models rely on the quality and completeness of input prompt words when generating answers. When the prompt words do not have enough reference information, especially when they lack context, the model may not be able to accurately understand the user's question, resulting in inaccurate answers or even errors. This not only affects the performance of the model, but may also mislead users.

[0004] Existing technologies have obvious deficiencies in dealing with the problem of missing context. First, the independent storage and recall of knowledge blocks ignores the correlation between knowledge blocks and fails to provide continuous and complete context. Second, the reference information in the prompt words is usually not arranged in the order of the original document, which increases the difficulty of understanding and reduces the possibility of the model generating accurate answers. In addition, the lack of a unified method in the processing and conversion of different types of document formats limits the scope of application of RAG technology.

[0005] Therefore, how to effectively solve the problem of insufficient reference information context of prompt words in RAG scenarios has become an important challenge in the current technical field. This requires a new method that can maintain efficient recall of knowledge chunks while providing sufficient context information to help large language models generate accurate and relevant answers. Summary of the invention

[0006] To overcome the shortcomings of the prior art, the present invention proposes a large language model RAG optimization method based on tree neighbor context. Through modular design, such as knowledge block segmentation, context expansion, and prompt word construction, each link can be adjusted and optimized according to actual needs. This flexibility allows the solution to adapt to different application scenarios and demand changes.

[0007] To achieve the above object, the present invention provides a large language model RAG optimization method based on tree neighbor context, comprising the following steps:

[0008] Step 1: Read the directory structure and content of the document, generate the corresponding directory tree and convert it into Markdown format containing titles and levels. If there is no title, it will be automatically extracted according to the document template.

[0009] Step 2: Divide the document into knowledge blocks according to priority (document directory, semantics, block size). Each knowledge block contains fields such as document ID, knowledge block ID, content, directory chain, summary and semantic block ID, and the number of tokens contained is roughly the same.

[0010] Step 3: After recalling the knowledge block, use the tree neighbor context method to expand the context. By adding relevant semantically similar blocks, sibling nodes, and parent nodes, a new list of knowledge blocks is formed to ensure that the total number of tokens is within the context length supported by the model.

[0011] Step 4: Group and sort the expanded knowledge blocks according to the document identifier, splice the reference materials based on different fields (such as directory chain, content, summary) and confidence (Rerank score), and finally generate a set of strings from the original document.

[0012] Step 5: Using the constructed reference material and the new list of knowledge chunks, the prompt words are assembled and sent to the big model for processing.

[0013] Furthermore, step 1 is as follows:

[0014] Step 1.1: Open and parse the source document, extract directory information (such as chapter titles and heading levels) and body content, identify headings and paragraphs in the document, and separate them for subsequent processing;

[0015] Step 1.2: Map the extracted titles, heading levels, and body content to Markdown format, using "#" to represent the first-level title, "##" to represent the second-level title, and so on, to ensure that the titles and body in the Markdown format are consistent with the structure and content of the source document;

[0016] Step 1.3: If there is no clear title in the document, automatically generate a title based on the document's template or content. You can use natural language processing technology to extract topic sentences from paragraphs as titles, set appropriate heading levels, and build a hierarchical structure for the document.

[0017] Step 1.4: Based on the heading levels in the Markdown format, construct a directory tree structure. Each heading corresponds to a node. The heading level reflects the hierarchical relationship of the nodes, ensuring that the directory tree accurately reflects the structure of the document, which is convenient for subsequent segmentation and retrieval.

[0018] Furthermore, step 2 is as follows:

[0019] Step 2.1: The priority order of segmentation is: document directory > semantics > block size. Segmentation should be based on the directory structure first to maintain the logical consistency of the content. If necessary, further segmentation can be performed based on semantics and block size.

[0020] Step 2.2: Divide the document into multiple parts according to the directory tree. Each part corresponds to a directory node or chapter, ensuring that each knowledge block contains a complete logical unit.

[0021] Step 2.3: In each chapter, analyze the semantics of the text, identify semantic units, use natural language processing technology to identify topics, keywords, and concepts, and group paragraphs or sentences with similar semantics into the same knowledge block.

[0022] Step 2.4: Calculate the number of tokens in each knowledge block and ensure that its size is within the preset range. If the knowledge block is too large, consider further segmentation; if it is too small, try to merge adjacent content;

[0023] Step 2.5: Add the necessary fields for each knowledge block:

[0024] Document ID: Identifies the source document to which the knowledge block belongs;

[0025] Knowledge Block ID: Assign a unique identifier to each knowledge block;

[0026] Content: The actual text content of the knowledge block;

[0027] Directory Chain: records the location of knowledge blocks in the document, including the document name, first-level directory, second-level directory, etc., separated by special symbols (such as "#");

[0028] Summary: A summary of the content of the current node and its subordinate nodes;

[0029] Starting from the leaf node, the summary of each knowledge block is generated layer by layer.

[0030] Semantic Block ID: Assign the same ID to knowledge blocks with similar semantics.

[0031] It helps to identify semantically related content in subsequent steps.

[0032] Furthermore, step 3 is as follows:

[0033] Step 3.1: Use multiple retrieval methods (such as keyword matching and semantic search) to recall knowledge blocks related to the user query and generate an initial list of knowledge blocks.

[0034] Step 3.2: De-duplicate the recalled knowledge block list, remove duplicate knowledge blocks, merge the knowledge block list, and prepare for sorting and expansion.

[0035] Step 3.3: Use a relevance algorithm or model to reorder the knowledge blocks according to the user query, where a higher score indicates a stronger relevance of the knowledge block to the query.

[0036] Step 3.4: Calculate the total number of tokens in the current knowledge block list, and determine the number of remaining tokens that can be used for expansion based on the maximum context length supported by the large model.

[0037] Step 3.5: Group the knowledge blocks according to the document identifiers. Each group represents a related knowledge block in a source document. Within the group, sort the knowledge blocks in descending order by the Rerank score. Then sort the groups in descending order according to the highest Rerank score in each group.

[0038] Step 3.6: Expand the context knowledge block for the grouping;

[0039] Step 3.7: Sort the knowledge blocks in each group in ascending order according to the knowledge block identifiers to ensure that the order of the knowledge blocks conforms to the logical structure of the document and facilitates model understanding.

[0040] Furthermore, the specific operations of step 3.6 are:

[0041] Step a: Create a new list for each knowledge block group and initialize a new list containing all knowledge blocks of the current group;

[0042] Step b: Traverse the knowledge blocks, check the context length, and for each knowledge block, check whether the context token number limit has been reached; if not, continue to expand; otherwise, stop expanding.

[0043] Step c: Get the knowledge block with the same semantic block identifier, find other knowledge blocks with the same semantic identifier as the current knowledge block, if these knowledge blocks are not in the new list, add them and update the token count, and get the sibling node knowledge block; if the current knowledge block is a leaf node, get its adjacent sibling nodes (previous and next), check and add the sibling nodes that are not in the new list; get the parent node knowledge block, in order from near to far, get the parent node of the current knowledge block, check and add the parent node knowledge block that is not in the new list;

[0044] Step d: Update the context length and check the limit. After each knowledge block is added, update the total number of tokens in the context; if the context length limit is reached or exceeded, stop the expansion process.

[0045] Step e: Repeat the above steps until all knowledge blocks have been processed or no more content can be added.

[0046] Furthermore, step 4 is as follows:

[0047] Step 4.1: Group each knowledge block and sort them in ascending order according to the knowledge block ID. This helps maintain the coherence of the content.

[0048] Step 4.2: Traverse the sorted knowledge blocks and concatenate them into strings according to their types and sources;

[0049] Step 4.3: Splice different fields according to the source of the knowledge block. For the original recalled knowledge block, semantically similar knowledge block, and sibling node knowledge block:

[0050] Concatenate the directory chain field, content field, and confidence (Rerank score).

[0051] Example format: [confidence] Directory chain: content

[0052] For the parent node knowledge block of the original recall knowledge block:

[0053] Concatenate the directory chain field, summary field, and confidence level.

[0054] Example format: [confidence] Directory chain: summary

[0055] Step 4.4: Set the confidence score, use the Rerank score of the knowledge block as its confidence, and for the expanded knowledge block, use the Rerank score of the original recalled knowledge block;

[0056] Step 4.5: Generate the final reference material set, combine all the concatenated strings to form a reference material list, each string corresponds to a knowledge block, and all the strings together constitute a complete reference material.

[0057] Furthermore, step 5 is as follows:

[0058] Step 5.1: Combine the user's query and reference materials into prompt words according to the preset template. Example template: User question: {user query}\nReference materials: \n{reference material list};

[0059] Sub-step 5.2: Check the length of the prompt word to ensure that the concatenated prompt word is within the maximum context length supported by the large model; if it exceeds the length limit, consider truncating the reference material or streamlining the content;

[0060] Step 5.3: Send the prompt words to the big model, input the constructed prompt words into the big model, and perform reasoning or generate answers;

[0061] Step 5.4: Receive and process the output of the large model, obtain the results returned by the large model, and check their completeness and accuracy;

[0062] Step 5.5: Return the result to the user, provide the answer of the large model to the user, and complete the whole process.

[0063] Furthermore, step 6 is also included, which is as follows:

[0064] Step 6.1: Clean and standardize user queries, including removing noise characters, special symbols, and extra spaces in the query; converting traditional Chinese characters to simplified Chinese characters and unifying terminology; detecting and correcting spelling errors in the query;

[0065] Step 6.2: Use natural language understanding (NLU) technology to identify the user’s main needs and goals, extract key entities, subject terms, and domain-specific terms in the query, and analyze the user’s sentiment inclination to adjust the system’s response.

[0066] Step 6.3: Use the vocabulary to replace the words in the query with synonyms or related words to enrich the query expression; according to the context, transform the query into a form that is more suitable for retrieval. If the query is incomplete, try to complete the query information based on historical interactions or domain knowledge;

[0067] Step 6.4: Extract core keywords from the processed query, and use the knowledge graph or word vector to find concepts or terms related to the keywords; combine keywords and extended vocabulary to form a diversified set of search terms to improve the coverage of knowledge block recall.

[0068] Furthermore, step 7 is also included, which is as follows:

[0069] Step 7.1: Compare the answers generated by the model with the information in the original knowledge base to ensure that the content is accurate and that the answers are logically consistent and have no inconsistencies.

[0070] Step 7.2: Optimize the language of the answer to improve readability and professionalism. According to the content of the answer, add paragraphs, lists or titles appropriately to enhance the hierarchy of the text and highlight the key points of the answer to facilitate users to quickly obtain important information.

[0071] Step 7.3: Add references to relevant knowledge blocks in your answers to facilitate user tracing; if allowed, provide links to the original documents or knowledge blocks for further reading;

[0072] Step 7.4: If you know the user's reading habits or professional background, adjust the depth and style of the answer and provide different forms of answers such as summaries, detailed descriptions, and charts to meet the user's diverse needs;

[0073] Step 7.5: Use user feedback data to continuously train and optimize the model to improve the quality of future answers.

[0074] Compared with the prior art, the present invention has the following beneficial effects:

[0075] 1. The present invention provides a large language model RAG optimization method based on tree neighbor context, which improves the quality of reference information by including sufficient and orderly context content in the prompt words. This enables the large model to understand a more comprehensive background when generating answers, thereby providing more accurate and high-quality replies. As the large model's support for longer conversation contexts is enhanced, the method can integrate more knowledge blocks, and the effect will be more significant.

[0076] 2. The present invention provides a large language model RAG optimization method based on tree neighbor context, which introduces the confidence field of the text block in the reference information of the prompt word, helping the large model to focus on the most relevant and important knowledge blocks. This mechanism improves the model's attention to key information and further improves the accuracy and relevance of the answer.

[0077] 3. The present invention provides a large language model RAG optimization method based on tree neighbor context, which converts the document into Markdown format before processing, so that the method can be applied to various document formats. Whether it is PDF, Word or HTML format, it can be uniformly converted and processed, which facilitates the parsing and knowledge extraction of different types of documents and enhances the versatility of the system.

[0078] 4. The present invention provides a large language model RAG optimization method based on tree neighbor context, and the reference information is sorted according to the original order of the content in the document, which conforms to human reading habits. This method helps users quickly locate key information during manual reading, improves the efficiency of problem troubleshooting and information retrieval, and reduces the time to find the required information in a large amount of data. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0080] Figure 1 It is a flow chart of the steps of the present invention;

[0081] Figure 2 A data preparation flow chart provided for an embodiment;

[0082] Figure 3 A flowchart of generating document summaries provided in an embodiment. DETAILED DESCRIPTION

[0083] The technical solution of the present invention will be more clearly and completely explained below through description of preferred embodiments of the present invention in combination with the accompanying drawings.

[0084] Terminology explanation:

[0085] RAG (Retrieval-Augmented Generation): Retrieval-augmented generation technology.

[0086] Token: In natural language processing, the smallest unit into which text is segmented, such as a word, subword, or character. Large language models are usually processed in token units, and the number of tokens also affects the input limit of the model.

[0087] Markdown: A lightweight markup language.

[0088] like Figure 1 As shown, the present invention is specifically:

[0089] The following steps are involved:

[0090] Step 1: Read the directory structure and content of the document, generate the corresponding directory tree and convert it into Markdown format with titles and levels. If there is no title, it will be automatically extracted according to the document template;

[0091] Step 2: Divide the document into knowledge blocks according to priority. Each knowledge block contains fields and the number of tokens contained is roughly the same.

[0092] Step 3: After recalling the knowledge block, use the tree neighbor context method to expand the context. By adding semantically similar blocks, sibling nodes, and parent nodes, a new list of knowledge blocks is formed to ensure that the total number of tokens is within the context length supported by the model;

[0093] Step 4: Group and sort the expanded knowledge blocks according to the document identifiers, concatenate reference materials based on different fields and confidence levels, and generate a set of strings from the original documents;

[0094] Step 5: Using the constructed reference material and the new list of knowledge chunks, the prompt words are assembled and sent to the big model for processing.

[0095] As a specific implementation method, the directory structure of the document is read to generate a directory tree of the document. In particular, if there is no directory in the document structure, a directory tree is automatically extracted according to the document template;

[0096] like Figure 2 As shown, select the appropriate component parsing file according to the document type entered by the user, and convert the document into mardown format. For example, select python-docx parsing for docx

[0097] 1. Read DOCX file: Use Document(docx_path) to open the specified DOCX file;

[0098] 2. Traverse paragraphs: Traverse all paragraphs in the document and use getattr(paragraph, 'toc_level',-1) to get the toc_level attribute. The default value is -1.

[0099] 3. Determine the title: If toc_level is greater than -1, it means that the paragraph is a title. Calculate the Markdown title level (the number of #) based on toc_level;

[0100] 4. Normal paragraph: If it is not a title, the paragraph text is added directly to the Markdown content;

[0101] 5. Save the result: Finally, write the generated Markdown content into a file.

[0102] If the generated markdown does not have a directory, a directory tree is automatically extracted based on the document template.

[0103] Steps:

[0104] 1. Traverse the text lines: check the content of the Markdown document line by line;

[0105] 2. Matching features: Use regular expressions to identify text lines that match a specific format to determine their directory level. For example:

[0106] The lines that conform to the format of "Article 1, Article 2, Article 3" are used as the first-level directory;

[0107] The lines that conform to the "1, 2, 3" format are used as secondary directories;

[0108] Other formats (such as "1.1, 1.2", etc.) can be used as lower-level directories;

[0109] 3. Add directory markers: According to the matching features, add a corresponding number of # symbols to each text line to indicate its level in the directory;

[0110] 4. Generate a directory tree: Aggregate the processed directory lines to form a complete multi-level directory structure.

[0111] Divide the markdown document into knowledge chunks according to the markdown document directory, semantics, and chunk size (chunk_size):

[0112] 1. Split by title: First, split according to the title to form preliminary knowledge blocks. If the knowledge block after segmentation is smaller than the specified chunk_size, no further segmentation will be performed;

[0113] 2. Semantic segmentation: Use open source components (such as LangChain) to semantically segment the preliminary knowledge blocks. If the block after semantic segmentation is still smaller than chunk_size, it will not be segmented again;

[0114] 3. Further segmentation: For semantic segmentation blocks larger than chunk_size, further segmentation is performed according to chunk_size. The segmented blocks will share the same brother_chunk_id to identify that they are from the same semantic block. If there is no further segmentation, brother_chunk_id is set to -1.

[0115] The construction process of the document summary tree:

[0116] 1. Parse the directory structure of each document and identify hierarchical elements such as chapters and subsections. These structural elements are regarded as nodes of information, and each node contains specific text content;

[0117] 2. Use the Map phase in the MapReduce framework to analyze the text content of each node and extract key information, such as generating summaries through large models;

[0118] 3. In the Reduce phase, the summary information of the child nodes is merged into the parent node. Through recursive merging, a comprehensive document summary is finally generated at the top-level node of the document;

[0119] 4. Combine the hierarchical summaries of all documents to form a complete document summary tree.

[0120] The document summary number generation flow chart provided in the embodiment is as follows Figure 3 As shown;

[0121] The following is an example of a processed knowledge block:

[0122] {

[0123] "doc_id": 15101,

[0124] "chunk_id": 4,

[0125] "heading_chain": "ZMQ command line operation and maintenance#Start and stop the service#Start and stop Namesrv and Broker#Execution steps",

[0126] "content": "1. First check whether there are namesrv and broker processes, and determine their installation paths...",

[0127] "summary": "This chapter mainly introduces how to check and manage the naming service (namesrv) and message queue service (broker) processes.....",

[0128] "type": "text",

[0129] "brother_chunk_id": -1,

[0130] }

[0131] The knowledge blocks are stored in the full-text search library and the vector library. In this embodiment, elastic search 8 is selected, which has both full-text search capabilities and vector search capabilities.

[0132] Based on the questions entered by the user, multi-channel recall of knowledge blocks is performed. The multi-channel recall can include the following six channels, such as: full-text search / vector of the title field, full-text search / vector of the body field, and full-text search / vector of the abstract field. Approximately 60 knowledge blocks can be recalled.

[0133] After multiple recalls of knowledge block lists, the knowledge block lists are merged to form a new knowledge block list. After deduplication of the knowledge list, there will be about 40-50 knowledge blocks left.

[0134] Rerank the knowledge block list according to the user's question to obtain a reordered knowledge block list. Each element in the list has a score, which represents the relevance of the current knowledge block to the user's question. The higher the score, the higher the relevance to the user's question.

[0135] The following is the knowledge block after reranking, with the score field added:

[0136] {

[0137] "score":0.5809610486030579

[0138] "data":{

[0139] "doc_id": 15101,

[0140] "chunk_id": 4, ... ...

[0141] }

[0142] }

[0143] Rerank is to reorder the multiple chunk lists retrieved so that their rankings better match the relevance of the user input question, so that more relevant and accurate chunks are ranked higher, so that they can be given priority when LLM is generated to improve the output quality.

[0144] The Rerank operation needs to call the Rerank model. The model can choose the SaaS version of the online model service, or the privately deployed open source Rerank model. The Rerank model used in this embodiment is bge-reranker-large. bge-reranker-large is a widely used Rerank model of domestic Zhiyuan open source, and has excellent results in many model tests. Due to the large amount of calculation, the Rerank model requires a graphics card. The graphics card used can be NVIDIA T4 / 2080TI / 3090 / etc., or it can be domestic Huawei 910B / Guoguang, etc.

[0145] Discard the knowledge blocks with lower Rerank scores. Generally, if the score is less than 0.001, it is considered that the relevance of the knowledge block to the question is very low and can be discarded. At this time, there are about 20 knowledge blocks left.

[0146] Calculate the number of tokens occupied by a single knowledge block;

[0147] Use the following python code to concatenate strings:

[0148] chunk_str = f"Title:{item['data']['heading_chain']} Confidence:{item['score']}\nContent:{item['data']['content']}\n"

[0149] The concatenated string example is:

[0150] Title: [heading_chain] Confidence: [score]

[0151] Content: [content]

[0152] Use the tokenizer tool in Python to calculate the token length of the string. Tokenizer is a tool used to split text into tokens, which is usually used in natural language processing tasks. It can be expressed as:

[0153] ;

[0154] Calculate the number of tokens occupied by a single knowledge list

[0155] The formula for calculating the number of tokens in the current knowledge block list is as follows:

[0156] ;

[0157] The formula for calculating the number of context tokens to be expanded is as follows:

[0158] ;

[0159] The knowledge blocks are grouped according to their document identifiers, and the knowledge blocks in the group are sorted in descending order according to the Rerank scores. The groups are sorted in descending order according to the highest Rerank score of the block in the group, so that the knowledge blocks in each group come from the same document.

[0160] Each knowledge block in a group has a Rerank score field, which represents the degree of relevance to the question asked by the user. The knowledge blocks are sorted in descending order according to the Rerank score field within the group. The knowledge block at the front has the highest relevance to the question asked by the user, which serves as the relevance between the document to which it belongs and the question asked.

[0161] According to the relevance between the document and the question, sort the knowledge block groups in descending order. The knowledge block groups at the front have a higher relevance to the question asked by the user, and the context is expanded first;

[0162] Create a second-layer loop:

[0163] The first layer loops through the knowledge block groups, creates a new list for each group, copies the knowledge block list to the new list, and places the knowledge blocks expanded by subsequent operations into the new list.

[0164] The knowledge blocks in the second-level loop grouping are contextual knowledge blocks that are expanded upon the looped knowledge blocks.

[0165] The logic of the second loop is as follows:

[0166] Step 1: Determine whether the total number of tokens of all currently expanded knowledge blocks is sufficient. If it is sufficient, exit the loop and stop expanding.

[0167] Step 2: Get the knowledge block and sibling knowledge block with the same semantic identifier as the current knowledge block, and determine whether the acquired knowledge block is already in the new list. If it is already in the current group, skip it and continue to the next knowledge block. Otherwise, add a score field to the newly acquired knowledge block (the value is the score value of the original knowledge block) and add the newly acquired knowledge block to the new list. Use the following python code to concatenate strings and increase the context length;

[0168] chunk_str = f"Title:{item['data']['heading_chain']} Confidence:{item['score']}\nContent:{item['data']['content']}\n"

[0169] Step 3: Get the parent knowledge block of the current knowledge block in order from recent to far, and determine whether the acquired knowledge block is already in the new list. If it is already in the current group, skip it and continue to the next knowledge block. Otherwise, add a score field to the newly acquired knowledge block (the value is the score value of the original knowledge block) and add the newly acquired knowledge block to the new list. Use the following python code to concatenate strings and increase the context length;

[0170] chunk_str = f"Title:{item['data']['heading_chain']} Confidence:{item['score']}\nContent:{item['data']['summary']}\n"

[0171] Step 4: Continue to the next knowledge block cycle

[0172] Use the new knowledge block list to piece together reference materials and construct prompt words to send to the big model

[0173] Step 1: Sort the knowledge blocks of each group in ascending order according to the knowledge block identification (id) field

[0174] sorted_chunks = sorted(knowledge_chunks, key=lambda x: x['id'])

[0175] Step 2: Splice reference materials

[0176] final_output = ""

[0177] for item in sorted_chunks:

[0178] if item['source'] == 'original':

[0179] # Original knowledge block

[0180] chunk_str = f"Title:{item['data']['heading_chain']} Confidence:{item['score']}\nContent:{item['data']['content']}\n"

[0181] if item['is_related'] or item['is_sibling']:

[0182] chunk_str = f"Title:{item['data']['heading_chain']} Confidence:{item['score']}\nContent:{item['data']['content']}\n"

[0183] final_output += chunk_str

[0184] if item['is_parent']:

[0185] chunk_str = f"Title:{item['data']['heading_chain']} Confidence:{item['score']}\nContent:{item['data']['summary']}\n"

[0186] final_output += chunk_str

[0187] The final reference consists of several strings, each of which comes from a document.

[0188] As a specific embodiment, the string is illustrated as follows:

[0189] Title: ZMQ Product Introduction#Function Overview#ZMQ Function OverviewConfidence: 0.015594851225614548

[0190] Content: ZMQ is a low-latency, high-concurrency, high-availability, and highly reliable distributed messaging middleware built on RocketMQ. ZMQ not only maintains the native functions of RocketMQ, but also extends and expands the existing functions to improve the scalability, availability, and concurrency of the entire system.

[0191] Title: ZMQ Product Introduction#Technical Architecture Confidence: 0.11433449387550354

[0192] Content: ZMQ as a whole consists of several parts: producer, consumer, namesrv, broker, zookeeper

[0193] Title: ZMQ Product Introduction#Technical Architecture Confidence: 0.019713740795850754

[0194] Content: Namesrv cluster ZK cluster Producer cluster Consumer cluster MasterSlaveMasterSlavemanager

[0195] Title: ZMQ Product Introduction#Technical Architecture Confidence: 0.17502164840698242

[0196] Content: Production side:

[0197] The production side is clustered and deployed, and the message is sent to the broker through Namesrv routing.

[0198] The consumer side is clustered and deployed, routing is performed through Namesrv, and messages are pulled from the broker for consumption.

[0199] Title: ZMQ Product Introduction#Technical Architecture#broker: Confidence: 0.30359387397766113

[0200] Content: Message transfer role, actually providing services to the outside world (the broker's address is perceived through namesrv, and the business does not need to configure this address), responsible for storing and forwarding messages.

[0201] Provide high-concurrency read and write services, load balancing and dynamic scaling.

[0202] Title: ZMQ Product Introduction#Technical Architecture#namesrv: Confidence: 0.08171118050813675

[0203] Content: Lightweight registration center, stateless node, provides naming services, responsible for updating and discovering Broker services. Producers and consumers can obtain relevant Broker information from the Name Server at regular intervals (30 seconds).

[0204] There is no communication between multiple Namesrv nodes. The failure of a single Namesrv node does not affect the functions of other Namesrv nodes and the cluster. Even if the entire Namesrv cluster fails, the Producers, Consumers, and Brokers that have been working normally can still work normally, but the new Producers, Consumers, and Brokers will not work.

[0205] Title: ZMQ Product Introduction#Technical Architecture#zookeeper: Confidence: 0.10208921879529953

[0206] Content: Introduce Zookeeper for failover and metadata management.

[0207] Namesrv and broker monitor the changes of zk nodes and coordinate the roles and conversions between the primary and backup nodes through the changes of zk nodes.

[0208] As a specific implementation method, first, start by generating a directory tree for the document. Read the directory structure and content of the document, extract titles (such as chapter titles and heading levels) and body content. This can be done by using Python libraries to read documents in different formats, such as PDF, Word, or HTML. Next, convert the document to Markdown format and map the extracted titles and body content to Markdown syntax. The title uses "#" to represent a first-level title, "##" to represent a second-level title, and so on. If there is no clear title in the document, use natural language processing technology to automatically generate a title based on the document's template or content.

[0209] Then, build the directory tree of the document, based on the heading level of the Markdown document, using a tree data structure, such as a nested dictionary. Next, split the document into knowledge blocks. Segment according to the priority of "document directory, semantics, and block size". First, divide the document into multiple parts according to the directory tree, and each part corresponds to a directory node or chapter to ensure that each knowledge block contains a complete logical unit. Within each chapter, further analyze the semantics of the text and identify semantic units. Techniques such as topic modeling or sentence embedding can be used to cluster sentences and classify paragraphs or sentences with similar semantics into the same knowledge block.

[0210] Control the size of each knowledge block, calculate the number of tokens of each knowledge block, and ensure that its size is within the preset range. If the knowledge block is too large, consider further segmentation; if it is too small, try to merge adjacent content. Add necessary fields to each knowledge block, including document identifier, knowledge block identifier, content, directory chain, summary, and semantic block identifier. The directory chain records the location of the knowledge block in the document, including the document name, first-level directory, second-level directory, etc., separated by special symbols (such as "#"). The summary is a summary of the content of the current node and subordinate nodes. Starting from the leaf node, the summary of each knowledge block is generated layer by layer. The semantic block identifier is used to give the same identifier to knowledge blocks with similar semantics, which helps to identify semantically related content in subsequent steps.

[0211] After recalling the knowledge blocks, expand the context to form a new list of knowledge blocks. Use a variety of retrieval methods, such as keyword matching and semantic search, to recall knowledge blocks related to the user query from the knowledge base and generate an initial list of knowledge blocks. De-duplicate the recalled knowledge block list, remove duplicate knowledge blocks, merge all unique knowledge blocks, and prepare for sorting and expansion. Re-sort the knowledge blocks, using a relevance algorithm or model to re-sort the knowledge blocks according to the user query. The higher the score, the more relevant the knowledge block is to the query. Calculate the total number of tokens in the current knowledge block list, and determine the number of remaining tokens that can be used for expansion based on the maximum context length supported by the large model.

[0212] Group the knowledge blocks according to the document identifier, sort the knowledge blocks in the group in descending order according to the reordered scores, and sort the groups in descending order according to the highest score in the group. Expand the context knowledge blocks for each knowledge block group. First, create a new list for each group, containing all the knowledge blocks in the current group. Then traverse the knowledge blocks and check whether the context token number limit has been reached. If not, continue to expand; otherwise, stop expanding. Get the related knowledge blocks. First, find other knowledge blocks with the same semantic identifier as the current knowledge block. If these knowledge blocks are not in the new list, add them and update the token count. Next, if the current knowledge block is a leaf node, get its adjacent sibling nodes (previous and next), check and add the sibling nodes that are not in the new list. Finally, get the parent nodes of the current knowledge block in order from near to far, check and add the parent node knowledge blocks that are not in the new list. After each addition, update the total number of tokens in the context. If the context length limit is reached or exceeded, stop the expansion process. Repeat the above steps until all knowledge blocks have been processed or no more content can be added.

[0213] Construct reference materials, group each knowledge block, sort in ascending order according to the knowledge block identifier, traverse the sorted knowledge blocks, and splice them into strings according to their type and source. According to the source of the knowledge block, use different fields to splice the reference materials, and use the re-sorting score of the knowledge block as its confidence. For the expanded knowledge block, use the score of the original recalled knowledge block. For the original recalled knowledge block, semantically similar knowledge blocks, and sibling node knowledge blocks, splice the directory chain field, content field, and confidence. For the parent node knowledge block of the original recalled knowledge block, splice the directory chain field, summary field, and confidence. Combine all the spliced ​​strings to form a reference material list. Each string corresponds to a knowledge block, and all strings together constitute a complete reference material.

[0214] Construct prompt words and send them to the big model. Use templates to combine user queries and reference materials, for example: "User question: {user query}\nReference materials:\n{reference material list}". Check the length of the prompt words to ensure that the concatenated prompt words are within the maximum context length supported by the big model. If the length limit is exceeded, consider truncating the reference materials or streamlining the content. Input the constructed prompt words into the big model, perform reasoning or generate answers, send the prompt words and obtain responses. Obtain the results returned by the big model, check their completeness and accuracy, and provide the optimized answers to the user to complete the entire process.

[0215] Clean and standardize user queries, remove noise characters, special symbols, extra spaces, convert traditional Chinese characters to simplified Chinese, unify terminology, and correct spelling errors. Use natural language processing technology to identify user intent and extract entities, and perform sentiment analysis to adjust responses. Expand and rewrite queries, replace synonyms, rewrite queries to suit retrieval, and complete context. Generate diverse search terms, extract keywords, use knowledge graphs or word vectors to expand related concepts, generate diverse search term sets, and improve the coverage of knowledge block recall.

[0216] Verify the accuracy of the model's answers, compare the answers generated by the model with the information in the original knowledge base to ensure the accuracy of the content, and ensure that the answers are logically self-consistent and without inconsistencies. Optimize the language of the answers to improve readability and professionalism. According to the content of the answers, add paragraphs, lists or titles as appropriate to enhance the hierarchy of the text. Highlight the key points in the answers to facilitate users to quickly obtain important information. Add reference marks to related knowledge blocks in the answers, and provide links to original documents or knowledge blocks for users to read further. Adjust the depth and style of the answers according to user preferences, and provide answers in different forms such as summaries, detailed explanations, and charts to meet the diverse needs of users. Implement a feedback mechanism to collect users' evaluation of the answers, and use feedback data to optimize the model and improve the quality of future answers.

[0217] The above specific implementations are only descriptions of the preferred implementations of the present invention, and do not limit the protection scope of the present invention. Without departing from the design concept and spirit of the present invention, various modifications, substitutions and improvements made by ordinary technicians in this field to the technical solution of the present invention based on the text description and drawings provided by the present invention should all fall within the protection scope of the present invention. The protection scope of the present invention is determined by the claims.

Claims

1. A large language model RAG optimization method based on tree neighbor context, characterized in that: The following steps are involved: Step 1: Read the directory structure and content of the document, generate the corresponding directory tree and convert it into Markdown format with titles and levels. If there is no title, it will be automatically extracted according to the document template; Step 2: Divide the document into knowledge blocks according to priority. Each knowledge block contains fields, and the number of tokens contained is within a predetermined range; specifically, it includes: Divide the markdown document into knowledge chunks according to the markdown document directory, semantics, and chunk size chunk_size; Split by title: First, split according to the title to form preliminary knowledge blocks. If the knowledge block after segmentation is smaller than the specified chunk_size, no further segmentation will be performed; Semantic segmentation: Use open source components to semantically segment the preliminary knowledge blocks. If the block after semantic segmentation is still smaller than chunk_size, it will not be segmented again. Further segmentation: For semantic segmentation blocks larger than chunk_size, further segmentation is performed according to chunk_size. The segmented blocks will share the same brother_chunk_id to indicate that they are from the same semantic block. If there is no further segmentation, brother_chunk_id is set to -1. Step 3: Use the retrieval method to recall the knowledge blocks related to the user query, expand the context, and form a new list of knowledge blocks by adding semantically similar blocks, sibling nodes, and parent nodes to ensure that the total number of tokens is within the context length supported by the model; Step 3 is as follows: Step 3.1: Use the retrieval method to recall the knowledge blocks related to the user query and generate an initial list of knowledge blocks; Step 3.2: De-duplicate the recalled knowledge block list, remove duplicate knowledge blocks, merge the knowledge block list, and prepare for sorting and expansion; Step 3.3: Use a relevance algorithm or model to reorder the knowledge blocks according to the user query. The higher the score, the more relevant the knowledge block is to the query. Step 3.4: Calculate the total number of tokens in the current knowledge block list, and determine the number of remaining tokens that can be used for expansion based on the maximum context length supported by the large model; Step 3.5: Group the knowledge blocks according to the document identifiers. Each group represents a related knowledge block in a source document. Within the group, sort the knowledge blocks in descending order by the Rerank score. Then sort the groups in descending order according to the highest Rerank score in each group. Step 3.6: Expand the context knowledge block for the grouping; Step 3.7: Sort the knowledge blocks in each group in ascending order according to the knowledge block identifiers to ensure that the order of the knowledge blocks conforms to the logical structure of the document and is easy for the model to understand; Step 4: Group and sort the expanded knowledge blocks according to the document identifiers, and splice the reference materials based on different fields and confidence levels to generate a string set from the original document; the spliced ​​reference materials are as follows: Construct reference materials, group each knowledge block, sort them in ascending order according to the knowledge block identifier, traverse the sorted knowledge blocks, and concatenate them into strings according to type and source; According to the source of the knowledge block, different fields are used to splice the reference materials, and the re-ranking score of the knowledge block is used as its confidence; For the expanded knowledge chunks, the scores of the original recalled knowledge chunks were used; For the original recalled knowledge blocks, semantically similar knowledge blocks, and sibling node knowledge blocks, concatenate the directory chain field, content field, and confidence level; For the parent node knowledge block of the original recalled knowledge block, concatenate the directory chain field, summary field and confidence; Combine all the concatenated strings to form a reference list, where each string corresponds to a knowledge block and all the strings together constitute a complete reference; Step 5: Using the constructed reference material and the new list of knowledge chunks, the prompt words are assembled and sent to the big model for processing.

2. The large language model RAG optimization method based on tree neighbor context according to claim 1, characterized in that: Step 1 is as follows: Step 1.1: Open and parse the source document, extract the directory information and text content, identify the titles and paragraphs in the document, and separate them for subsequent processing; Step 1.2: Map the extracted titles, heading levels, and body content to Markdown format, using "#" to represent the first-level title, "##" to represent the second-level title, and so on, to ensure that the titles and body in the Markdown format are consistent with the structure and content of the source document; Step 1.3: If there is no clear title in the document, automatically generate a title based on the document template or content, use natural language processing technology to extract topic sentences from paragraphs as titles, set appropriate title levels, and build a hierarchical structure for the document; Step 1.4: Based on the heading levels in the Markdown format, construct a directory tree structure. Each heading corresponds to a node. The heading level reflects the hierarchical relationship of the nodes, ensuring that the directory tree accurately reflects the structure of the document, which is convenient for subsequent segmentation and retrieval.

3. The large language model RAG optimization method based on tree neighbor context according to claim 1, characterized in that: Step 2 is as follows: Step 2.1: The priority order of segmentation is: document directory > semantics > block size. Segmentation is prioritized according to the directory structure to maintain the logical consistency of the content, and further segmentation is performed based on semantics and block size; Step 2.2: Divide the document into multiple parts according to the directory tree. Each part corresponds to a directory node or chapter, ensuring that each knowledge block contains a complete logical unit. Step 2.3: In each chapter, analyze the semantics of the text, identify semantic units, use natural language processing technology to identify topics, keywords, and concepts, and group paragraphs or sentences with similar semantics into the same knowledge block; Step 2.4: Calculate the number of tokens in each knowledge block and ensure that its size is within the preset range. If the knowledge block is too large, consider further segmentation; if it is too small, try to merge adjacent content; Step 2.5: Add necessary fields to each knowledge block, starting from the leaf node, and generate the summary of each knowledge block layer by layer.

4. The large language model RAG optimization method based on tree neighbor context according to claim 1, characterized in that: The specific operations of step 3.6 are as follows: Step a: Create a new list for each knowledge block group and initialize a new list containing all knowledge blocks of the current group; Step b: Traverse the knowledge blocks and check the context length. For each knowledge block, check whether the context token number limit has been reached; if not, continue to expand; Otherwise, stop expanding; Step c: Get the knowledge block with the same semantic block identifier, and search for other knowledge blocks with the same semantic identifier as the current knowledge block. If these knowledge blocks are not in the new list, add them and update the token count. Get the knowledge block of the sibling node. If the current knowledge block is a leaf node, get its adjacent sibling nodes, check and add the sibling nodes that are not in the new list; Get the parent node knowledge block, get the parent node of the current knowledge block in order from nearest to farthest, check and add the parent node knowledge block that is not in the new list; Step d: Update the context length and check the limit. After each knowledge block is added, update the total number of tokens in the context. If the context length limit is reached or exceeded, stop the expansion process. Step e: Repeat the above steps until all knowledge blocks have been processed or no more content can be added.

5. The large language model RAG optimization method based on tree neighbor context according to claim 1, characterized in that: Step 4 is as follows: Step 4.1: Group each knowledge block and sort them in ascending order according to the knowledge block ID. This helps maintain the coherence of the content. Step 4.2: Traverse the sorted knowledge blocks and concatenate them into strings according to their types and sources; Step 4.3: Concatenate different fields according to the source of the knowledge block. For the original recalled knowledge block, semantically similar knowledge block, and sibling node knowledge block, concatenate the directory chain field, content field, and confidence. For the parent node knowledge block of the original recalled knowledge block, concatenate the directory chain field, summary field, and confidence. Step 4.4: Set the confidence score, use the Rerank score of the knowledge block as its confidence, and for the expanded knowledge block, use the Rerank score of the original recalled knowledge block; Step 4.5: Generate the final reference material set, combine all the concatenated strings to form a reference material list, each string corresponds to a knowledge block, and all the strings together constitute a complete reference material.

6. The large language model RAG optimization method based on tree neighbor context according to claim 1, characterized in that: Step 5 is as follows: Step 5.1: Splice the user's query and reference materials into prompt words according to the preset template; Step 5.2: Check the length of the prompt word to ensure that the concatenated prompt word is within the maximum context length supported by the large model; if it exceeds the length limit, consider truncating the reference material or streamlining the content; Step 5.3: Send the prompt words to the big model, input the constructed prompt words into the big model, and perform reasoning or generate answers; Step 5.4: Receive and process the output of the large model, obtain the results returned by the large model, and check their completeness and accuracy; Step 5.5: Return the result to the user, provide the answer of the large model to the user, and complete the whole process.

Images