Search enhancement generation system based on thinking chain and data processing method

Abstract

The invention discloses a thinking chain-based retrieval enhancement generation system and a data processing method. The system comprises a user interface module, an answer generation module, an answer segmentation module, an answer correction module, a knowledge retrieval module and an answer optimization module. The user interface module, the answer generation module, the answer segmentation module, the answer correction module and the answer optimization module sequentially form closed-loop connection, and the answer correction module is further connected with the knowledge retrieval module. According to the method, the thinking chain is combined with the RAG, each thinking step is modified one by one by using information obtained from knowledge retrieval, the transparency and the accuracy of answers are improved, the problem of error accumulation caused by single retrieval and global generation of the RAG in a complex task is solved through step-by-step reasoning and dynamic retrieval correction, and the accuracy of the RAG is improved. And the method is especially applicable to complex scenes requiring multi-step verification.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a retrieval enhancement generation system based on thought chain and a data processing method. Background Art

[0002] Large language models (LLMs) can acquire extensive knowledge about various topics by training on large data sets. The output of large language models is essentially a series of numerical operations. Large language models may have knowledge gaps or hallucination problems, especially in scenarios where the large language model itself does not have knowledge or is not good at a certain aspect. It is difficult to distinguish these problems because it requires the user to have knowledge in the corresponding field.

[0003] In order to avoid the knowledge deficiencies and hallucinations of the large language model, a retrieval augmented generation (RAG) method is currently used. The user's question is retrieved through the knowledge base of the own knowledge base to obtain relevant information, which is then merged with the original prompt words of the large language model into a prompt word template to generate beautiful answers, avoiding the knowledge limitations and hallucinations of the large language model LLM.

[0004] However, the answers currently obtained through RAG lack transparency, and users cannot track how large language models integrate the content of knowledge retrieval. In addition, RAG's error accumulation problem in complex tasks will also affect the reliability of the final answer.

[0005] Therefore, the prior art still needs to be improved and developed. Summary of the invention

[0006] The main purpose of the present invention is to provide a retrieval enhancement generation system and data processing method based on thought chain, aiming to solve the problem in the prior art that the answers obtained through RAG lack transparency and the accumulated errors of RAG will affect the reliability of the final answer.

[0007] To achieve the above object, the present invention provides a retrieval enhancement generation system based on thought chain, the retrieval enhancement generation system based on thought chain includes: The user interface module, the answer generation module, the answer segmentation module, the answer correction module and the answer optimization module are sequentially connected to form a closed loop, and the answer correction module is also connected to the knowledge retrieval module; The user interface module is used to receive questions input by users and send the questions to the answer generation module; The answer generation module is used to process the question, generate a preliminary answer, and send the preliminary answer to the answer segmentation module; The answer segmentation module is used to segment and reorganize the preliminary answer according to a preset structure to obtain a plurality of initial answer paragraphs, package the plurality of initial answer paragraphs to obtain a thought chain, and send the thought chain and the preliminary answer to the answer correction module; The answer correction module is used to generate a query statement for each of the initial answer paragraphs in the thought chain, and send a plurality of the query statements to the knowledge retrieval module; The knowledge retrieval module is used to generate retrieval results according to the multiple query statements respectively, and send the multiple retrieval results to the answer correction module; The answer correction module is used to use the multiple search results to correct and update the corresponding initial answer paragraphs to obtain multiple updated answer paragraphs, combine the multiple updated answer paragraphs into an updated answer, and send the updated answer and the preliminary answer to the answer optimization module; The answer optimization module is used to optimize and integrate the updated answer to obtain an optimized answer, and send the optimized answer and the preliminary answer to the user interface module; The user interface module is also used to display the optimized answer and the preliminary answer to the user.

[0008] Optionally, in the thought chain-based retrieval enhancement generation system, the answer correction module includes a query statement generation unit, a paragraph revision unit and a verification and merging unit; The query statement generating unit is used to receive each of the initial answer paragraphs in the thought chain, extract the core questions of each paragraph from each of the initial answer paragraphs, generate multiple query statements according to the multiple core questions of the paragraphs, and send the multiple query statements to the knowledge retrieval module; The paragraph revision unit is used to receive multiple search results sent by the knowledge search module, revise the multiple search results and the multiple initial answer paragraphs respectively to obtain multiple revised answer paragraphs, and send the multiple revised answer paragraphs and the multiple initial answer paragraphs to the verification merging unit; The verification merging unit is used to receive multiple revised answer paragraphs and multiple initial answer paragraphs, compare the differences between the multiple revised answer paragraphs and the multiple initial answer paragraphs to form multiple updated answer paragraphs, merge the multiple updated answer paragraphs to obtain updated answers, and send the updated answers to the answer optimization module.

[0009] Optionally, in the thought chain-based retrieval enhancement generation system, the knowledge retrieval module includes a knowledge source unit and a retrieval unit; The knowledge source unit is used to process the original document in advance to form content storage, or to obtain relevant web pages through real-time networking to form content storage; The retrieval unit is used to receive the plurality of query statements, call a preset retrieval method, and send the plurality of query statements and the preset retrieval method as retrieval information to the knowledge source unit; The knowledge source unit is also used to receive the search information, screen and integrate knowledge from the content storage according to the search information, obtain multiple search results, and send the multiple search results to the search unit; The retrieval unit is also used to receive a plurality of the retrieval results, and integrate the plurality of the retrieval results and send them to the answer correction module; In addition, to achieve the above-mentioned purpose, the present invention also provides a data processing method of a retrieval enhancement generation system based on a thought chain, wherein the data processing method comprises: The user interface module receives a question input by a user and sends the question to the answer generation module, the answer generation module processes the question, generates a preliminary answer, and sends the preliminary answer to the answer segmentation module; The answer segmentation module segments and reorganizes the preliminary answer according to a preset structure to obtain a plurality of initial answer paragraphs, packages the plurality of initial answer paragraphs to obtain a thought chain, and sends the thought chain and the preliminary answer to the answer correction module; The answer correction module generates a query statement for each of the initial answer paragraphs in the thought chain, and sends a plurality of the query statements to the knowledge retrieval module; the knowledge retrieval module generates retrieval results according to the plurality of query statements, and sends the plurality of the retrieval results to the answer correction module; The answer correction module uses the plurality of search results to correct and update the corresponding initial answer paragraphs to obtain a plurality of updated answer paragraphs, combines the plurality of updated answer paragraphs into an updated answer, and sends the updated answer and the preliminary answer to the answer optimization module; The answer optimization module optimizes and integrates the updated answer to obtain an optimized answer, and sends the optimized answer and the preliminary answer to the user interface module, and the user interface module displays the optimized answer and the preliminary answer to the user.

[0010] Optionally, in the data processing method of the thought chain-based retrieval enhancement generation system, the preset structure includes: time structure, place structure, fact structure and character structure.

[0011] Optionally, the data processing method of the retrieval enhancement generation system based on the thought chain, wherein the answer correction module generates a query statement for each of the initial answer paragraphs in the thought chain, and sends the multiple query statements to the knowledge retrieval module, specifically includes: The query statement generating unit receives each of the initial answer paragraphs in the thought chain, and extracts the core question of each paragraph from each of the initial answer paragraphs; According to the core questions of the multiple paragraphs, multiple query statements are generated through a large language model, and the multiple query statements are sent to the knowledge retrieval module.

[0012] Optionally, the data processing method of the thought chain-based retrieval enhancement generation system, wherein the sending of the plurality of query statements to the knowledge retrieval module further comprises: The knowledge source unit processes the original document in advance to form content storage, or obtains relevant web pages through the Internet in real time to form content storage.

[0013] Optionally, the data processing method of the retrieval enhancement generation system based on thought chain, wherein the knowledge retrieval module generates retrieval results according to the multiple query statements respectively, and sends the multiple retrieval results to the answer correction module, specifically includes: The retrieval unit receives the plurality of query statements, calls a preset retrieval method, and sends the plurality of query statements and the preset retrieval method as retrieval information to the knowledge source unit; The knowledge source unit receives the search information, screens and integrates knowledge from the content storage according to the search information, obtains a plurality of search results, and sends the plurality of search results to the search unit; The retrieval unit receives a plurality of the retrieval results, integrates the plurality of the retrieval results and sends them to the answer correction module.

[0014] Optionally, in the data processing method of the thought chain-based retrieval enhancement generation system, the preset retrieval method includes vector retrieval and keyword retrieval.

[0015] Optionally, the data processing method of the retrieval enhancement generation system based on thought chain, wherein the answer optimization module optimizes and integrates the updated answer to obtain an optimized answer, and sends the optimized answer and the preliminary answer to the user interface module, specifically includes: The answer optimization module receives the updated answer, adds a structured title to the updated answer, and obtains an optimized answer; The answer optimization module integrates the optimized answer and the preliminary answer to obtain a final answer, and sends the final answer to the user interface module.

[0016] The present invention discloses a retrieval enhancement generation system and data processing method based on thought chain, the system comprises: a user interface module, an answer generation module, an answer segmentation module, an answer correction module, a knowledge retrieval module and an answer optimization module; the user interface module, the answer generation module, the answer segmentation module, the answer correction module and the answer optimization module sequentially form a closed loop connection, and the answer correction module is also connected to the knowledge retrieval module. The present invention combines thought chain with RAG, uses the information obtained from knowledge retrieval to modify each thought step one by one, increases transparency and accuracy of answers, solves the error accumulation problem of RAG in complex tasks caused by single retrieval and global generation through step-by-step reasoning and dynamic retrieval correction, and is particularly good at complex scenarios that require multi-step verification. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is the overall architecture diagram of the retrieval enhancement generation system based on the thought chain of the present invention; Figure 2 It is a flow chart of a preferred embodiment of the data processing method of the retrieval enhancement generation system based on the thought chain of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0019] To solve the problems in the prior art, this embodiment provides a search enhancement generation system based on thought chain, such as Figure 1 As shown, the thought chain-based retrieval enhancement generation system includes: a user interface module, an answer generation module, an answer segmentation module, an answer correction module, a knowledge retrieval module and an answer optimization module.

[0020] Among them, the user interface module, the answer generation module, the answer segmentation module, the answer correction module and the answer optimization module form a closed loop connection in sequence, and the answer correction module is also connected to the knowledge retrieval module.

[0021] The user interface module is used to receive questions input by users and send the questions to the answer generation module; The answer generation module is used to process the question, generate a preliminary answer, and send the preliminary answer to the answer segmentation module; The answer segmentation module is used to segment and reorganize the preliminary answer according to a preset structure to obtain a plurality of initial answer paragraphs, package the plurality of initial answer paragraphs to obtain a thought chain, and send the thought chain and the preliminary answer to the answer correction module; The answer correction module is used to generate a query statement for each of the initial answer paragraphs in the thought chain, and send a plurality of the query statements to the knowledge retrieval module; The knowledge retrieval module is used to generate retrieval results according to the multiple query statements respectively, and send the multiple retrieval results to the answer correction module; The answer correction module is used to use the multiple search results to correct and update the corresponding initial answer paragraphs to obtain multiple updated answer paragraphs, combine the multiple updated answer paragraphs into an updated answer, and send the updated answer and the preliminary answer to the answer optimization module; The answer optimization module is used to optimize and integrate the updated answer to obtain an optimized answer, and send the optimized answer and the preliminary answer to the user interface module; The user interface module is also used to display the optimized answer and the preliminary answer to the user.

[0022] It is understandable that the model's own knowledge comes entirely from its training data, and the training sets of existing mainstream large language models are basically built on public data on the Internet. Some real-time, non-public or offline data cannot be obtained, so this part of knowledge is impossible to obtain. The underlying principles of all AI models are based on mathematical probability, and their model output is essentially a series of numerical operations. Large language models are no exception, so they sometimes cannot give correct answers, especially in scenarios where the large language model itself does not have certain knowledge or is not good at it. It is difficult to distinguish this kind of hallucination problem because it requires the user to have knowledge in the corresponding field.

[0023] In order to avoid the knowledge deficiencies and hallucinations of the large language model, the retrieval-augmented generation (RAG) method is adopted. The user's questions are searched through the knowledge base of the own knowledge base to obtain relevant information, which is then merged with the original prompt words of the large language model LLM to form a prompt word template, which is then given to the large language model LLM to generate beautiful answers, avoiding the knowledge limitations and hallucinations of the large language model LLM.

[0024] However, the answers generated by this RAG system lack transparency, and users cannot track how the large language model integrates the content of knowledge retrieval. Therefore, the present invention creatively combines thought chaining with RAG to create a method and system for thought chaining RAG, which uses the information obtained from knowledge retrieval to modify each thought step one by one, thereby increasing transparency and accuracy of answers.

[0025] The question-answering model of RAG is: question-knowledge retrieval-large language model LLM-answer. The RAG question-answering model based on the thinking chain of the present invention is: question-large language model LLM preliminary answer-answer paragraph 1-knowledge retrieval-preliminary answer revision 1-answer paragraph 2-knowledge retrieval-preliminary answer revision 2…answer paragraph n-knowledge retrieval-preliminary answer revision n-answer.

[0026] Specifically, the user interface module is used to receive a question input by a user (such as "introduce the history of a company") and send the question to the answer generation module to generate an initial answer. At the same time, the user interface module is also used to receive a final answer from the answer optimization module and output the final answer, wherein the final answer includes the initial answer and the revised optimized answer.

[0027] Furthermore, the answer generation module is used to process the question, generate a preliminary answer, and send the preliminary answer to the answer segmentation module. In this embodiment, the answer generation module can use a large language model LLM (such as DEEPSEEK-R1) to generate a preliminary answer, output the preliminary answer (which may contain errors or hallucinations), and send it to the answer segmentation module.

[0028] Furthermore, the answer segmentation module is used to receive information sent by the answer generation module, split the received information, and reorganize it into initial answer paragraphs organized according to a certain structure. All initial answer paragraphs organized together according to this structure are called thought chains. A large language model LLM can be used as a splitting and reorganization tool, and the time, place, facts, characters, etc. contained in the content of the initial answer can be split and reorganized. The answer segmentation module is also used to send the thought chain and the preliminary answer to the answer correction module.

[0029] Furthermore, the answer correction module includes a query statement generation unit, a paragraph revision unit and a verification merging unit.

[0030] The query statement generating unit is used to receive each of the initial answer paragraphs in the thought chain, extract the core questions of each paragraph from each of the initial answer paragraphs, generate multiple query statements according to the multiple core questions of the paragraphs, and send the multiple query statements to the knowledge retrieval module; The paragraph revision unit is used to receive multiple search results sent by the knowledge search module, revise the multiple search results and the multiple initial answer paragraphs respectively to obtain multiple revised answer paragraphs, and send the multiple revised answer paragraphs and the multiple initial answer paragraphs to the verification merging unit; The verification merging unit is used to receive multiple revised answer paragraphs and multiple initial answer paragraphs, compare the differences between the multiple revised answer paragraphs and the multiple initial answer paragraphs to form multiple updated answer paragraphs, merge the multiple updated answer paragraphs to obtain updated answers, and send the updated answers to the answer optimization module.

[0031] It can be understood that, in this embodiment, the answer correction module performs the following steps one by one for each initial answer paragraph in the received thought chain until all initial answer paragraphs are processed: First, generate accurate query statements for the initial answer paragraph, which can be generated using the large language model LLM; second, send the query statement to the knowledge retrieval module and wait for the return result of the knowledge retrieval module; then, receive the retrieval results of the knowledge retrieval module, correct the initial answer paragraph, form a corrected answer paragraph, and compare it with the initial answer paragraph to generate an updated answer paragraph (the large language model LLM can be used as a tool for correcting answer paragraphs and generating updated answer paragraphs). Finally, all updated answer paragraphs form an updated answer and send it to the answer optimization module.

[0032] Furthermore, the knowledge retrieval module includes a knowledge source unit and a retrieval unit; wherein the knowledge source unit is used to pre-process the original document to form content storage, or to obtain relevant web pages through real-time networking to form content storage; the retrieval unit is used to receive multiple query statements, call a preset retrieval method, and send the multiple query statements and the preset retrieval method as retrieval information to the knowledge source unit.

[0033] The knowledge source unit is also used to receive the retrieval information, screen and integrate knowledge from the content storage according to the retrieval information, obtain multiple retrieval results, and send the multiple retrieval results to the retrieval unit; the retrieval unit is also used to receive multiple retrieval results, and integrate the multiple retrieval results and send them to the answer correction module.

[0034] In this embodiment, first, the knowledge source unit forms content storage by processing the original document in advance, or obtains relevant web page content storage in real time through the Internet. Then, the retrieval unit is used to receive multiple query statements, call a preset retrieval method (vector retrieval, keyword retrieval or a mixed retrieval method), and send the multiple query statements and the preset retrieval method as retrieval information to the knowledge source unit. After the knowledge source unit forms the content storage, it receives the query statement of the answer correction module, and uses the preset retrieval method to filter and integrate the knowledge source content to form a retrieval result.

[0035] Furthermore, the answer optimization module is used to receive information from the answer correction module, optimize and integrate the received updated answers, form optimized answers, and send them to the user interface module. A large model LLM can be used as an optimization and integration tool.

[0036] As can be seen from the above, in the present invention, the RAG based on the thinking chain solves the error accumulation problem caused by single retrieval and global generation in complex tasks of RAG through step-by-step reasoning + dynamic retrieval correction. It is particularly good at complex scenarios that require multi-step verification. Its essence is to combine the knowledge injection of RAG with the process transparency of the thinking chain to form a closed-loop error correction mechanism, which significantly improves the reliability and explainability of the RAG system.

[0037] Based on the thought chain-based retrieval enhancement generation system described in the above embodiment, the present invention also provides a data processing method for the thought chain-based retrieval enhancement generation system, specifically as follows: Figure 2 As shown in , the data processing method of the retrieval enhancement generation system based on thought chain includes the following steps: Step S10: The user interface module receives a question input by a user, and sends the question to the answer generation module. The answer generation module processes the question, generates a preliminary answer, and sends the preliminary answer to the answer segmentation module.

[0038] In this embodiment, the user interface module receives questions input by the user and sends the questions to the answer generation module. The answer generation module uses a large model LLM (such as DEEPSEEK-R1) to generate preliminary answers, outputs preliminary answers (which may contain errors or hallucinations), and sends them to the answer segmentation module.

[0039] Step S20, the answer segmentation module segments and reorganizes the preliminary answer according to a preset structure to obtain multiple initial answer paragraphs, packages the multiple initial answer paragraphs to obtain a thought chain, and sends the thought chain and the preliminary answer to the answer correction module.

[0040] Specifically, the answer segmentation module receives the information sent by the answer generation module, splits the received information, and reorganizes it into initial answer paragraphs of a certain preset structure (the preset structure includes: time structure, location structure, fact structure and character structure). All the initial answer paragraphs organized together by this structure are called thinking chains. The large model LLM can be used as a splitting and reorganization tool, and can be split and reorganized according to the time structure, location structure, fact structure or character structure contained in the content of the initial answer.

[0041] Step S30, the answer correction module generates a query statement for each of the initial answer paragraphs in the thought chain, and sends multiple query statements to the knowledge retrieval module. The knowledge retrieval module generates retrieval results according to the multiple query statements, and sends multiple retrieval results to the answer correction module.

[0042] The answer correction module generates a query statement for each of the initial answer paragraphs in the thought chain, and sends the multiple query statements to the knowledge retrieval module, specifically including: The query statement generating unit receives each of the initial answer paragraphs in the thought chain, and extracts the core question of each paragraph from each of the initial answer paragraphs; According to the core questions of the multiple paragraphs, multiple query statements are generated through a large language model, and the multiple query statements are sent to the knowledge retrieval module.

[0043] In this embodiment, the answer correction module receives information from the answer segmentation module, and performs the following processing on each initial answer paragraph in the received information: Generate a search query statement: extract the core questions of the paragraph. Retrieve knowledge: send the query statement to the knowledge retrieval module, and receive the retrieval results output by the knowledge retrieval module. Revise the paragraph: compare the retrieval results to correct the errors in the initial answer paragraph and add details. Verify and merge: compare the initial answer paragraph and the revised answer paragraph through differences to form the final updated answer paragraph, and keep the revision record. Finally, after each initial answer paragraph is fully revised, it is sent to the answer optimization module.

[0044] Furthermore, the sending of the plurality of query statements to the knowledge retrieval module also includes: The knowledge source unit processes the original document in advance to form content storage, or obtains relevant web pages through the Internet in real time to form content storage.

[0045] In this embodiment, the original document is processed in advance to form content storage, including document preprocessing, content analysis and structuring, data storage, content optimization and enhancement, maintenance and updating. Through these steps, the original document can be effectively processed and stored for subsequent retrieval, analysis and use. Alternatively, the relevant webpage content storage can be obtained through real-time networking.

[0046] Furthermore, the knowledge retrieval module generates retrieval results according to the multiple query statements respectively, and sends the multiple retrieval results to the answer correction module, specifically including: The retrieval unit receives the plurality of query statements, calls a preset retrieval method, and sends the plurality of query statements and the preset retrieval method as retrieval information to the knowledge source unit; the knowledge source unit receives the retrieval information, screens and integrates knowledge from the content storage according to the retrieval information, obtains a plurality of retrieval results, and sends the plurality of retrieval results to the retrieval unit; the retrieval unit receives the plurality of retrieval results, and sends the plurality of retrieval results to the answer correction module after integration; Among them, the preset search methods include vector search and keyword search. Vector search is a search technology based on the vector space model. It achieves efficient search by converting text, images, audio and other data into high-dimensional vectors and calculating the similarity between vectors. It is widely used in natural language processing, computer vision, recommendation systems and other fields, especially when processing unstructured data. Keyword search is a search technology based on specific keywords or phrases in text content. It finds documents or paragraphs containing these keywords by matching the keywords entered by the user with the vocabulary in the document. Keyword search is widely used in search engines, database queries, document management systems and other scenarios.

[0047] Step S40, the answer correction module uses the multiple search results to correct and update the corresponding initial answer paragraphs to obtain multiple updated answer paragraphs, combines the multiple updated answer paragraphs into an updated answer, and sends the updated answer and the preliminary answer to the answer optimization module.

[0048] It can be understood that the answer correction module receives the search results of the knowledge retrieval module, and uses multiple search results to correct the initial answer paragraph to form a corrected answer paragraph, and compares it with the initial answer paragraph to generate an updated answer paragraph. The large model LLM can be used as a tool for correcting answer paragraphs and a tool for generating updated answer paragraphs. Finally, the updated answer and the preliminary answer are sent to the answer optimization module.

[0049] Step S50: the answer optimization module optimizes and integrates the updated answer to obtain an optimized answer, and sends the optimized answer and the preliminary answer to the user interface module, and the user interface module displays the optimized answer and the preliminary answer to the user.

[0050] The answer optimization module optimizes and integrates the updated answer to obtain an optimized answer, and sends the optimized answer and the preliminary answer to the user interface module, specifically including: The answer optimization module receives the updated answer, adds a structured title to the updated answer, and obtains an optimized answer; the answer optimization module integrates the optimized answer and the preliminary answer to obtain a final answer, and sends the final answer to the user interface module.

[0051] In this embodiment, the answer optimization module receives the message from the answer correction module, optimizes the received answer, and outputs it to the user interface module, for example, adding a structured title (such as chapter division in lightweight markup language format), and outputting a final answer (initial answer + multiple iterations) containing a revision history.

[0052] Furthermore, the specific example process of the present invention is as follows: 1. Question entered by the user: "Introduce the history and achievements of a company."

[0053] 2. Generate initial answer: The answer generation module generates an initial answer, which may contain some inaccurate information.

[0054] 3. Answer segmentation: The answer segmentation module segments the initial answer into multiple initial answer paragraphs, for example: Initial answer paragraph 1: The early days of a certain company.

[0055] Initial answer paragraph 2: A company’s existing achievements.

[0056] Initial answer paragraph 3: The development direction of a certain company.

[0057] 4. Paragraph-by-paragraph correction: The answer correction module generates query statements for each initial answer paragraph and performs knowledge retrieval, for example: Query statement 1: "Founder, team and early business of a company".

[0058] Query statement 2: "Business categories of a company and the operating conditions of each business category".

[0059] Query statement 3: "R&D direction and business direction of a company".

[0060] The knowledge retrieval module returns relevant retrieval results for each query statement, and the answer correction module corrects the content of each paragraph based on the retrieval results.

[0061] 5. Optimize answers: The answer optimization module adds titles and subtitles to the corrected answers, for example: Title: The Road to a Certainty—History and Achievements.

[0062] Subtitle 1: Early stage entrepreneurship.

[0063] Subtitle 2: Growth process.

[0064] Subtitle 3: Looking to the future.

[0065] 6. Return answer: The user interface module returns the initial answer and the corrected final answer.

[0066] In summary, the present invention provides a retrieval enhancement generation system and data processing method based on thought chain, the system includes: a user interface module, an answer generation module, an answer segmentation module, an answer correction module, a knowledge retrieval module and an answer optimization module; the user interface module, the answer generation module, the answer segmentation module, the answer correction module and the answer optimization module form a closed loop connection in sequence, and the answer correction module is also connected to the knowledge retrieval module. The present invention combines the thought chain with RAG, uses the information obtained from knowledge retrieval to modify each thinking step one by one, increases transparency and accuracy of the answer, and solves the error accumulation problem caused by single retrieval and global generation in RAG in complex tasks through step-by-step reasoning and dynamic retrieval correction, and is particularly good at complex scenarios that require multi-step verification.

[0067] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or terminal including the element.

[0068] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer-readable storage medium that can be read by a computer, and the program can include the processes of the above-mentioned method embodiments when executed. The computer-readable storage medium can be a memory, a disk, an optical disk, etc.

[0069] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A retrieval enhancement generation system based on thought chain, characterized in that: The thought chain-based retrieval enhancement generation system includes: a user interface module, an answer generation module, an answer segmentation module, an answer correction module, a knowledge retrieval module and an answer optimization module; The user interface module, the answer generation module, the answer segmentation module, the answer correction module and the answer optimization module are sequentially connected to form a closed loop, and the answer correction module is also connected to the knowledge retrieval module; The user interface module is used to receive questions input by users and send the questions to the answer generation module; The answer generation module is used to process the question, generate a preliminary answer, and send the preliminary answer to the answer segmentation module; The answer segmentation module is used to segment and reorganize the preliminary answer according to a preset structure to obtain a plurality of initial answer paragraphs, package the plurality of initial answer paragraphs to obtain a thought chain, and send the thought chain and the preliminary answer to the answer correction module; The answer correction module is used to generate a query statement for each of the initial answer paragraphs in the thought chain, and send a plurality of the query statements to the knowledge retrieval module; The knowledge retrieval module is used to generate retrieval results according to the multiple query statements respectively, and send the multiple retrieval results to the answer correction module; The answer correction module is used to use the multiple search results to correct and update the corresponding initial answer paragraphs to obtain multiple updated answer paragraphs, combine the multiple updated answer paragraphs into an updated answer, and send the updated answer and the preliminary answer to the answer optimization module; The answer optimization module is used to optimize and integrate the updated answer to obtain an optimized answer, and send the optimized answer and the preliminary answer to the user interface module; The user interface module is also used to display the optimized answer and the preliminary answer to the user.

2. The retrieval enhancement generation system based on thought chain according to claim 1 is characterized in that: The answer correction module includes a query statement generation unit, a paragraph revision unit and a verification merging unit; The query statement generating unit is used to receive each of the initial answer paragraphs in the thought chain, extract the core questions of each paragraph from each of the initial answer paragraphs, generate multiple query statements according to the multiple core questions of the paragraphs, and send the multiple query statements to the knowledge retrieval module; The paragraph revision unit is used to receive multiple search results sent by the knowledge search module, revise the multiple search results and the multiple initial answer paragraphs respectively to obtain multiple revised answer paragraphs, and send the multiple revised answer paragraphs and the multiple initial answer paragraphs to the verification merging unit; The verification merging unit is used to receive multiple revised answer paragraphs and multiple initial answer paragraphs, compare the differences between the multiple revised answer paragraphs and the multiple initial answer paragraphs to form multiple updated answer paragraphs, merge the multiple updated answer paragraphs to obtain updated answers, and send the updated answers to the answer optimization module.

3. The retrieval enhancement generation system based on thought chain according to claim 1 is characterized in that: The knowledge retrieval module includes a knowledge source unit and a retrieval unit; The knowledge source unit is used to process the original document in advance to form content storage, or to obtain relevant web pages through real-time networking to form content storage; The retrieval unit is used to receive the plurality of query statements, call a preset retrieval method, and send the plurality of query statements and the preset retrieval method as retrieval information to the knowledge source unit; The knowledge source unit is also used to receive the search information, screen and integrate knowledge from the content storage according to the search information, obtain multiple search results, and send the multiple search results to the search unit; The retrieval unit is also used to receive multiple retrieval results, and integrate the multiple retrieval results and send them to the answer correction module.

4. A data processing method based on the thought chain-based retrieval enhancement generation system according to any one of claims 1 to 3, characterized in that: The data processing method comprises: The user interface module receives a question input by a user and sends the question to the answer generation module, the answer generation module processes the question, generates a preliminary answer, and sends the preliminary answer to the answer segmentation module; The answer segmentation module segments and reorganizes the preliminary answer according to a preset structure to obtain a plurality of initial answer paragraphs, packages the plurality of initial answer paragraphs to obtain a thought chain, and sends the thought chain and the preliminary answer to the answer correction module; The answer correction module generates a query statement for each of the initial answer paragraphs in the thought chain, and sends a plurality of the query statements to the knowledge retrieval module; the knowledge retrieval module generates retrieval results according to the plurality of query statements, and sends the plurality of the retrieval results to the answer correction module; The answer correction module uses the plurality of search results to correct and update the corresponding initial answer paragraphs to obtain a plurality of updated answer paragraphs, combines the plurality of updated answer paragraphs into an updated answer, and sends the updated answer and the preliminary answer to the answer optimization module; The answer optimization module optimizes and integrates the updated answer to obtain an optimized answer, and sends the optimized answer and the preliminary answer to the user interface module, and the user interface module displays the optimized answer and the preliminary answer to the user.

5. The data processing method of the retrieval enhancement generation system based on thought chain according to claim 4 is characterized in that: The preset structure includes: time structure, location structure, fact structure and character structure.

6. The data processing method of the thought chain-based retrieval enhancement generation system according to claim 4 is characterized in that: The answer correction module generates a query statement for each of the initial answer paragraphs in the thought chain, and sends the multiple query statements to the knowledge retrieval module, specifically including: The query statement generating unit receives each of the initial answer paragraphs in the thought chain, and extracts the core question of each paragraph from each of the initial answer paragraphs; According to the core questions of the multiple paragraphs, multiple query statements are generated through a large language model, and the multiple query statements are sent to the knowledge retrieval module.

7. The data processing method of the thought chain-based retrieval enhancement generation system according to claim 4 is characterized in that: The sending of the plurality of query statements to the knowledge retrieval module also includes: The knowledge source unit processes the original document in advance to form content storage, or obtains relevant web pages through the Internet in real time to form content storage.

8. The data processing method of the thought chain-based retrieval enhancement generation system according to claim 7 is characterized in that: The knowledge retrieval module generates retrieval results according to the multiple query statements respectively, and sends the multiple retrieval results to the answer correction module, specifically including: The retrieval unit receives the plurality of query statements, calls a preset retrieval method, and sends the plurality of query statements and the preset retrieval method as retrieval information to the knowledge source unit; The knowledge source unit receives the search information, screens and integrates knowledge from the content storage according to the search information, obtains a plurality of search results, and sends the plurality of search results to the search unit; The retrieval unit receives a plurality of the retrieval results, integrates the plurality of the retrieval results and sends them to the answer correction module.

9. The data processing method of the thought chain-based retrieval enhancement generation system according to claim 8 is characterized in that: The preset search methods include vector search and keyword search.

10. The data processing method of the retrieval enhancement generation system based on thought chain according to claim 4 is characterized in that: The answer optimization module optimizes and integrates the updated answer to obtain an optimized answer, and sends the optimized answer and the preliminary answer to the user interface module, specifically including: The answer optimization module receives the updated answer, adds a structured title to the updated answer, and obtains an optimized answer; The answer optimization module integrates the optimized answer and the preliminary answer to obtain a final answer, and sends the final answer to the user interface module.

Images