Method and system for relieving large model illusion

Abstract

The invention relates to a method and system for relieving large model illusion, and belongs to the technical field of computers. The method comprises the following steps: identifying and analyzing the intention of a question input by a user through a first-level agent, and judging whether an answer is obtained by calling a local knowledge base, a search engine or the capability of a large model; if a search engine is called, pulling the question through a first-level agent, and inputting the content of each webpage, website information, the question input by the user and a prompt cue word into a large model to obtain an answer, wherein the previous set number of webpages are related to the content inquired by the search engine; if a local knowledge base is called, analyzing the question through a secondary agent and generating a query instruction corresponding to the question, performing interactive query with professional databases of industries to which at least two questions with different query modes belong through the query instruction to obtain a query result, and inputting the question input by a user, a prompt prompt word and the query result into a large model to obtain an answer; if the capacity of the large model is called, an answer is obtained through the large model according to the question.

Description

Technical Field

[0001] This invention belongs to the field of computer technology, specifically relating to a method and system for alleviating large model illusion. Background Technology

[0002] Currently, general-purpose large models are applied to various fields and tasks. However, when faced with vertical scenarios, they often exhibit the illusion of conflicting factual knowledge, affecting industry applications. Power equipment operation and maintenance encompasses equipment inspection, repair and maintenance, fault handling, performance testing, etc. Condition monitoring and fault diagnosis require ensuring accuracy and professionalism. Generative large models often produce illusions due to a lack of relevant industry knowledge, thus providing incorrect answers.

[0003] The existing approach is to alleviate the illusion of a large model by using a vector knowledge base. Specifically, this involves recalling knowledge fragments related to the question using vector similarity, which the large model then references and answers. However, this approach has three problems: (1) User questions may require cross-document summarization, and the vector knowledge base approach makes it difficult to accurately and completely recall relevant knowledge fragments. (2) The information in the knowledge base and the knowledge of the large model are fixed, making it difficult to obtain the latest information in real time. (3) For some device information and parameter information, the vectorized model has difficulty accurately mapping these numerical information to the semantic space, leading to recall failure. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for alleviating the illusion of large models, in order to solve the problems in the prior art where the use of only vector libraries makes it difficult to obtain the latest information in real time and easily leads to incomplete recall and recall failure.

[0005] To address the aforementioned technical problems, this invention provides a method for mitigating the illusion of large models. The method includes: analyzing the intent of a user-inputted question using a primary agent to determine whether an answer should be obtained by calling a local knowledge base, a search engine, or the capabilities of the large model itself; if a search engine is used, the primary agent retrieves a predetermined number of web pages related to the content retrieved by the search engine, and inputs the content, URL information, user-inputted question, and prompt words of each web page into the large model to obtain an answer; if a local knowledge base is used, a secondary agent analyzes the question and generates a corresponding query instruction, and interactively queries at least two industry-specific databases with different query methods to obtain query results, then inputs the user-inputted question, prompt words, and query results into the large model to obtain an answer; if the capabilities of the large model itself are used, the large model obtains an answer based on the question.

[0006] Furthermore, methods for interactively querying at least two databases include: querying one or all databases simultaneously with a query command and outputting the corresponding query results.

[0007] Furthermore, the method for interactive querying with at least two databases includes: query commands are executed sequentially according to the database order, and the database order is arranged by customization.

[0008] Furthermore, the industry-specific databases for the problem include: a knowledge graph database, an SQL relational database, and a RAG vector database for storing industry-specific data for the problem.

[0009] To address the aforementioned technical problems, the present invention also provides a system for alleviating large model illusion, comprising a processor for executing a computer program to implement the steps of a method for alleviating large model illusion.

[0010] The beneficial effects of the above technical solution are as follows: This invention provides a new method to alleviate the illusion of large models. This invention includes a two-level agent. The first-level agent first performs intent recognition analysis on the user's question and selects the corresponding call capability type. When the selected call capability type is to call a search engine, the first-level agent queries the user's question through the search engine and inputs the query results into the large model. Real-time knowledge information is obtained through the search engine, thus ensuring the timeliness of the large model's answer to the question and solving the problem of poor timeliness in the large model's answer caused by existing methods using vector libraries. When the selected call capability type is to call a local knowledge base, the second-level agent generates corresponding query instructions based on the user's real-time input question and performs interactive queries on professional databases with different query methods. This is equivalent to using professional database assets with different query methods, effectively improving the accuracy of intelligent question answering by the large model. It also solves the problem that existing methods using only vector libraries result in relatively singular knowledge information acquisition and may have difficulty accurately and completely recalling relevant knowledge fragments. Furthermore, the method of interactively querying by calling search engines and various databases can minimize the difficulty in accurately and completely recalling relevant knowledge fragments and the easy failure to recall device information and parameter information, which are common in existing methods using vector libraries. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of the overall process of the method for alleviating large model hallucination according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the interaction process between the first-level agent and the search engine in an embodiment of the method for alleviating large model illusion of the present invention. Figure 3This is a schematic diagram illustrating the process of simultaneous interaction between a secondary agent and multiple databases in an embodiment of the method for alleviating large model illusion according to the present invention. Figure 4 This is a schematic diagram of the secondary agent and chained orchestration of multiple database interaction processes in an embodiment of the method for alleviating large model illusion of the present invention; Figure 5 This is a schematic diagram of the secondary agent and knowledge graph database query process of the method for alleviating large model illusion according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the secondary agent and SQL relational database query process of the method for alleviating large model illusion according to an embodiment of the present invention; Figure 7 This is a schematic diagram of the secondary agent and RAG vector database query process of the method for alleviating large model illusion according to an embodiment of the present invention. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0013] Examples of methods to alleviate large model hallucinations: This embodiment offers a method to alleviate the "big model illusion" by employing a multi-level agent collaborative approach to process user input questions. A first-level agent identifies and analyzes the user's input question, selecting the appropriate calling capability based on the analysis results. This involves determining whether to obtain an answer by calling a local knowledge base, a search engine, or the big model's own capabilities. By using a search engine to acquire real-time knowledge information, the timeliness of the big model's answers is addressed. When the selected calling capability is a local knowledge base, a second-level agent interactively queries specialized databases with different query methods based on the user's real-time input question. This effectively utilizes specialized database assets with varying query methods, significantly improving the accuracy of the big model's intelligent question answering.

[0014] Methods to alleviate large model hallucinations include: The primary agent analyzes the intent of the user's input question to determine whether to obtain an answer by calling a local knowledge base, a search engine, or the capabilities of the large model itself. If a search engine is used, the primary agent retrieves a predetermined number of web pages related to the content retrieved by the search engine. The content and URL information of each web page, along with the user's input question and prompt, are then input into the large model to obtain an answer. If a local knowledge base is used, a secondary agent analyzes the question and generates a corresponding query command. This query command is then used to perform interactive queries with professional databases of the question's industry, using at least two different query methods, to obtain query results. The user's input question, prompt, and query results are then input into the large model to obtain an answer. If the large model's own capabilities are used, the large model obtains an answer based on the question.

[0015] This embodiment includes a two-level agent. The first-level agent first performs intent recognition analysis on the user's question and selects the corresponding call capability type. When the selected call capability type is to call a search engine, the first-level agent queries the user's question through the search engine and inputs the query results into the large model. The search engine obtains real-time knowledge information, thereby ensuring the timeliness of the large model's answer to the question and solving the problem of poor timeliness in the large model's answer to the question caused by the existing method of using vector libraries. When the selected call capability type is to call a local knowledge base, the second-level agent generates corresponding query instructions based on the user's real-time input question and performs interactive queries on professional databases with different query methods. This is equivalent to using professional database assets with different query methods, which effectively improves the accuracy of the large model's intelligent question answering and solves the problem that the existing method of using only vector libraries has relatively limited knowledge information acquisition and may be difficult to accurately and completely recall relevant knowledge fragments. Furthermore, the method of calling search engines and various databases for interactive queries can also avoid the difficulty in accurately and completely recalling relevant knowledge fragments and the easy failure to recall device information and parameter information, which are common problems in the existing method of using vector libraries.

[0016] In this embodiment, for the power operation and maintenance field, the professional database for the industry to which the problem belongs includes: a knowledge graph database, an SQL relational database, and a RAG vector database for storing professional data for the industry to which the problem belongs.

[0017] Specifically, firstly, the user's input question passes through a first-level agent. This agent then analyzes the user's intent to determine whether to invoke a search engine, local knowledge base, or the capabilities of the large model itself. If the local knowledge base is invoked, the question proceeds to a second-level agent. This agent further analyzes the user's input, generates corresponding query instructions, and interacts with knowledge graph databases, SQL relational databases, and RAG vector databases. The retrieved knowledge, the user's question, and prompt words are then input into the large model. This retrieved industry-specific knowledge significantly compensates for the lack of knowledge in specific industries within the general large model, guiding it to accurately answer the user's question. Simultaneously, it outputs the corresponding knowledge sources from the knowledge base, enhancing credibility. For example... Figure 1 As shown.

[0018] If the first-level agent invokes a search engine to obtain online search capabilities, the interaction process between the first-level agent and the search engine is as follows: Figure 2 As shown, by using a custom hyperparameter k, the top k relevant web pages retrieved by the search engine for the user's question are extracted. The user's question and prompt are then input into the large model. The large model will utilize the information from the search engine to obtain real-time information and the URL information referenced from the search engine. This can solve the problem of the timeliness of knowledge in the large model, thereby greatly alleviating the problem of the large model experiencing illusions when faced with real-time information.

[0019] If the user inputs a question related to scenarios that the large model excels at, such as creative writing, copywriting, or coding, the first-level agent will directly invoke the large model's own capabilities to complete the answer according to the user's prompts and requirements.

[0020] In this embodiment, the method for interactively querying at least two databases includes: querying one or all databases simultaneously with a query command and outputting the corresponding query results.

[0021] Specifically, leveraging the intent analysis and decision-making capabilities of the secondary agent, it can select and invoke knowledge graph databases, SQL relational databases, and RAG vector databases based on the user's question type, and complete the knowledge base retrieval by generating corresponding query instructions. For example... Figure 3 As shown.

[0022] In this embodiment, the method for interactively querying at least two databases includes: query commands are performed sequentially according to the database order, and the database order is arranged by customization.

[0023] Specifically, a two-level agent chain-like interaction method is adopted, where the two-level agents query knowledge graph databases, SQL relational databases, and RAG vector databases in a sequential or custom order to complete the retrieval. For example... Figure 4 As shown, the specific design of the interaction method of multiple databases using a two-level agent chain is existing technology and will not be elaborated here.

[0024] The process of a second-level agent calling a knowledge graph database is as follows: Figure 5 As shown, firstly, the secondary agent performs entity recognition and relation extraction on the user's input question, and then queries the knowledge graph database. The relevant knowledge retrieved is passed to a large language model along with the user's original question and prompt words to provide an accurate answer. If the initial query fails to find relevant knowledge, the secondary agent rewrites the user's question using paraphrasing and performs entity recognition and relation extraction again, thus avoiding query failures caused by the user's input question not accurately describing the relevant entities. This calling method is an existing technology that the agent can implement through its inherent functions, and will not be elaborated further here.

[0025] The process of a second-level agent calling an SQL relational database is as follows: Figure 6 As shown, the secondary agent first analyzes and interprets the user's input question, generating a corresponding SQL query to retrieve information from the database. If the query is successful, it passes the retrieved knowledge, along with the original question and prompt words, to a large language model to generate an accurate answer. If no relevant information is found in the database, the query feedback is sent back to the secondary agent. The secondary agent then interprets this feedback and generates a new query to attempt the retrieval again, thus gradually improving the accuracy of the generated query. This method of invocation is based on existing technology that the agent can achieve through its inherent functionality, and will not be elaborated upon here.

[0026] The process of a second-level agent calling the RAG vector database is as follows: Figure 7As shown, firstly, the secondary agent analyzes the user's original question and generates its synonyms. Then, the original question and its synonyms are converted into semantic vector representations using an embedding model. Next, the agent searches a vector database for the k most similar information fragments to these semantic vectors, with similarity exceeding a preset threshold. The found fragments are sorted using a semantic reordering model to ensure they are arranged from highest to lowest relevance. Finally, the sorted fragments, the prompt, and the original question are submitted to a large language model to generate an accurate answer. This method of invocation is based on existing technology that the agent can achieve through its inherent functions, and will not be elaborated upon here.

[0027] System implementation examples for alleviating large model illusion: This invention also provides a system for alleviating large model illusion, including a processor for executing a computer program to implement the steps of a method for alleviating large model illusion. The specific processes, principles, and effects of the system for alleviating large model illusion have been described in detail in the embodiments of the method for alleviating large model illusion, and will not be repeated here.

[0028] Specific implementation methods have been given above, but the present invention is not limited to the described implementation methods. The basic idea of ​​the present invention lies in the above basic scheme. For those skilled in the art, designing various modified models, formulas, and parameters based on the teachings of the present invention does not require creative effort. Changes, modifications, substitutions, and variations made to the implementation methods without departing from the principles and spirit of the present invention still fall within the protection scope of the present invention.

Claims

1. A method for alleviating large model hallucinations, characterized in that, The method includes: analyzing the intent of the user-inputted question through a first-level agent to determine whether to obtain an answer by calling a local knowledge base, a search engine, or the capabilities of the large model itself; if a search engine is called, the first-level agent retrieves a predetermined number of web pages related to the content retrieved by the search engine, and inputs the content, URL information, user-inputted question, and prompt words of each web page into the large model to obtain an answer; if a local knowledge base is called, a second-level agent analyzes the question and generates a query instruction corresponding to the question, and performs interactive queries with at least two professional databases of the industry to which the question belongs through different query methods to obtain query results, and inputs the user-inputted question, prompt words, and query results into the large model to obtain an answer; if the capabilities of the large model itself are called, the large model obtains an answer based on the question.

2. The method for alleviating large model hallucinations according to claim 1, characterized in that, Methods for interactive queries with at least two databases include: querying any one or all databases simultaneously with a query command and outputting the corresponding query results.

3. The method for alleviating large model hallucinations according to claim 1, characterized in that, A method for interactively querying at least two databases includes: query commands are executed sequentially according to the database order, wherein the database order is custom-arranged.

4. The method for alleviating large model hallucinations according to claim 1, characterized in that, The industry-specific databases for the problem include: a knowledge graph database, an SQL relational database, and a RAG vector database, used to store industry-specific data for the problem.

5. A system for alleviating large model illusion, comprising a processor, characterized in that, The processor is used to execute a computer program to implement the steps of the method for alleviating large model illusion as described in any one of claims 1 to 4.

Images