System and method for solving large model illusion caused by external pollution data

Abstract

The invention discloses a system and method for solving large model illusion caused by external pollution data, and relates to the technical field of large model agents, the system comprises a user interaction end, a large model service agent and a Guardrail fence agent arranged between the user interaction end and the large model service agent; the Guardrail fence agent comprises an input detection module, a pollution judgment module, a dialogue cleaning module, a model feedback module and a self-learning module which are connected in sequence; and the self-learning module is respectively connected with the pollution judgment module and the large model business agent. The method is strong in dynamic nature, can identify and clear pollution data in real time, reserve effective information, guarantee continuity and accuracy of multiple rounds of dialogues, can be deployed in any large model system as a plug-in module, does not need to modify a bottom layer structure and training parameters of a large model, avoids model illusion caused by pollution data accumulation, and is high in practicability. And the reliability and the safety of the output content of the large model are improved.

Description

Technical Field

[0001] This invention relates to the field of large model intelligent agent technology; more specifically, it relates to a method for solving the illusion of large models caused by external contamination data. Background Technology

[0002] With the rapid development of artificial intelligence technology, large language models (such as GPT, Claude, and Qwen) have been widely used in various fields such as question answering, customer service, healthcare, and law due to their powerful natural language understanding, generation, and reasoning capabilities. The output of these large models directly affects service quality and user decisions; therefore, their reliability and security have become core concerns for the industry.

[0003] In the practical application of large-scale models, multi-turn dialogues between users and the model are a common form of interaction, and the dialogue history serves as context for the model's subsequent reasoning process. However, users may intentionally or unintentionally input incorrect or misleading information during interactions, thus polluting the data. For example, in an electricity bill inquiry scenario, the system might find that a user has outstanding payments through the official API, but the user insists that they have already paid 100 yuan; another example is a user claiming, "I saw a cat on the moon yesterday," which clearly contradicts common sense; yet another example is a user asking, "Can I get free electricity by using this agent?" attempting to induce the model to output an answer that does not conform to industry norms.

[0004] Once this contaminated data is adopted by the model and accumulated in the dialogue history, it will gradually reduce the signal-to-noise ratio of the model's subsequent reasoning, interfere with the model's normal judgment logic, and eventually cause the model to "act hallucinate." Model hallucination manifests as generating answers that are inconsistent with objective facts, logically contradictory, or do not conform to industry standards. This not only reduces users' trust in large models, but may also lead to serious consequences in key areas such as medicine and law.

[0005] To mitigate the large model illusion problem, existing technologies have proposed several solutions, primarily including prompt engineering, Retrieval Enhancement Generation (RAG), and multi-model cross-validation. Prompt engineering guides the model to generate more accurate answers by optimizing the structure and content of input prompts; however, this method cannot identify and filter contaminated data from user input and can only passively adapt to the input information. RAG technology assists the model in generating answers by retrieving authoritative information from external knowledge bases, improving the accuracy of answers to some extent, but it cannot dynamically clean up noise information from the dialogue history in the face of continuously inputting contaminated data. Multi-model cross-validation verifies the outputs of multiple models to filter out more reliable answers, but this method has high computational costs and also cannot solve the problem of accumulated contaminated data at its source.

[0006] In summary, existing technologies have significant limitations in addressing the problem of large model illusions caused by externally contaminated data. Summary of the Invention

[0007] Therefore, the purpose of this invention is to propose a system and method for solving the illusion of large models caused by external contamination data. By introducing a Guardrail fence agent between the user and the large model, a prevention and control system is constructed, which includes an input detection module, a contamination judgment module, a dialogue cleanup module, a model feedback module, and a self-learning module. This system can identify, isolate, and remove contamination interaction data in real time, ensuring the long-term stability and reliability of the large model in multi-turn dialogues. This solves the technical problem that existing technologies cannot dynamically identify and remove contamination information in multi-turn dialogues, which leads to the large model being prone to illusion.

[0008] This invention provides a system for resolving the illusion of a large model caused by external contamination data, comprising: a user interaction terminal, a large model business intelligence agent, and a Guardrail fence intelligence agent disposed between the user interaction terminal and the large model business intelligence agent; the Guardrail fence intelligence agent includes an input detection module, a contamination determination module, a dialogue cleanup module, a model feedback module, and a self-learning module connected in sequence; the self-learning module is connected to the contamination determination module and the large model business intelligence agent respectively.

[0009] Specifically, contaminated data mainly refers to misleading data that does not conform to the facts, whether intentionally or unintentionally entered by customers. The more times this type of data is entered, the more likely it is to create an illusion for the large model's business intelligence agent, thereby adopting the customer's incorrect data and generating incorrect answers.

[0010] The Guardrail fence agent is an intelligent intermediate layer module proposed in this invention, set between the user and the large model, used to detect, judge and clean up contaminated data in real time, and avoid the large model from creating illusions.

[0011] This system connects the self-learning module to both the contamination detection module and the large-scale business intelligence agent, enabling the Guardrail fence agent to achieve closed-loop adaptive evolution. This not only improves the accuracy of contamination data identification but also allows the output of the large-scale business intelligence agent to provide feedback and enhance fence capabilities, upgrading from "passively intercepting contamination" to "actively adapting to business needs and dynamically optimizing protection." Simultaneously, it achieves bidirectional collaborative optimization between contamination detection and the large-scale model output, specifically manifested as follows:

[0012] (i) The self-learning module obtains judgment data samples (including misjudged contaminated data, missed contaminated data, and accurately judged cases) from the contamination judgment module. Through continuous machine learning / deep learning iterations, it can achieve the following three major improvements:

[0013] Reduce false positives / false negatives: For new types of external contamination data that are not covered by the initial judgment rules (such as maliciously misleading data disguised as normal questions, false professional data specific to the industry, and fragmented low-quality contamination information), the self-learning module can quickly learn its characteristics, update the feature library and algorithm model for contamination judgment, and change the contamination judgment module from fixed rule judgment to dynamic feature judgment, thereby reducing the illusion of a large model caused by inaccurate judgment from the root.

[0014] Adapting to business scenarios: The characteristics of external contamination data vary greatly across different business scenarios (such as false data in the financial scenario and misleading medical information in the medical scenario). The self-learning module can learn the patterns of contamination data in the specific business scenarios served by the system based on the historical data of the judgment module, enabling the judgment module to form a business-customized contamination identification capability, rather than a generalized and crude judgment.

[0015] No frequent manual intervention required: This avoids the drawbacks of traditional protection systems where pollution judgment rules need to be updated and maintained manually one by one. The self-learning module achieves automatic iteration through closed-loop data, reducing the system's operation and maintenance costs while improving the timeliness of rule updates.

[0016] (ii) The self-learning module obtains output data samples from the large model business agent (including hallucinatory outputs caused by failure to intercept contaminated data, output loss caused by excessive interception of normal data, and normal output cases without hallucinations). It can use the output results of the large model as the final feedback basis for contamination judgment, achieving the following effects:

[0017] If the large model business agent experiences hallucinations, the self-learning module can trace back to determine whether the contamination judgment module missed external contamination data or misjudged and allowed low-quality data to pass through. Then, the features and rules of the contamination judgment module can be optimized in a targeted manner to ensure that the standards for contamination judgment are precisely matched with the anti-hallucination capabilities of the large model.

[0018] Traditional protective fences often suffer from the problem of "over-intercepting normal and valid data in order to prevent illusions," causing large models to be unable to answer user questions correctly. The self-learning module of this invention learns the characteristics of normal business data by analyzing normal output cases of large models, enabling the contamination judgment module to accurately distinguish between contaminated data and normal and valid data, thus preventing illusions while ensuring the business responsiveness of large models;

[0019] The large-scale business intelligence agent itself will continuously iterate (e.g., model version updates, business fine-tuning), and its anti-illusion capabilities and data understanding capabilities will change. The self-learning module, by acquiring the output data of the large model in real time, enables the Guardrail fence agent to dynamically adapt to the changes of the large-scale business intelligence agent, avoiding protection failure caused by the fence becoming disconnected from the large model.

[0020] (III) The self-learning module acts as a connecting hub, enabling the input data from the user interface, the Guardrail protection process, and the output results of the large model to form a complete closed-loop data link, achieving an improvement from static protection to dynamic adaptive protection. Initially, the contamination judgment rules of the system are general. However, through continuous iteration of the self-learning module, the Guardrail fence agent gradually develops its own customized protection capabilities that fit its business scenarios and match the characteristics of the large model. Its ability to intercept external contamination data continuously improves over time, fundamentally reducing the probability of the large model experiencing illusions due to external contamination.

[0021] When a large model exhibits hallucinations or abnormal outputs, the self-learning module can clearly pinpoint the root cause of the problem (input contamination, decision failure, or model-specific issues) through closed-loop data, rather than simply blocking or allowing the system to proceed. This facilitates targeted optimization of the system by technical personnel.

[0022] In this invention, the Guardrail fence agent and the large model business agent are no longer two modules that run independently (first the fence intercepts, then the large model outputs). Instead, they achieve data exchange and mutual capability enhancement through a self-learning module. The fence's protective capabilities are optimized and improved, which in turn improves the output quality of the large model. The output feedback of the large model further optimizes the fence's protective capabilities, thereby achieving an overall improvement in the anti-illusion capabilities of the entire system.

[0023] Preferably, the Guardrail fence agent supports both synchronous and asynchronous inspection and cleanup modes, which can be flexibly switched according to the scenario latency requirements.

[0024] In the synchronous inspection and cleanup mode, the Guardrail fence agent completes the detection and judgment of user input before the large model business agent generates an answer. If it is judged to be contaminated data, it is cleaned up immediately to ensure that the large model generates an answer based on a clean context. This mode is suitable for scenarios with low latency requirements.

[0025] In the asynchronous inspection and cleanup mode, the large model business agent first generates an initial response, while the fence agent simultaneously performs detection and judgment. If contaminated data is found, the dialogue history is cleaned up in a timely manner and the model response is updated. This mode is suitable for scenarios with high real-time requirements.

[0026] Furthermore, the input detection module uses semantic analysis, intent recognition, and knowledge graph cross-validation to determine whether the user input contains potentially misleading information. The semantic analysis is based on criteria including self-consistency, knowledge conflict, temporal inconsistency, numerical / unit anomalies, and emotional polarity conflict. The knowledge graph cross-validation compares the user input with known facts in a preset knowledge graph to determine whether there are any contradictions.

[0027] Specifically, the semantic analysis is based on five dimensions:

[0028] First, self-consistency, meaning that there should be no contradictions in the customer's dialogue history;

[0029] Second, knowledge conflict, meaning that customer input should not conflict with existing common sense and local knowledge base;

[0030] Thirdly, there is the issue of time discrepancy, namely, whether there is a discrepancy between the time described by the customer and the actual time of the event.

[0031] Fourth, numerical / unit anomalies, that is, whether the data entered by the customer is obviously too large or too small, exceeding the objective and reasonable range, or whether there is an error in the use of units;

[0032] Fifth, there is the conflict of emotional polarity, that is, whether there are contradictions between positive and negative emotions in the customer's dialogue history.

[0033] Intent recognition is used to accurately determine the true purpose of user input and to identify whether there is a tendency to induce a model.

[0034] Furthermore, the contamination determination module assesses whether the current user input constitutes contaminated data based on preset rules, user historical behavior patterns, and an external knowledge base; the preset rules include at least data source rules, data consistency rules, and logical rationality rules.

[0035] The external knowledge base includes industry standards, laws and regulations, and official authoritative data.

[0036] Specifically, taking the electricity bill consultation scenario as an example, the data source rule requires that the intelligent agent can only use the electricity bill data obtained from the official API as the basis for judgment, and cannot adopt unverified numbers provided by customers; the data consistency rule requires that the electricity bill data entered by the user must be consistent with the data obtained from the API, and if there is a contradiction, it is considered as polluted data; the logical rationality rule requires that the electricity bill data entered by the user must conform to objective logic, such as the electricity bill amount cannot be negative and the electricity consumption cannot exceed a reasonable range.

[0037] User history behavior patterns refer to the records of a user's past interaction data. If a user repeatedly inputs polluting data, the detection level for that user's input content will be increased.

[0038] If the information entered by the customer does not match the content in the external knowledge base, it is determined to be contaminated data.

[0039] Furthermore, when the dialogue cleanup module determines that the user input is polluted data, it will remove or isolate the polluted interaction records of the corresponding round from the dialogue history, while retaining the remaining normal dialogue information.

[0040] After clearing contaminated interaction records, the dialogue cleanup module transmits the cleaned dialogue history to the large model business agent for inference and response generation.

[0041] The advantage of the dialogue cleanup module is that it does not need to clear the entire dialogue history, but only processes the polluted rounds, effectively preserving the contextual information that is valuable to the model's inference and improving the signal-to-noise ratio of the model's output.

[0042] Furthermore, the model feedback module selectively issues warning prompts to the user or asks the user to clarify the input content; the self-learning module feeds back the pollution judgment results to the training or prompt optimization stage of the large model business intelligence agent to improve the robustness of the model.

[0043] For example, when the system determines that the user's input is contaminated data, the model feedback module can send the user a prompt message such as "The content you entered may be misleading, please confirm"; for serious contaminated data or malicious inducement behavior, it can be directly transferred to human processing, where human customer service will verify and communicate.

[0044] The self-learning module records the types, characteristics, and corresponding processing methods of contaminated data, forming a contaminated data sample library. This library is used to optimize the judgment rules of the input detection module and the contaminated judgment module. It can also serve as fine-tuning data for the large model's business intelligence agent, thereby improving the large model's ability to identify contaminated data.

[0045] This invention also provides a method for resolving the large model illusion caused by externally contaminated data, applied to the system described above for resolving the large model illusion caused by externally contaminated data, comprising the following steps:

[0046] S1: Users input content through the user interaction terminal, and the large model business intelligence agent obtains the user input content and historical dialogue records;

[0047] S2: The large model business intelligence agent parses the user input, extracts key information and user intent, integrates the context with the dialogue history and prompts, retrieves relevant information from the external knowledge base as needed, and then calls the large model to perform reasoning to generate a preliminary answer;

[0048] Preferably, the input parsing step requires the large model business intelligence agent to extract key information according to a preset format. For example, in the electricity bill consultation scenario, it extracts information such as the month of electricity bill query by the user and outputs it in JSON format.

[0049] S3: The Guardrail fence agent acquires user input, dialogue history, prompts from the large model business agent, and external knowledge or data. Through the input detection module and pollution judgment module, it judges whether the user input contains factual errors, logical conflicts, or leading expressions.

[0050] S4: If the user input is determined to be contaminated data, the dialogue cleanup module will remove or isolate the contaminated interaction records of the corresponding round from the dialogue history and record the contamination type; if it is determined to be normal input, proceed directly to step S5.

[0051] S5: The cleaned dialogue history is fed into the large model business agent, which performs reasoning based on the clean context to generate the final answer;

[0052] S6: The model feedback module selectively returns warning messages to the user or transfers them to manual processing.

[0053] Furthermore, the logical conflict in step S3 includes: the user input information contradicting known facts or common sense;

[0054] The leading expressions include user input information designed to guide the large model to draw incorrect conclusions.

[0055] Furthermore, the criteria for determining the contaminated data in step S4 include: user input data is inconsistent with data obtained from the official API and cannot be verified by other reliable sources; user input information contradicts known facts or common sense; user input information is intended to guide the large model to draw incorrect conclusions; and user input information does not conform to standards and regulations in external knowledge bases.

[0056] Furthermore, step S6 is followed by:

[0057] The self-learning module feeds back the pollution determination results, pollution type, and corresponding processing methods to the training or optimization module of the large model, thereby improving the model's ability to identify pollution data. Through continuous self-learning, the system can continuously improve the detection accuracy and processing efficiency of pollution data.

[0058] This invention utilizes semantic analysis, intent recognition, and cross-validation with knowledge graphs, combined with preset rules, historical behavior patterns, and external knowledge bases, to detect in real time whether user input is contaminated data. For data determined to be contaminated, it accurately removes it from the dialogue history and records its type, retaining valid dialogue information, and can also issue warnings to the user. Furthermore, it can feed the contamination determination results back to the model training stage to continuously improve robustness.

[0059] The present invention also provides a computer device, the computer device including a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the program, it implements the steps of the method for resolving the large model illusion caused by external contamination data as described above.

[0060] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0061] This invention provides a system and method for resolving the illusion of large models caused by external contamination data. Through the Guardrail fence agent, it performs real-time detection and judgment of each round of user input. Once contamination data is detected, it is immediately removed from the dialogue history. This dynamic approach allows for real-time identification and removal of contamination data, preventing its accumulation from affecting subsequent model inference and effectively solving the problem of existing technologies' inability to dynamically clean up contamination information. The dialogue cleanup module only clears the interaction records of contamination rounds, without clearing the entire dialogue history, maximizing the preservation of valuable information for model inference, improving the signal-to-noise ratio of the model output, and ensuring the coherence and accuracy of multi-round dialogues. As an independent intermediate layer module, the Guardrail fence agent can be deployed in any large model system without modifying the underlying structure and training parameters of the large model. It possesses flexible, plug-in-like deployment characteristics and is suitable for GPT, Claude, and Qwen. It has broad applicability across various large language models and multiple application scenarios in fields such as healthcare, law, and customer service. The self-learning module can feed back the judgment results and processing methods of contaminated data to the system optimization stage, continuously optimizing the detection rules and judgment weights, improving the system's accuracy in identifying contaminated data, achieving continuous iterative upgrades of system performance, and continuously improving system robustness. Through the early warning mechanism, it promptly provides feedback to users on issues with the input content, allowing users to clearly understand the rationality of the input information, while ensuring the reliability of the large model's output content, effectively increasing users' trust in the large model system. Especially in critical fields such as healthcare and law, it can avoid serious consequences caused by model illusions, enhance user trust, and improve service security. Attached Figure Description

[0062] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention.

[0063] In the attached diagram:

[0064] Figure 1 This is a flowchart illustrating the data cleanup process of the Guardrail fence agent in an embodiment of the present invention.

[0065] Figure 2 This is a flowchart illustrating the method for resolving the large model illusion caused by external contamination data, according to an embodiment of the present invention.

[0066] Figure 3 This is a schematic diagram of the configuration of a computer device according to an embodiment of the present invention. Detailed Implementation

[0067] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and products consistent with some aspects of this disclosure as detailed in the appended claims.

[0068] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0069] It should be understood that although the terms first, second, third, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0070] The embodiments of the present invention will be described in further detail below.

[0071] This invention provides a system for resolving the illusion of a large model caused by external contamination data, comprising: a user interaction terminal, a large model business intelligence agent, and a Guardrail fence intelligence agent set between the user interaction terminal and the large model business intelligence agent; the Guardrail fence intelligence agent includes an input detection module, a contamination determination module, a dialogue cleanup module, a model feedback module, and a self-learning module connected in sequence; the self-learning module is connected to both the contamination determination module and the large model business intelligence agent.

[0072] The Guardrail fence agent supports two working modes: synchronous inspection and cleanup, and asynchronous inspection and cleanup. In synchronous inspection and cleanup mode, the Guardrail fence agent detects and judges the user input before the large model business agent generates a response. If the data is judged to be contaminated, it is cleaned up immediately to ensure that the large model generates a response based on a clean context. This mode is suitable for scenarios with low latency requirements.

[0073] In the asynchronous inspection and cleanup mode, the large model business agent first generates an initial response, while the fence agent simultaneously performs detection and judgment. If contaminated data is found, the dialogue history is cleaned up in a timely manner and the model response is updated. This mode is suitable for scenarios with high real-time requirements.

[0074] As shown in Figure 1, after the user inputs content, the system first retrieves the dialogue history, and then selects either a synchronous or asynchronous fence agent inspection mode based on the scenario's latency requirements. In synchronous inspection mode, the fence agent first determines whether the user input is contaminated data or misleading. If determined to be yes, it cleans up the contaminated data, updates the dialogue history, and sends a warning risk alert to the customer's page, or transfers the issue to manual processing. If determined to be no, the large-scale business agent replies to the customer normally. In asynchronous inspection mode, the large-scale business agent first generates an answer and replies normally to the customer, while the fence agent simultaneously performs inspection. If contaminated data is found, it cleans up the contaminated data, updates the dialogue history, and adds a flag to the customer's page as a reminder.

[0075] The input detection module uses semantic analysis, intent recognition, and knowledge graph cross-validation to determine whether user input contains potentially misleading information. Semantic analysis is based on criteria including self-consistency, knowledge conflict, temporal inconsistency, numerical / unit anomalies, and sentiment polarity conflict. Intent recognition accurately determines the true purpose of the user's input and identifies any tendency to induce a specific model. Knowledge graph cross-validation compares the user input with known facts in a pre-defined knowledge graph to determine if there are any contradictions.

[0076] The criteria for semantic analysis include five dimensions: first, self-consistency, meaning that there should be no inconsistencies in the customer's dialogue history; second, knowledge conflict, meaning that the customer's input should not conflict with existing common sense and local knowledge; third, temporal contradiction, meaning that there is a contradiction between the time described by the customer and the actual time of the event; fourth, numerical / unit anomalies, meaning that the data entered by the customer is significantly too large or too small, exceeding the objective and reasonable range, or there is an error in the use of units; and fifth, emotional polarity conflict, meaning that there are contradictions between positive and negative emotions in the customer's dialogue history.

[0077] The contamination determination module assesses whether the current user input constitutes contaminated data based on preset rules, user historical behavior patterns, and an external knowledge base. The preset rules include at least data source rules, data consistency rules, and logical rationality rules. In the electricity bill consultation scenario, the data source rule requires the agent to use only electricity bill data obtained from the official API as the basis for judgment and cannot accept unverified figures provided by the customer. The data consistency rule requires that the user-inputted electricity bill data must be consistent with the data obtained from the API; if there is a contradiction, it is considered contaminated data. The logical rationality rule requires that the user-inputted electricity bill data must conform to objective logic, such as the electricity bill amount cannot be negative and the electricity consumption cannot exceed a reasonable range. User historical behavior patterns record the user's past interaction data; if a user repeatedly inputs contaminated data, the detection level for that user's input is increased. The external knowledge base includes industry standards, laws and regulations, and official authoritative data. If the information input by the customer does not match the content in the external knowledge base, it is determined to be contaminated data.

[0078] When the dialogue cleanup module determines that user input is contaminated data, it removes or isolates the contaminated interaction records for the corresponding round from the dialogue history, while retaining the remaining normal dialogue information. After removing the contaminated interaction records, the cleaned dialogue history is passed to the large model business agent for inference and response generation. The dialogue cleanup module does not need to clear the entire dialogue history; it only processes contaminated rounds, effectively preserving contextual information valuable for model inference and improving the signal-to-noise ratio of the model output.

[0079] The model feedback module selectively issues warnings to users or requests clarification of their input. The self-learning module feeds back the contamination determination results to the training or optimization stages of the large model business agent, improving model robustness. When the system determines that user input is contaminated data, the model feedback module sends a message to the user stating, "Your input may be misleading; please confirm." For severely contaminated data or malicious misleading behavior, the case is directly transferred to human intervention for verification and communication by customer service.

[0080] The self-learning module records the types, characteristics, and corresponding processing methods of contaminated data, forming a contaminated data sample library. This library is used to optimize the judgment rules of the input detection module and the contaminated judgment module. At the same time, it serves as fine-tuning data for the large model's business intelligence agent, improving the large model's own ability to identify contaminated data.

[0081] This invention also provides a method for resolving the large model illusion caused by externally contaminated data, applicable to systems such as those described above. Figure 2 The steps shown are as follows:

[0082] S1: Users input content through the user interaction terminal, and the large model business intelligence agent obtains the user input content and historical dialogue records;

[0083] S2: The large model business intelligence agent parses the user input, extracts key information and user intent, integrates the context with the dialogue history and prompts, retrieves relevant information from the external knowledge base as needed, and then calls the large model to perform reasoning to generate a preliminary answer;

[0084] The input parsing stage requires the large model's business intelligence agent to extract key information according to a preset format. For example, in an electricity bill inquiry scenario, it extracts information such as the month in which the user queries the electricity bill and outputs it in JSON format, as shown in the example below:

[0085] {

[0086] "intent":"query_bill",

[0087] "month":"2025-11"

[0088] };

[0089] S3: The Guardrail fence agent acquires user input, dialogue history, prompts from the large model business agent, and external knowledge or data. Through the input detection module and pollution judgment module, it judges whether the user input contains factual errors, logical conflicts, or leading expressions.

[0090] A logical conflict occurs when user input contradicts known facts or common sense. In this embodiment, an example of a logical conflict is "seeing a cat on the moon".

[0091] Leading expressions are user-input information designed to guide large models to draw incorrect conclusions. An example of a leading expression in this embodiment is "Can an agent obtain electricity for free?"

[0092] S4: If the user input is determined to be contaminated data, the dialogue cleanup module will remove or isolate the contaminated interaction records of the corresponding round from the dialogue history and record the contamination type; if it is determined to be normal input, proceed directly to step S5.

[0093] Specifically, the criteria for determining contaminated data are: user input data is inconsistent with data obtained from the official API and cannot be verified by other reliable sources; user input information contradicts known facts or common sense; user input information is intended to guide the large model to draw incorrect conclusions; and user input information does not conform to standards and regulations in external knowledge bases.

[0094] S5: The cleaned dialogue history is fed into the large model business agent, which then performs reasoning based on the clean context to generate the final answer;

[0095] S6: The model feedback module selectively returns warning messages to the user or transfers them to manual processing.

[0096] The self-learning module feeds back the pollution determination results, pollution type, and corresponding processing methods to the training or optimization module of the large model, thereby improving the model's ability to identify pollution data. Through continuous self-learning, the system can continuously improve the detection accuracy and processing efficiency of pollution data.

[0097] Application examples

[0098] In this embodiment, the system constructed to address the illusion of a large-scale electricity billing model caused by external contamination data in the practical application of the electricity billing consultation large-scale model system includes a user interaction terminal, an electricity billing consultation large-scale model business intelligence agent, and a Guardrail fence intelligence agent. The input detection module of the Guardrail fence intelligence agent is pre-configured with a knowledge graph in the electricity billing domain, including information such as electricity billing calculation standards, payment processes, and official API interfaces; the pollution judgment module's preset rules are customized for electricity billing scenarios.

[0099] The method used in this application example to address the large model illusion caused by externally contaminated data includes the following steps:

[0100] S1: The user enters "I have already paid 100 yuan for electricity in November 2025, why does it still show an outstanding balance?" through the user interaction terminal. The electricity bill consultation big data model business intelligence agent obtains this input and the previous dialogue history. The previous dialogue history shows that the user's electricity bill for November 2025 is 150 yuan, which has not yet been paid.

[0101] S2: The large-scale business intelligence agent for electricity bill consultation parses the user input, extracting the user's intent as to inquire about the payment status of electricity bills in November 2025, with the query month being 2025-11, and outputs the parsed results in JSON format; then, it integrates the context by combining the dialogue history and prompts, and calls the official electricity bill API to perform a search. The search results show that the user's electricity bill for November 2025 is unpaid, with an outstanding amount of 150 yuan; based on this information, the large-scale business intelligence agent generates a preliminary answer: "According to official data, your electricity bill for November 2025 has not been paid, and the outstanding amount is 150 yuan."

[0102] S3: The Guardrail fence agent acquires user input, dialogue history, prompts from the large-scale business agent, and official API retrieval results. It then performs semantic analysis and knowledge graph cross-validation through the input detection module. Semantic analysis reveals a self-consistency contradiction between the user's current input "paid 100 yuan" and the historical dialogue "owes 150 yuan." Knowledge graph cross-validation shows a discrepancy between the user's input payment amount and the official API data, indicating a knowledge conflict. Subsequently, the contamination determination module evaluates the data according to preset rules. The data source rules require official API data as the standard; since the user's input payment data contradicts the API data, and the user did not provide any payment receipts, the user's input is determined to be contaminated data.

[0103] S4: The dialogue cleanup module will remove the current user input of "I have paid 100 yuan for electricity in November 2025, why does it still show an arrears" from the dialogue history and record the type of the polluted data as "data consistency conflict".

[0104] S5: The cleaned dialogue history is fed into the business intelligence agent of the electricity bill consultation big model. The big model performs reasoning based on the clean context (only retaining the record of the user's previous query of electricity bill amount) and generates the final answer: "Hello, according to the query, your electricity bill for November 2025 is 150 yuan, which has not yet been paid. It is recommended that you pay the bill in time to avoid affecting your normal electricity use."

[0105] S6: The model feedback module sends a warning message to the user: "The payment information you provided is inconsistent with the official data. Please verify and query again."

[0106] S7: The self-learning module feeds back the type of contaminated data, "data consistency conflict", the judgment basis, "contradiction between user input data and official API data", and the handling method, "clearing the corresponding round of dialogue records + sending an early warning prompt", to the system's rule optimization module to optimize the judgment weight of the contamination judgment module and improve the detection accuracy of similar data.

[0107] In another application scenario, a user inputs, "I saw a cat on the moon yesterday, is that true?" The initial answer from the large model's business intelligence agent might be misleading. At this point, the Guardrail fence agent's input detection module, through semantic analysis, discovers a clear knowledge conflict in the user's input, violating the common sense that "humans cannot survive on the moon." The contamination judgment module then classifies the input as contaminated data; the dialogue cleanup module clears the dialogue record for that round; the large model generates an answer based on the cleaned context: "According to common sense, the moon has no atmosphere and liquid water, and therefore lacks the conditions for life, so it's impossible to see a cat." The model feedback module sends a prompt to the user: "Your description does not conform to common sense; please confirm your input." The self-learning module records this contaminated data type as "knowledge conflict type" to optimize subsequent detection rules.

[0108] This invention also provides a computer device. Figure 3 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention; see the accompanying drawings. Figure 3 As shown, the computer device includes: an input device 23, an output device 24, a memory 22, and a processor 21; the memory 22 is used to store one or more programs; when the one or more programs are executed by the one or more processors 21, the one or more processors 21 implement the method for solving the large model illusion caused by external contamination data as provided in the above embodiments; wherein the input device 23, the output device 24, the memory 22, and the processor 21 can be connected via a bus or other means. Figure 3 Taking the example of a connection between China and Israel via a bus.

[0109] The memory 22, as a read / write storage medium for a computing device, can be used to store software programs and computer-executable programs, such as the program instructions corresponding to the method for resolving the large model illusion caused by external contamination data as described in this embodiment of the invention. The memory 22 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created based on the use of the device, etc. Furthermore, the memory 22 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 22 may further include memory remotely located relative to the processor 21, and these remote memories can be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0110] Input device 23 can be used to receive input digital or character information, and generate key signal inputs related to user settings and function control of the device; output device 24 may include display devices such as a display screen.

[0111] The processor 21 executes various functional applications and data processing of the device by running software programs, instructions and modules stored in the memory 22, thereby realizing the above-mentioned method for solving the large model illusion caused by external contamination data.

[0112] The computer equipment provided above can be used to execute the method for solving the large model illusion caused by external pollution data provided in the above embodiments, and has corresponding functions and beneficial effects.

[0113] This invention also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the method for resolving the large model illusion caused by externally contaminated data as provided in the above embodiments. The storage medium is any type of memory device or storage device, including: mounting media such as CD-ROM, floppy disk, or magnetic tape; computer system memory or random access memory such as DRAM, DDRRAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory such as flash memory, magnetic media (e.g., hard disk or optical storage); registers or other similar types of memory elements; the storage medium may also include other types of memory or combinations thereof; furthermore, the storage medium may reside in a first computer system in which the program is executed, or it may reside in a different second computer system connected to the first computer system via a network (such as the Internet); the second computer system can provide program instructions to the first computer for execution. The storage medium includes two or more storage media that may reside in different locations (e.g., in different computer systems connected via a network). The storage medium may store program instructions (e.g., specifically implemented as a computer program) executable by one or more processors.

[0114] Of course, the computer-executable instructions provided in the embodiments of the present invention are not limited to the method for solving the large model illusion caused by external contamination data as described in the above embodiments, but can also perform related operations in the method for solving the large model illusion caused by external contamination data provided in any embodiment of the present invention.

[0115] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

[0116] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A system for resolving the illusion of large models caused by externally contaminated data, characterized in that, include: The user interaction terminal, the large model business intelligence agent, and the Guardrail fence intelligence agent set between the user interaction terminal and the large model business intelligence agent. The Guardrail fence agent includes an input detection module, a contamination determination module, a dialogue cleanup module, a model feedback module, and a self-learning module connected in sequence; the self-learning module is connected to the contamination determination module and the large model business agent.

2. The system for resolving the large model illusion caused by externally contaminated data according to claim 1, characterized in that, The input detection module uses semantic analysis, intent recognition, and knowledge graph cross-validation to determine whether the user input contains potentially misleading information. The semantic analysis criteria include self-consistency, knowledge conflict, temporal inconsistency, numerical / unit anomalies, and emotional polarity conflict. The knowledge graph cross-validation compares user input information with known facts in a preset knowledge graph to determine whether there are any contradictions.

3. The system for resolving the large model illusion caused by externally contaminated data according to claim 1, characterized in that, The contamination determination module assesses whether the current user input constitutes contaminated data based on preset rules, user historical behavior patterns, and an external knowledge base; the preset rules include at least data source rules, data consistency rules, and logical rationality rules. The external knowledge base includes industry standards, laws and regulations, and official authoritative data.

4. The system for resolving the large model illusion caused by externally contaminated data according to claim 1, characterized in that, When the dialogue cleanup module determines that the user input is polluted data, it will remove or isolate the polluted interaction records of the corresponding round from the dialogue history, while retaining the remaining normal dialogue information. After clearing contaminated interaction records, the dialogue cleanup module transmits the cleaned dialogue history to the large model business agent for inference and response generation.

5. The system for resolving the large model illusion caused by externally contaminated data according to claim 1, characterized in that, The model feedback module selectively issues warning prompts to the user or asks the user to clarify the input content; The self-learning module feeds back the pollution determination results to the training or optimization stages of the large model business agent, thereby improving the robustness of the model.

6. A method for resolving the large model illusion caused by externally contaminated data, applied to the system for resolving the large model illusion caused by externally contaminated data as described in any one of claims 1-5, characterized in that, Includes the following steps: S1: Users input content through the user interaction terminal, and the large model business intelligence agent obtains the user input content and historical dialogue records; S2: The large model business intelligence agent parses the user input, extracts key information and user intent, integrates the context with the dialogue history and prompts, retrieves relevant information from the external knowledge base as needed, and then calls the large model to perform reasoning to generate a preliminary answer; S3: The Guardrail fence agent acquires user input, dialogue history, prompts from the large model business agent, and external knowledge or data. Through the input detection module and pollution judgment module, it judges whether the user input contains factual errors, logical conflicts, or leading expressions. S4: If the user input is determined to be contaminated data, the dialogue cleanup module will remove or isolate the contaminated interaction records of the corresponding round from the dialogue history and record the contamination type; if it is determined to be normal input, proceed directly to step S5. S5: The cleaned dialogue history is fed into the large model business agent, which performs reasoning based on the clean context to generate the final answer; S6: The model feedback module selectively returns warning messages to the user or transfers them to manual processing.

7. The method for resolving the large model illusion caused by external contamination data according to claim 6, characterized in that, The logical conflict in step S3 includes: the user input information contradicts known facts or common sense; the leading expression includes: the user input information is intended to guide the large model to draw incorrect conclusions.

8. The method for resolving the large model illusion caused by external contamination data according to claim 6, characterized in that, The criteria for determining contaminated data in step S4 include: user input data is inconsistent with data obtained from the official API and cannot be verified by other reliable sources; user input information contradicts known facts or common sense; user input information is intended to guide the large model to draw incorrect conclusions; and user input information does not conform to standards and regulations in external knowledge bases.

9. The method for resolving the large model illusion caused by external contamination data according to claim 6, characterized in that, The process after step S6 also includes: The self-learning module feeds back the pollution determination results, pollution type and corresponding processing method to the training or optimization module of the large model to improve the model's ability to identify pollution data.

10. A computer device, the computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method for resolving the large model illusion caused by external contamination data as described in any one of claims 6-9.

Images