A System for Real-time Monitoring and Intervention of Hallucination Content Generated by Large Language Models

Abstract

The present invention provides a system for real-time monitoring and intervention of hallucination content generated by large models, which relates to the field of electrical digital data processing. The system includes a large model docking module, an input parsing module, an output monitoring module, and an intervention and modification module. The large model docking module is used for data docking with the large model system. The input parsing module is used for content parsing of the input information of the large model. The output monitoring module is used for hallucination monitoring of the output content of the large model. The intervention and modification module is used for modifying and adjusting the hallucination content. The system first analyzes the input questions of the large model, determines the types of hallucinations that are likely to occur based on the type of input content, greatly narrowing the analysis scope, and then specifically detects and analyzes the output content, which can effectively improve the accuracy rate of hallucination detection.

Description

Technical Field

[0001] The present invention relates to the field of digital data processing, and in particular to a system for real-time monitoring and intervening in hallucination content generated by large models. Background Art

[0002] With the rapid development and wide application of large language models, their performance in natural language processing, information generation, and interactive tasks has been significantly improved. However, since the content generated by the models is mainly based on statistical language probability distributions, their mastery of factual knowledge has limitations, and there are often phenomena of content fabrication or logical contradictions, which is called "hallucination". Existing research and technologies mainly focus on improving the generation ability and accuracy of the models, such as reducing hallucinations by optimizing pre-training data, introducing knowledge graphs, or constructing more complex network structures. However, these methods cannot completely avoid hallucination content, especially in cases where the user input context is complex or knowledge is sparse. On the other hand, some detection systems can mark hallucination content, but they often stay in the passive detection stage and lack an effective intervention and correction mechanism for hallucination content.

[0003] Many hallucination detection systems have been developed. After a large amount of retrieval and reference, it is found that existing hallucination detection systems are like the system disclosed in CN118333149B. The general methods of these systems include: obtaining the output of a first language model, dividing the output into multiple units with a preset order according to a preset unit, performing named entity recognition on each unit to obtain the category of each unit; using a second language model to predict the probability of generating the unit corresponding to the position at each position of the output; calculating a first hallucination degree value corresponding to the output based on each unit, its category, and its corresponding probability; generating multiple questions based on the output and the category of each unit, using the first language model to answer each question to obtain the answer to each question; calculating a second hallucination degree value corresponding to the output based on the output and the answers to each question; and judging whether the output is a hallucination generated by the first language model according to the first hallucination degree value and the second hallucination degree value. However, this system only performs hallucination detection on the output of the large model, lacks pertinence, and there is room for improvement in the success rate of hallucination detection. Summary of the Invention

[0004] The object of the present invention is to propose a system for real-time monitoring and intervening in hallucination content generated by large models in view of the existing deficiencies.

[0005] The present invention adopts the following technical solutions:

[0006] A system for real-time monitoring and intervening in hallucination content generated by large models, comprising a large model docking module, an input parsing module, an output monitoring module, and an intervention and modification module;

[0007] The large model docking module is used to perform data docking with the large model system. The input parsing module is used to parse the content of the input information of the large model. The output monitoring module is used to monitor hallucinations in the output content of the large model. The intervention and modification module is used to modify and adjust the hallucinated content;

[0008] The large model docking module includes an input detection unit, an output intercepting unit, and an output feedback unit. The input detection unit is used to detect the input events of the large model and obtain the input content. The output intercepting unit is used to intercept the output information of the large model. The output feedback unit is used to feedback the actual output information to the large model;

[0009] The input parsing module includes an information classification unit, a hallucination mapping unit, and a rule output unit. The information classification unit is used to classify and identify the input information. The hallucination mapping unit is used to map the classified information to hallucination types. The rule output unit is used to send the judgment rules of hallucinations to the output monitoring module;

[0010] The output monitoring module includes a content management unit, a rule execution unit, and a result output unit. The content management unit is used to manage the data content that needs to be detected for hallucinations. The rule execution unit is used to judge and analyze the data content based on the judgment rules. The result output unit is used to send the judgment results to the intervention and modification module;

[0011] The intervention and modification module includes a result shunting unit, a hallucination modification unit, and a content output unit. The result shunting unit is used to shunt and process the judgment results. The hallucination modification unit is used to modify and adjust the hallucinated content. The content output unit is used to send the final content information to the output feedback unit.

[0012] Furthermore, the rule output unit includes a hallucination rule library, a rule matching processor, and a rule packaging processor. The hallucination rule library is used to store the rule information for judging hallucinations. The rule matching processor matches the corresponding rule information from the hallucination rule library based on the hallucination type. The rule packaging processor is used to package all the matched rule information and output it;

[0013] The content management unit includes a detection content register, a detection content update processor, and a detection control processor. The detection content register creates a storage area based on the hallucination intervention items to save the output content of the large model. The detection content update processor is used to update the content information in the storage area. The detection control processor organizes the content information in the storage area into an execution data packet for hallucination detection each time an update is made;

[0014] The execution data packet includes environmental data and target data. The environmental data is all the output content of the hallucination intervention project at the current time point, and the target data is the sentence content newly intercepted by the hallucination intervention project.

[0015] Further, the rule execution unit includes an environmental assessment processor, a rule verification processor, and a multi-dimensional scoring processor. The environmental assessment processor is used to evaluate and analyze the environmental data. The rule verification processor verifies and analyzes the target data based on the rule information and the environmental assessment result. The multi-dimensional scoring processor scores whether the target data meets the hallucination condition based on the verification results of multiple rules.

[0016] Further, the process of the rule verification processor verifying and analyzing the target data includes the following steps:

[0017] S1. Obtain the evaluation result of the environmental data and the evaluation result of the target data;

[0018] S2. Obtain a piece of rule information;

[0019] S3. Based on the rule information, screen out the evaluation items and the corresponding evaluation values in the evaluation result;

[0020] S4. Calculate the verification value Vc under this rule information according to the following formula:

[0021] ;

[0022] where n is the number of evaluation items screened out, Am1(i) represents the evaluation value of the i-th evaluation item in the environmental data, Am2(i) represents the evaluation value of the i-th evaluation item in the target data, and Y i represents the judgment threshold for the i-th evaluation item in the rule information;

[0023] S5. Repeat steps S2 to S4 until all the rule information is processed.

[0024] Further, the multi-dimensional scoring processor calculates the hallucination index P according to the following formula:

[0025] ;

[0026] where m is the number of rule information, k i is the weight of the i-th rule information, and Vc(i) is the verification value of the i-th rule information;

[0027] When the hallucination index is greater than the hallucination threshold, it indicates that the target data has the corresponding hallucination type.

[0028] The beneficial effects achieved by the present invention are:

[0029] This system does not directly detect and analyze the output content of the large model. Instead, it first analyzes the input questions of the large model, determines the types of hallucinations that are likely to occur based on the type of input content, greatly narrowing the analysis scope. Then, it specifically detects and analyzes the output content, which can effectively improve the accuracy of hallucination detection. It intervenes and adjusts specifically based on the detected hallucination types to optimize the output content of the large model. At the same time, this system performs real-time hallucination detection during the process of the large model outputting content, replacing the solution of detecting hallucinations in the complete output content in the ordinary system. It can timely discover and modify hallucination content, avoid the impact on analysis when multiple hallucination contents appear simultaneously, and reduce the detection difficulty.

[0030] To enable a further understanding of the features and technical content of the present invention, please refer to the following detailed description and drawings related to the present invention. However, the provided drawings are only for reference and illustration, and are not used to limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a schematic diagram of the overall structural framework of the present invention;

[0032] Figure 2 It is a schematic diagram of the composition of the input parsing module of the present invention;

[0033] Figure 3 It is a schematic diagram of the composition of the output monitoring module of the present invention;

[0034] Figure 4 It is a schematic diagram of the composition of the intervention and modification module of the present invention;

[0035] Figure 5 It is a schematic diagram of the composition of the rule execution unit of the present invention;

[0036] Figure 6 It is a comparison chart of the actual test hallucination quantities of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0037] The following are specific embodiments to illustrate the implementation manners of the present invention. Those skilled in the art can understand the advantages and effects of the present invention from the content disclosed in this specification. The present invention can be implemented or applied through other different specific embodiments, and various details in this specification can also be modified and changed based on different viewpoints and applications without departing from the spirit of the present invention. Additionally, the drawings of the present invention are only simple schematic illustrations and are not drawn according to actual sizes, which is hereby stated in advance. The following embodiments will further detail the related technical content of the present invention, but the disclosed content is not used to limit the protection scope of the present invention.

[0038] Example 1: This example provides a system for real-time monitoring and intervention of hallucination content generated by a large model, in combination with Figure 1, including a large model docking module, an input parsing module, an output monitoring module, and an intervention modification module;

[0039] The large model docking module is used to perform data docking with the large model system. The input parsing module is used to parse the content of the input information of the large model. The output monitoring module is used to monitor hallucinations in the output content of the large model. The intervention modification module is used to modify and adjust the hallucination content;

[0040] The large model docking module includes an input detection unit, an output intercepting unit, and an output feedback unit. The input detection unit is used to detect the input events of the large model and obtain the input content. The output intercepting unit is used to intercept the output information of the large model. The output feedback unit is used to feedback the actual output information to the large model;

[0041] The input parsing module includes an information classification unit, a hallucination mapping unit, and a rule output unit. The information classification unit is used to classify and identify the input information. The hallucination mapping unit is used to map the classified information to the hallucination type. The rule output unit is used to send the judgment rules of hallucinations to the output monitoring module;

[0042] The output monitoring module includes a content management unit, a rule execution unit, and a result output unit. The content management unit is used to manage the data content that needs to be detected for hallucinations. The rule execution unit is used to judge and analyze the data content based on the judgment rules. The result output unit is used to send the judgment result to the intervention modification module;

[0043] The intervention modification module includes a result shunting unit, a hallucination modification unit, and a content output unit. The result shunting unit is used to shunt and process the judgment result. The hallucination modification unit is used to modify and adjust the hallucination content. The content output unit is used to send the final content information to the output feedback unit.

[0044] The rule output unit includes a hallucination rule library, a rule matching processor, and a rule packaging processor. The hallucination rule library is used to store the rule information for judging hallucinations. The rule matching processor is used to match the corresponding rule information from the hallucination rule library based on the hallucination type. The rule packaging processor is used to package all the matched rule information and output it;

[0045] The content management unit includes a detection content register, a detection content update processor, and a detection control processor. The detection content register creates a storage area based on the hallucination intervention item to save the output content of the large model. The detection content update processor is used to update the content information in the storage area. The detection control processor is used to organize the content information in the storage area into an execution data packet for hallucination detection every time an update is made;

[0046] The execution data packet includes environmental data and target data. The environmental data is all the output content of the hallucination intervention project at the current time point, and the target data is the sentence content newly intercepted by the hallucination intervention project.

[0047] The rule execution unit includes an environmental assessment processor, a rule verification processor, and a multi-dimensional scoring processor. The environmental assessment processor is used to evaluate and analyze the environmental data. The rule verification processor verifies and analyzes the target data based on the rule information and the environmental assessment result. The multi-dimensional scoring processor scores whether the target data meets the hallucination condition based on the verification results of multiple rules.

[0048] The process of the rule verification processor verifying and analyzing the target data includes the following steps:

[0049] S1. Obtain the evaluation result of the environmental data and the evaluation result of the target data;

[0050] S2. Obtain a piece of rule information;

[0051] S3. Screen out the evaluation items and the corresponding evaluation values in the evaluation result based on the rule information;

[0052] S4. Calculate the verification value Vc under this rule information according to the following formula:

[0053] ;

[0054] Among them, n is the number of screened evaluation items, Am1(i) represents the evaluation value of the i-th evaluation item in the environmental data, Am2(i) represents the evaluation value of the i-th evaluation item in the target data, and Y i represents the judgment threshold for the i-th evaluation item in the rule information;

[0055] S5. Repeat steps S2 to S4 until all the rule information is processed.

[0056] The multi-dimensional scoring processor calculates the hallucination index P according to the following formula:

[0057] ;

[0058] Among them, m is the number of rule information, k i is the weight of the i-th rule information, and Vc(i) is the verification value of the i-th rule information;

[0059] When the hallucination index is greater than the hallucination threshold, it indicates that there is a corresponding hallucination type in the target data.

[0060] Embodiment 2: This embodiment includes all the content of Embodiment 1 and provides a system for real-time monitoring and intervention of hallucination content generated by a large model, including a large model docking module, an input parsing module, an output monitoring module, and an intervention modification module;

[0061] The large model docking module is used to perform data docking with the large model system. The input parsing module is used to parse the content of the input information of the large model. The output monitoring module is used to monitor hallucinations in the output content of the large model. The intervention modification module is used to modify and adjust the hallucination content;

[0062] The large model docking module includes an input detection unit, an output intercepting unit, and an output feedback unit. The input detection unit is used to detect the input events of the large model and obtain the input content. The output intercepting unit is used to intercept the output information of the large model. The output feedback unit is used to feedback the actual output information to the large model;

[0063] Combined with Figure 2 The input parsing module includes an information classification unit, a hallucination mapping unit, and a rule output unit. The information classification unit is used to classify and identify the input information. The hallucination mapping unit is used to map the classified information to hallucination types. The rule output unit is used to send the judgment rules of hallucinations to the output monitoring module;

[0064] Combined with Figure 3 The output monitoring module includes a content management unit, a rule execution unit, and a result output unit. The content management unit is used to manage the data content that needs to be detected for hallucinations. The rule execution unit is used to perform judgment and analysis on the data content based on the judgment rules. The result output unit is used to send the judgment results to the intervention modification module;

[0065] Combined with Figure 4 The intervention modification module includes a result shunting unit, a hallucination modification unit, and a content output unit. The result shunting unit is used to perform shunting processing on the judgment results. The hallucination modification unit is used to modify and adjust the hallucination content. The content output unit is used to send the final content information to the output feedback unit;

[0066] The input detection unit includes an event response processor, an input acquisition processor, and a project management processor. The event response processor is used to connect to the input port of the large model and respond to input events. The input acquisition processor is used to acquire the input information of the user. The project management processor is used to establish a new hallucination intervention project and manage the project information;

[0067] The output truncation unit includes a sentence pattern judgment processor, an output acquisition processor, and an item attribution processor. The sentence pattern judgment processor is used to connect to the output port of the large model and judge the sentence pattern structure of the output content. The output acquisition processor is used to acquire the complete sentence pattern content. The item attribution processor is used to add corresponding hallucination intervention item tags to the acquired sentence pattern content;

[0068] The output feedback unit includes an output management processor, a port selection processor, and a content feedback processor. The output management processor is used to receive content information and manage it. The port selection processor selects a corresponding output port based on the item tag of the content information. The content feedback processor is used to feedback the content information to the user based on the output port;

[0069] The information classification unit includes a keyword extraction processor, a feature orientation processor, and a classification judgment processor. The keyword extraction processor is used to extract keywords from the input information. The feature orientation processor is used to direct the keywords to the corresponding feature information. The classification judgment processor judges the classification type of the input information based on the feature information;

[0070] The hallucination mapping unit includes a hallucination information library, a mapping relationship library, and a hallucination output processor. The hallucination information library is used to store the type information of hallucinations. The mapping relationship library is used to store the mapping relationship between each content type and multiple hallucination types. The hallucination output processor outputs the mapped hallucination type to the rule output unit based on the content type;

[0071] The rule output unit includes a hallucination rule library, a rule matching processor, and a rule packaging processor. The hallucination rule library is used to store the rule information for judging hallucinations. The rule matching processor matches the corresponding rule information from the hallucination rule library based on the hallucination type. The rule packaging processor is used to package all the matched rule information and output it;

[0072] The content management unit includes a detection content register, a detection content update processor, and a detection control processor. The detection content register creates a storage area based on the hallucination intervention item to save the output content of the large model. The detection content update processor is used to update the content information in the storage area. The detection control processor sorts the content information in the storage area into an execution data packet for hallucination detection each time an update occurs;

[0073] The execution data packet includes environmental data and target data. The environmental data is all the output content of the hallucination intervention item at the current time point. The target data is the newly truncated sentence pattern content of the hallucination intervention item;

[0074] It should be noted that the current target data has not been officially output to the user, so the environmental data does not include the target data;

[0075] When the detection content update processor supplements the newly intercepted sentence content to the storage area, it is regarded as an update operation;

[0076] Combined with Figure 5 , the rule execution unit includes an environment evaluation processor, a rule verification processor, and a multi-dimensional scoring processor. The environment evaluation processor is used to evaluate and analyze the environmental data. The rule verification processor verifies and analyzes the target data based on the rule information and the environment evaluation result. The multi-dimensional scoring processor scores the satisfaction of the target data with the hallucination situation based on the verification results of multiple rules;

[0077] The evaluation result of the environmental data is in the form of an evaluation item and an evaluation value. The environment evaluation processor saves the evaluation result of the environmental data after the previous hallucination detection and evaluates the target data. When the target data has no hallucination, the evaluation result of the target data is merged with the evaluation result of the previous environmental data to obtain a new evaluation result of the environmental data. If the target data has a hallucination, the evaluation result of the target data after intervention and modification is merged with the evaluation result of the previous environmental data to obtain a new evaluation result of the environmental data;

[0078] The process of the rule verification processor verifying and analyzing the target data includes the following steps:

[0079] S1. Obtain the evaluation result of the environmental data and the evaluation result of the target data;

[0080] S2. Obtain a piece of rule information;

[0081] S3. Based on the rule information, screen out the evaluation items and the corresponding evaluation values in the evaluation result;

[0082] S4. Calculate the verification value Vc under this rule information according to the following formula:

[0083] ;

[0084] where n is the number of evaluation items screened out, Am1(i) represents the evaluation value of the i-th evaluation item in the environmental data, Am2(i) represents the evaluation value of the i-th evaluation item in the target data, and Y i represents the judgment threshold for the i-th evaluation item in the rule information;

[0085] S5. Repeat steps S2 to S4 until all the rule information is processed;

[0086] The multi-dimensional scoring processor calculates the hallucination index P according to the following formula:

[0087] ;

[0088] where m is the number of rule information, k i is the weight of the i-th rule information, and Vc(i) is the check value of the i-th rule information;

[0089] When the hallucination index is greater than the hallucination threshold, it indicates that there is a corresponding hallucination type in the target data;

[0090] The result output unit includes a hallucination summary processor and a hallucination packaging processor. The hallucination summary processor is used to summarize the existing hallucination information, and the hallucination packaging processor is used to package the target data and the summarized hallucination information into a target data packet;

[0091] The result shunt unit includes a receiving and identifying processor and a shunt control processor. The receiving and identifying processor is used to receive the packaged target data packet and identify whether there is at least one hallucination, and the shunt control processor performs different processing on the target data packet based on the presence of hallucinations;

[0092] When there is no hallucination in the target data packet, the shunt control processor directly sends the target data in the target data packet to the content output unit. When there is a hallucination in the target data packet, the shunt control processor sends the target data packet to the hallucination modification unit;

[0093] The hallucination modification unit includes a modification policy register, a policy invocation processor, and an editing and modification processor. The modification policy register is used to store the modification policies for each type of hallucination. The policy invocation processor retrieves the corresponding modification policy based on the hallucination type existing in the target data packet, and the editing and modification processor modifies and adjusts the target data based on the modification policy;

[0094] The content output unit includes a target data receiving processor and a response transmission processor. The target data receiving processor is used to receive the target data, and the response transmission processor is used to perform response processing on the target data;

[0095] When the target data receiving processor receives the target data from the result shunt unit, the response transmission processor sends a merge signal to the rule execution unit. When the target data receiving processor receives the target data from the hallucination modification unit, the response transmission processor sends the modified target data to the rule execution unit;

[0096] The response transmission processor also sends the target data to the output feedback unit;

[0097] After receiving the merging signal, the rule execution unit directly merges the evaluation items. After receiving the target data, the rule execution unit first evaluates the target data and then merges it;

[0098] The i appearing in the above text is an ordinal number used to represent the serial number and has no actual meaning.

[0099] Part of the code of this system is as described below:

[0100] class LargeModelInterface:

[0101] """

[0102] Large model docking module: Processes input, output, and feedback.

[0103] """

[0104] def detect_input_event(self, input_content):

[0105] """

[0106] Input detection unit: Detects and obtains the input content.

[0107] """

[0108] print("Detecting input event...")

[0109] self.input_data = input_content

[0110] def capture_output(self, output_content):

[0111] """

[0112] Output capture unit: Captures the output information of the large model.

[0113] """

[0114] print("Capturing output information...")

[0115] self.output_data = output_content

[0116] def feedback_output(self, feedback_content):

[0117] """

[0118] Output feedback unit: Feeds back the modified content to the large model or the user.

[0119] """

[0120] print("Feedback final output content...")

[0121] return feedback_content

[0122] class InputParsingModule:

[0123] """

[0124] Input parsing module: Parse the input information and generate judgment rules.

[0125] """

[0126] def classify_information(self, input_data):

[0127] """

[0128] Information classification unit: Classify and identify the input information.

[0129] """

[0130] print("Classify input information...")

[0131] self.parsed_info = {"type": "query", "content": input_data}

[0132] def map_to_hallucination(self, parsed_info):

[0133] """

[0134] Hallucination mapping unit: Map the classified information to the hallucination type.

[0135] """

[0136] print("Map hallucination type...")

[0137] hallucination_type = "factual_inaccuracy" # Example type

[0138] return hallucination_type

[0139] def generate_rules(self, hallucination_type):

[0140] """

[0141] Rule Output Unit: Generate judgment rules.

[0142] """

[0143] print("Generating judgment rules...")

[0144] self.rules = {"type": hallucination_type, "checks": ["consistency", "accuracy"]}

[0145] return self.rules

[0146] class OutputMonitoringModule:

[0147] """

[0148] Output Monitoring Module: Monitor the content generated by the large model.

[0149] """

[0150] def __init__(self):

[0151] self.analysis_result = None

[0152] def manage_content(self, content):

[0153] """

[0154] Content Management Unit: Manage the data content to be detected.

[0155] """

[0156] print("Managing content data...")

[0157] return content

[0158] def execute_rules(self, content, rules):

[0159] """

[0160] Rule Execution Unit: Judge the content based on the rules.

[0161] """

[0162] print("Executing rules...")

[0163] results = {"content": content, "status": "hallucination_detected"}

[0164] return results

[0165] def output_results(self, analysis_results):

[0166] """

[0167] Result output unit: Pass the analysis results to the next module.

[0168] """

[0169] print("Outputting analysis results...")

[0170] self.analysis_result = analysis_results

[0171] return self.analysis_result

[0172] class InterventionModule:

[0173] """

[0174] Intervention and modification module: Process the detected hallucinated content.

[0175] """

[0176] def __init__(self):

[0177] self.modified_content = None

[0178] def distribute_results(self, results):

[0179] """

[0180] Result shunting unit: Shunt the judgment results.

[0181] """

[0182] print("Shunting process...")

[0183] return results["status"]

[0184] def modify_hallucination(self, content):

[0185] """

[0186] Hallucination modification unit: Modify the hallucination content.

[0187] """

[0188] print("Modifying hallucination content...")

[0189] self.modified_content = content.replace("hallucination_detected", "verified_information")

[0190] return self.modified_content

[0191] def output_final_content(self):

[0192] """

[0193] Content output unit: Output the finally corrected content.

[0194] """

[0195] print("Outputting final content...")

[0196] return self.modified_content

[0197] Now, ask 10 groups of questions to the same large model, and manually detect and count the hallucination situations of the output content when the system is connected and not connected respectively, and obtain Figure 6 the effect comparison diagram shown.

[0198] The content disclosed above is only the preferred feasible embodiment of the present invention, and does not limit the protection scope of the present invention. Therefore, all equivalent technical changes made by using the content of the specification and drawings of the present invention are included in the protection scope of the present invention. In addition, with the development of technology, the elements therein can be updated.

Claims

1. A system for real-time monitoring and intervention of large models to generate hallucination content, characterized in that: It includes large model docking module, input parsing module, output monitoring module and intervention modification module; The large model docking module is used to perform data docking with the large model system, the input parsing module is used to perform content parsing on the input information of the large model, the output monitoring module is used to perform hallucination monitoring on the output content of the large model, and the intervention modification module is used to modify and adjust the hallucination content; The large model docking module includes an input detection unit, an output interception unit and an output feedback unit. The input detection unit is used to detect the input event of the large model and obtain the input content. The output interception unit is used to intercept the output information of the large model. The output feedback unit is used to feed back the actual output information to the large model. The input parsing module includes an information classification unit, an illusion mapping unit and a rule output unit, wherein the information classification unit is used to classify and identify the input information, the illusion mapping unit is used to map the classified information to the illusion type, and the rule output unit is used to send the illusion judgment rule to the output monitoring module; the input information is text information; The output monitoring module includes a content management unit, a rule execution unit and a result output unit, wherein the content management unit is used to manage the data content that needs to be hallucinated, the rule execution unit performs judgment analysis on the data content based on the judgment rule, and the result output unit is used to send the judgment result to the intervention modification module; The intervention modification module includes a result diversion unit, an illusion modification unit and a content output unit. The result diversion unit is used to divert the judgment result, the illusion modification unit is used to modify and adjust the illusion content, and the content output unit is used to send the final content information to the output feedback unit.

2. A system for real-time monitoring and intervention of large models to generate hallucination content as claimed in claim 1, characterized in that: The rule output unit includes an illusion rule base, a rule matching processor and a rule packaging processor, wherein the illusion rule base is used to store rule information for judging illusions, the rule matching processor matches corresponding rule information from the illusion rule base based on the illusion type, and the rule packaging processor is used to package all matched rule information and output them; The content management unit includes a detection content register, a detection content update processor and a detection control processor. The detection content register creates a storage area based on the hallucination intervention project to store the output content of the large model. The detection content update processor is used to update the content information in the storage area. The detection control processor organizes the content information in the storage area into an execution data packet for hallucination detection at each update. The execution data packet includes environment data and target data, the environment data is all output content of the hallucination intervention project at the current time point, and the target data is the sentence content newly intercepted by the hallucination intervention project.

3. A system for real-time monitoring and intervention of large models to generate hallucination content as claimed in claim 2, characterized in that: The rule execution unit includes an environment assessment processor, a rule verification processor and a multidimensional scoring processor. The environment assessment processor is used to evaluate and analyze the environment data. The rule verification processor verifies and analyzes the target data based on the rule information and the environment assessment results. The multidimensional scoring processor scores the target data for satisfying the illusion based on the verification results of multiple rules.

4. A system for real-time monitoring and intervention of large models to generate hallucination content as claimed in claim 3, characterized in that: The process of the rule verification processor performing verification analysis on the target data includes the following steps: S1. Obtaining the evaluation results of environmental data and target data; S2, obtaining a rule information; S3, filtering out evaluation items and corresponding evaluation values ​​in the evaluation results based on the rule information; S4. Calculate the check value Vc under the rule information according to the following formula: ； Where n is the number of selected evaluation items, Am1(i) represents the evaluation value of the i-th evaluation item in the environmental data, Am2(i) represents the evaluation value of the i-th evaluation item in the target data, and Y i represents the judgment threshold for the i-th evaluation item in the rule information; S5. Repeat steps S2 to S4 until all rule information is processed.

5. A system for real-time monitoring and intervention of large models to generate hallucination content as claimed in claim 4, characterized in that: The multidimensional scoring processor calculates the hallucination index P according to the following formula: ； Among them, m is the number of rule information, k i is the weight of the i-th rule information, Vc(i) is the check value of the i-th rule information; When the hallucination index is greater than the hallucination threshold, it indicates that the target data has the corresponding hallucination type.

Images