Large model illusion governance system combining user feedback

Abstract

The invention provides a system for carrying out large model illusion treatment by combining user feedback, which is characterized by comprising a feedback collection module, an illusion detection module, a generation optimization module and a real-time correction module, the illusion detection module is used for carrying out illusion detection on the generated content of the large model, the generation optimization module is used for carrying out optimization processing on the content generation process of the large model, and the real-time correction module is used for carrying out error correction rectification on the generated content of the large model; according to the system, feedback information is sorted and analyzed, optimization is carried out from three parts of a generation end, a detection end and a modification end, and the illusion problem can be treated more effectively.

Description

Technical Field

[0001] The present invention relates to the field of electronic digital data processing, and in particular to a large model hallucination management system combining user feedback. Background Art

[0002] The "hallucination" problem existing in the content generated by large models is also becoming increasingly prominent. The hallucination problem usually manifests itself as the model generating seemingly reasonable but actually wrong content without sufficient knowledge or logical reasoning ability. Traditional model optimization methods, such as those based on generative adversarial networks and reinforcement learning, can improve the overall quality of generated content, but they still have deficiencies in accuracy, logical consistency and the application of domain-specific knowledge. Especially when faced with highly complex content and detailed logical reasoning scenarios, these optimization methods are difficult to completely avoid the occurrence of hallucinations. In order to meet the above challenges, there is an urgent need for a comprehensive management system that can combine user feedback, detect hallucination content in real time, dynamically optimize the generation process and accurately correct errors.

[0003] The foregoing discussion of the background art is intended only to facilitate an understanding of the present invention. This discussion does not acknowledge or admit that any of the material referred to is part of the common general knowledge.

[0004] Now many large model governance systems have been developed. After a lot of searching and reference, it is found that the existing large model governance systems include the system disclosed in the publication number CN117556920B. These system methods generally include: obtaining a target question and answer pair, wherein the target question and answer pair includes: a target question and an answer to be verified; inputting the target question and answer pair into the target model to obtain the question and answer type corresponding to the target question and answer pair, wherein the target model is obtained by iteratively training the model to be trained through the training sample set; determining the verification rules according to the question and answer type, and verifying the answer to be verified based on the verification rules and the target question and answer pair. If the verification passes, the answer to be verified is determined to be the target answer. However, this system can only improve the illusion of the answer to the target question, and is not suitable for the conventional application answers of large models, and the scope of illusion governance is limited. Summary of the invention

[0005] The purpose of the present invention is to propose a large model hallucination management system in combination with user feedback to address the existing deficiencies.

[0006] The present invention adopts the following technical solution:

[0007] A large model hallucination management system based on user feedback, including a feedback collection module, a hallucination detection module, a generation optimization module and a real-time correction module;

[0008] The feedback collection module is used to collect user feedback information on generated content, the hallucination detection module is used to perform hallucination detection on the generated content of the large model, the generation optimization module is used to optimize the content generation process of the large model, and the real-time correction module is used to correct errors and rectify the content generated by the large model;

[0009] The feedback collection module includes a feedback interface management unit, a feedback data management unit and a feedback output processing unit. The feedback interface management unit is used to receive user feedback information, the feedback data management unit is used to organize and analyze the feedback information, and the feedback output processing unit outputs adjustment information based on the analysis result of the feedback information;

[0010] The hallucination detection module includes a content verification unit, a logic verification unit and a detection optimization unit. The content verification unit is used to compare and analyze the generated content to verify the correctness of the content. The logic verification unit verifies the correctness of the content based on the logical relationship between the generated content. The detection optimization unit optimizes the verification parameters based on the adjustment information.

[0011] The generation optimization module includes a feedback-driven fine-tuning unit, an optimization monitoring unit and an optimization control unit. The feedback-driven fine-tuning unit makes a slight adjustment to the generation parameters in the large model based on the adjustment information. The optimization monitoring unit is used to monitor the optimization state of the large model. The optimization control unit controls the optimization process based on the monitoring result.

[0012] The real-time correction module includes a generation information cache unit, a correction content locking unit and a correction content replacement unit. The generation information cache unit is used to receive the generated content of the large model, the correction content locking unit adjusts information and detection information to determine the content that needs to be corrected, and the correction content replacement unit performs replacement processing based on the locked content.

[0013] Furthermore, the feedback output processing unit includes a detection output processor, a generation output processor and a correction output processor. The detection output processor is used to output detection adjustment information to the hallucination detection module, the generation output processor is used to output generation adjustment information to the generation optimization module, and the correction output processor is used to output correction adjustment information to the real-time correction module.

[0014] Further, the optimization monitoring unit includes a content test processor and a change evaluation processor, wherein the content test processor is used to perform content generation test processing on the adjusted large model, and the change evaluation processor is used to evaluate the rationality of changes in the test results;

[0015] The change evaluation processor calculates the change evaluation value S according to the following formula:

[0016] S=w1·(Aa-Ab)+w2·(La-Lb)+w3·Fr;

[0017] Among them, Aa represents the knowledge accuracy of the test content after the modification of the large model, Ab represents the knowledge accuracy of the test content before the modification of the large model, La represents the logic accuracy of the test content after the modification of the large model, Lb represents the logic accuracy of the test content before the modification of the large model, Fr represents the reduction of negative feedback, and w1, w2 and w3 are three weight coefficients respectively;

[0018] When S is a negative value, the iterative control processor starts the iterative program. Each time the iterative control processor starts the iterative program, a new target node is selected for parameter adjustment.

[0019] Furthermore, the parameter control processor calculates the parameter adjustment amount Δv(n) according to the following formula:

[0020]

[0021] Where n is the number of iterations, S(i) represents the change evaluation value of the i-th iteration, and Δv(i) represents the parameter adjustment value of the i-th iteration.

[0022] Furthermore, the correction content replacement unit includes a code interpretation processor, a replacement content database and a replacement execution processor, the code interpretation processor is used to interpret and analyze the annotation code, the replacement content database is used to store replacement text information, and the replacement execution processor selects corresponding text information based on the interpretation result to replace the annotation code.

[0023] The beneficial effects achieved by the present invention are:

[0024] Based on feedback information, this system outputs three types of adjustment information. The first type is used to directly optimize the large model parameters and govern from the content generation end. The second type is used to optimize the detection end to improve the detection rate of factual errors and logical errors. The third type is optimized from the modification end and can modify and replace non-real-time and logical erroneous content, comprehensively reducing the probability of hallucination problems in the content presented by the large model.

[0025] To further understand the features and technical contents of the present invention, please refer to the following detailed description and drawings of the present invention. However, the drawings provided are only for reference and description and are not intended to limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a schematic diagram of the overall structural framework of the present invention;

[0027] Figure 2 This is a schematic diagram of the feedback collection module of the present invention;

[0028] Figure 3 This is a schematic diagram of the composition of the hallucination detection module of the present invention;

[0029] Figure 4 A schematic diagram of the composition of the optimization module generated for the present invention;

[0030] Figure 5 This is a schematic diagram of the real-time deviation correction module of the present invention;

[0031] Figure 6 This is a diagram showing the effect of actual feedback quantity changes in the present invention. DETAILED DESCRIPTION

[0032] The following is an explanation of the embodiments of the present invention through specific embodiments. Those skilled in the art can understand the advantages and effects of the present invention from the contents disclosed in this specification. The present invention can be implemented or applied through other different specific embodiments, and the details in this specification can also be modified and changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. In addition, the drawings of the present invention are only simple schematic illustrations and are not depicted according to actual sizes. It is stated in advance. The following embodiments will further explain the relevant technical contents of the present invention in detail, but the disclosed contents are not intended to limit the scope of protection of the present invention.

[0033] Embodiment 1.

[0034] This embodiment provides a large model hallucination management system based on user feedback. Figure 1 , including feedback collection module, hallucination detection module, generation optimization module and real-time correction module.

[0035] The feedback collection module is used to collect user feedback information on generated content, the hallucination detection module is used to perform hallucination detection on the generated content of the large model, the generation optimization module is used to optimize the content generation process of the large model, and the real-time correction module is used to correct and rectify the content generated by the large model.

[0036] The feedback collection module includes a feedback interface management unit, a feedback data management unit and a feedback output processing unit. The feedback interface management unit is used to receive user feedback information, the feedback data management unit is used to organize and analyze the feedback information, and the feedback output processing unit outputs adjustment information based on the analysis result of the feedback information.

[0037] The hallucination detection module includes a content verification unit, a logic verification unit and a detection optimization unit. The content verification unit is used to compare and analyze the generated content to verify the correctness of the content. The logic verification unit verifies the correctness of the content based on the upper and lower logical relationship of the generated content. The detection optimization unit optimizes the verification parameters based on the adjustment information.

[0038] The generation optimization module includes a feedback-driven fine-tuning unit, an optimization monitoring unit and an optimization control unit. The feedback-driven fine-tuning unit makes slight adjustments to the generation parameters in the large model based on the adjustment information. The optimization monitoring unit is used to monitor the optimization state of the large model. The optimization control unit controls the optimization process based on the monitoring results.

[0039] The real-time correction module includes a generation information cache unit, a correction content locking unit and a correction content replacement unit. The generation information cache unit is used to receive the generated content of the large model, the correction content locking unit adjusts information and detection information to determine the content that needs to be corrected, and the correction content replacement unit performs replacement processing based on the locked content.

[0040] The feedback output processing unit includes a detection output processor, a generation output processor and a correction output processor. The detection output processor is used to output detection adjustment information to the hallucination detection module, the generation output processor is used to output generation adjustment information to the generation optimization module, and the correction output processor is used to output correction adjustment information to the real-time correction module.

[0041] The optimization monitoring unit includes a content test processor and a change evaluation processor. The content test processor is used to perform content generation test processing on the adjusted large model, and the change evaluation processor is used to evaluate the rationality of changes in the test results.

[0042] The change evaluation processor calculates the change evaluation value S according to the following formula:

[0043] S=w1·(Aa-Ab)+w2·(La-Lb)+w3·Fr.

[0044] Among them, Aa represents the knowledge accuracy of the test content after the big model is modified, Ab represents the knowledge accuracy of the test content before the big model is modified, La represents the logical accuracy of the test content after the big model is modified, Lb represents the logical accuracy of the test content before the big model is modified, Fr is the reduction of negative feedback, and w1, w2 and w3 are three weight coefficients respectively.

[0045] When S is a negative value, the iterative control processor starts the iterative program. Each time the iterative control processor starts the iterative program, a new target node is selected for parameter adjustment.

[0046] The parameter control processor calculates the parameter adjustment amount Δv(n) according to the following formula:

[0047]

[0048] Where n is the number of iterations, S(i) represents the change evaluation value of the i-th iteration, and Δv(i) represents the parameter adjustment value of the i-th iteration.

[0049] The correction content replacement unit includes a code interpretation processor, a replacement content database and a replacement execution processor. The code interpretation processor is used to interpret and analyze the annotation code, the replacement content database is used to store replacement text information, and the replacement execution processor selects corresponding text information based on the interpretation result to replace the annotation code.

[0050] Embodiment 2.

[0051] This embodiment includes all the contents of the first embodiment, and provides a large model hallucination management system combining user feedback, including a feedback collection module, a hallucination detection module, a generation optimization module and a real-time correction module.

[0052] The feedback collection module is used to collect user feedback information on generated content, the hallucination detection module is used to perform hallucination detection on the generated content of the large model, the generation optimization module is used to optimize the content generation process of the large model, and the real-time correction module is used to correct and rectify the content generated by the large model.

[0053] Combination Figure 2 The feedback collection module includes a feedback interface management unit, a feedback data management unit and a feedback output processing unit. The feedback interface management unit is used to receive user feedback information, the feedback data management unit is used to organize and analyze the feedback information, and the feedback output processing unit outputs adjustment information based on the analysis results of the feedback information.

[0054] Combination Figure 3 The hallucination detection module includes a content verification unit, a logic verification unit and a detection optimization unit. The content verification unit is used to compare and analyze the generated content to verify the correctness of the content. The logic verification unit verifies the correctness of the content based on the upper and lower logical relationship of the generated content. The detection optimization unit optimizes the verification parameters based on the adjustment information.

[0055] Combination Figure 4The generation optimization module includes a feedback-driven fine-tuning unit, an optimization monitoring unit and an optimization control unit. The feedback-driven fine-tuning unit makes slight adjustments to the generation parameters in the large model based on the adjustment information. The optimization monitoring unit is used to monitor the optimization state of the large model. The optimization control unit controls the optimization process based on the monitoring results.

[0056] Combination Figure 5 .The real-time correction module includes a generation information cache unit, a correction content locking unit and a correction content replacement unit. The generation information cache unit is used to receive the generation content of the large model, the correction content locking unit adjusts the information and detection information to determine the content that needs to be corrected, and the correction content replacement unit performs replacement processing based on the locked content.

[0057] The feedback interface management unit includes an interface input processor, an interface setting processor and an interface monitoring processor. The interface input processor is used to provide an interface for inputting feedback information, the interface setting processor is used to set the network information of the interface, and the interface monitoring processor is used to monitor the input status of the interface and obtain feedback information.

[0058] The feedback data management unit includes an information classification processor, a feedback data register and an information analysis processor. The information classification processor is used to classify each feedback information. The feedback data register creates a storage space based on each classification to save the feedback information of the corresponding classification. The information analysis processor is used to summarize and analyze the feedback information under the classification whose quantity reaches the analysis standard.

[0059] The feedback output processing unit includes a detection output processor, a generation output processor and a correction output processor. The detection output processor is used to output detection adjustment information to the hallucination detection module, the generation output processor is used to output generation adjustment information to the generation optimization module, and the correction output processor is used to output correction adjustment information to the real-time correction module.

[0060] The content verification unit includes a knowledge database register, a comparison and verification processor and a marking output processor. The knowledge database register is used to store correct knowledge data, the comparison and verification processor is used to compare the generated content with the knowledge data, and the marking output processor is used to mark the erroneous parts in the generated content and forward them to the logic verification unit.

[0061] The logic verification unit includes a logic information register, a logic extraction processor and a logic judgment processor. The logic information register is used to store logic rule information. The logic extraction processor extracts logic keywords from the generated content based on logic rules and converts them into logic codes. The logic judgment processor is used to perform contradiction judgment on a sequence composed of logic codes.

[0062] The logic rule information includes logic categories, words belonging to each logic category, conversion relationships between words and logic codes, and conflicting code sequences.

[0063] The detection optimization unit includes an adjustment guidance processor, a knowledge update processor and a rule update processor. The adjustment guidance processor is used to receive detection adjustment information and guide it to corresponding update content. The knowledge update processor updates the knowledge data in the knowledge database register based on the update content. The rule update processor updates the logic rule information in the logic information register based on the update content.

[0064] The feedback driven fine-tuning unit includes a fine-tuning locking processor and a parameter adjustment processor. The fine-tuning locking processor locks the target node in the large model based on the generated adjustment information, and the parameter adjustment processor is used to make fine adjustments to the parameters in the target node.

[0065] The optimization monitoring unit includes a content test processor and a change evaluation processor. The content test processor is used to perform content generation test processing on the adjusted large model, and the change evaluation processor is used to evaluate the rationality of changes in the test results.

[0066] The optimization control unit includes an iterative control processor and a parameter control processor. The iterative control processor performs iterative control on the adjustment process based on the evaluation result, and the parameter control processor is used to control the change amount of the adjusted parameters during the iterative process.

[0067] The change evaluation processor calculates the change evaluation value S according to the following formula:

[0068] S=w1·(Aa-Ab)+w2·(La-Lb)+w3·Fr.

[0069] Among them, Aa represents the knowledge accuracy of the test content after the big model is modified, Ab represents the knowledge accuracy of the test content before the big model is modified, La represents the logical accuracy of the test content after the big model is modified, Lb represents the logical accuracy of the test content before the big model is modified, Fr is the reduction of negative feedback, and w1, w2 and w3 are three weight coefficients respectively.

[0070] When S is a negative value, the iterative control processor starts the iterative program. Each time the iterative control processor starts the iterative program, a new target node is selected for parameter adjustment.

[0071] The parameter control processor calculates the parameter adjustment amount Δv(n) according to the following formula:

[0072]

[0073] Where n is the number of iterations, S(i) represents the change evaluation value of the i-th iteration, and Δv(i) represents the parameter adjustment value of the i-th iteration.

[0074] The correction content locking unit includes a labeling recognition processor, an adjustment recognition processor and a conversion preprocessor. The labeling recognition processor is used to recognize the labeling information of the hallucination detection module. The adjustment recognition processor recognizes similar error information based on the correction adjustment information. The conversion preprocessor is used to convert the recognized information into annotation code.

[0075] The correction content replacement unit includes a code interpretation processor, a replacement content database and a replacement execution processor. The code interpretation processor is used to interpret and analyze the annotation code, the replacement content database is used to store replacement text information, and the replacement execution processor selects corresponding text information based on the interpretation result to replace the annotation code.

[0076] The governance process of the system for large model generated content includes the following steps:

[0077] S1. Adjust the large model parameters by generating an optimization module.

[0078] S2. Use the hallucination detection module to detect the content generated by the large model and find out factual errors and logical errors.

[0079] S3. The real-time error correction module is used to identify the erroneous contents that are not factual errors and logical errors, and all the detected erroneous contents are replaced at the same time.

[0080] S4. Output the generated content after governance.

[0081] The i in the above text is an ordinal number used to represent a sequence number.

[0082] Some code information of this system is as follows:

[0083]

[0084] def __init__(self):

[0085] self.feedback_interface_manager=

[0086] self.FeedbackInterfaceManager()

[0087] self.feedback_data_manager=self.FeedbackDataManager()

[0088] self.feedback_output_handler=self.FeedbackOutputHandler()

[0089] class FeedbackInterfaceManager:

[0090] def receive_feedback(self,feedback):

[0091] """Receive user feedback."""

[0092] print("Feedback received:",feedback)

[0093] class HallucinationDetectionModule:

[0094] """Hallucination detection module, including content verification, logic verification and detection optimization."""

[0095] def __init__(self):

[0096] self.content_validator=self.ContentValidator()

[0097] self.logic_validator=self.LogicValidator()

[0098] self.detection_optimizer=self.DetectionOptimizer()

[0099] class ContentValidator:

[0100] def verify_content(self,generated_content):

[0101] """Verify the correctness of the generated content."""

[0102] #TODO: Implement content comparison analysis logic

[0103] print("Verifying content:",generated_content)

[0104] class LogicValidator:

[0105] def validate_logic(self,generated_content):

[0106] """Verify the correctness of the content based on the logical relationship between contexts."""

[0107] #TODO: Implement logical relationship detection

[0108]

[0109] print("Controlling optimization process with:",monitoring_result)

[0110] class RealTimeCorrectionModule:

[0111] """Real-time correction module, including information caching, locking and replacement units."""

[0112] def __init__(self):

[0113] self.generation_info_cache=self.GenerationInfoCache()

[0114] self.correction_content_locker=self.CorrectionContentLocker()

[0115] self.correction_content_replacer=

[0116] self.CorrectionContentReplacer()

[0117] class GenerationInfoCache:

[0118] def cache_generated_content(self,content):

[0119] """Cache generated content."""

[0120] #TODO: Implement caching logic

[0121] print("Caching generated content:",content)

[0122] class CorrectionContentLocker:

[0123] def identify_content_for_correction(self,adjustment_info,detection_info):

[0124] """Lock the content that needs to be corrected based on the adjustment and detection information."""

[0125] #TODO: Implement content locking logic

[0126] print("Locking content for correction with info:",adjustment_info,detection_info)

[0127] class CorrectionContentReplacer:

[0128] def replace_locked_content(self,locked_content,replacement_content):

[0129] """Replace locked error content."""

[0130] #TODO: Implement replacement logic

[0131] print("Replacing locked content:",locked_content,"with:",replacement_content).

[0132] Now we take 100 times of generating content as a statistical point and count the number of user feedbacks to get Figure 6 The changes shown in the figure.

[0133] The contents disclosed above are only preferred feasible embodiments of the present invention, and do not limit the protection scope of the present invention. Therefore, all equivalent technical changes made using the contents of the present invention specification and drawings are included in the protection scope of the present invention. In addition, the elements therein can be updated as technology develops.

Claims

1. A large model hallucination management system based on user feedback, characterized in that: It includes feedback collection module, hallucination detection module, generation optimization module and real-time correction module; The feedback collection module is used to collect user feedback information on generated content, the hallucination detection module is used to perform hallucination detection on the generated content of the large model, the generation optimization module is used to optimize the content generation process of the large model, and the real-time correction module is used to correct errors and rectify the content generated by the large model; The feedback collection module includes a feedback interface management unit, a feedback data management unit and a feedback output processing unit. The feedback interface management unit is used to receive user feedback information, the feedback data management unit is used to organize and analyze the feedback information, and the feedback output processing unit outputs adjustment information based on the analysis result of the feedback information; The hallucination detection module includes a content verification unit, a logic verification unit and a detection optimization unit. The content verification unit is used to compare and analyze the generated content to verify the correctness of the content. The logic verification unit verifies the correctness of the content based on the logical relationship between the generated content. The detection optimization unit optimizes the verification parameters based on the adjustment information. The generation optimization module includes a feedback-driven fine-tuning unit, an optimization monitoring unit and an optimization control unit. The feedback-driven fine-tuning unit makes a slight adjustment to the generation parameters in the large model based on the adjustment information. The optimization monitoring unit is used to monitor the optimization state of the large model. The optimization control unit controls the optimization process based on the monitoring result. The real-time correction module includes a generation information cache unit, a correction content locking unit and a correction content replacement unit. The generation information cache unit is used to receive the generated content of the large model, the correction content locking unit adjusts information and detection information to determine the content that needs to be corrected, and the correction content replacement unit performs replacement processing based on the locked content.

2. A system for managing large model hallucinations in combination with user feedback as claimed in claim 1, characterized in that: The feedback output processing unit includes a detection output processor, a generation output processor and a correction output processor. The detection output processor is used to output detection adjustment information to the hallucination detection module, the generation output processor is used to output generation adjustment information to the generation optimization module, and the correction output processor is used to output correction adjustment information to the real-time correction module.

3. A system for managing large model hallucinations in combination with user feedback as claimed in claim 2, characterized in that: The optimization monitoring unit includes a content test processor and a change evaluation processor, wherein the content test processor is used to perform content generation test processing on the adjusted large model, and the change evaluation processor is used to evaluate the rationality of changes in the test results; The change evaluation processor calculates the change evaluation value S according to the following formula: S=w1·(Aa-Ab)+w2·(La-Lb)+w3·Fr; Among them, Aa represents the knowledge accuracy of the test content after the modification of the large model, Ab represents the knowledge accuracy of the test content before the modification of the large model, La represents the logic accuracy of the test content after the modification of the large model, Lb represents the logic accuracy of the test content before the modification of the large model, Fr represents the reduction of negative feedback, and w1, w2 and w3 are three weight coefficients respectively; When S is a negative value, the iterative control processor starts the iterative program. Each time the iterative control processor starts the iterative program, a new target node is selected for parameter adjustment.

4. A system for managing large model hallucinations in combination with user feedback as claimed in claim 3, characterized in that: The parameter control processor calculates the parameter adjustment amount Δv(n) according to the following formula: Where n is the number of iterations, S(i) represents the change evaluation value of the i-th iteration, and Δv(i) represents the parameter adjustment value of the i-th iteration.

5. A system for managing large model hallucinations in combination with user feedback as claimed in claim 4, characterized in that: The correction content replacement unit includes a code interpretation processor, a replacement content database and a replacement execution processor. The code interpretation processor is used to interpret and analyze the annotation code, the replacement content database is used to store replacement text information, and the replacement execution processor selects corresponding text information based on the interpretation result to replace the annotation code.

Images