A system and method for large model generation content information security reinforcement

Abstract

The application discloses a system and method for large model generated content information security reinforcement, first, the large model is fine-tuned by bidirectional LoRA and knowledge distillation technology, so that the large model adapts to the factual output task from the two dimensions of illusion and fact; the intermediate layer representation of the fine-tuned large model is extracted in the fact space, and different mappings from the fine-grained illusion category to the fact direction are obtained by the contrast learning method; finally, the intermediate layer representation of the large model is extracted to judge the potential illusion type in the inference generation process of the large model, and then the learned mapping corresponding to different illusions is applied to the intermediate representation of the inference process, the factual expression is corrected, and the probability of illusion output is reduced; in the pre-training stage and the inference generation stage, the parameters and the generated content of the large model are controlled, the somatic generation ability of the large model is ensured, and the factuality and reliability of the large model generated content are greatly improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence, and particularly relates to a system and method for content information security reinforcement of a large model. BACKGROUND

[0002] Large Language Models (LLMs) are a class of models based on deep learning that typically use neural networks (especially variants like the Transformer model) to process and generate natural language. They are trained on large amounts of text data to learn the structure, patterns, and contextual relationships of language, enabling them to perform various language-related tasks such as text generation, question answering, translation, dialogue systems, etc. LoRA fine-tuning is a technique that decomposes the parameters of a large model into low-rank matrices and fine-tunes them based on this decomposition. Its purpose is to reduce training time and parameter size while maintaining model performance, especially suitable for fine-tuning specific tasks in resource-constrained environments. Knowledge distillation is a process that transfers the knowledge of a large model to a smaller model. It helps smaller models learn the most important features in a task by using the predictions of a large model as a supervisory signal, thus achieving model compression without significantly reducing model performance. Directional editing techniques are used in large models to change the specific direction of the model's generated text or images. By adjusting the model's latent representation, researchers can guide the model to generate outputs that conform to a specific style or attribute, so it has been widely applied in generative tasks to adjust the content of large model outputs. Contrastive learning is an unsupervised learning method that constructs positive and negative sample pairs to learn representations. In the field of large models, contrastive learning can be used to divide positive and negative sample pairs so that positive and negative samples are pushed away from each other, distinguishing representations and improving the performance of large models in classification tasks.

[0003] Through the use of existing contract text generation large models, it is found that the existing large model adopts the LoRA fine-tuning scheme to adapt the underlying large model to downstream contract reading comprehension, keyword extraction, and report generation. In the running process, there are problems such as a large number of hallucination outputs in the generated content, including entity extraction hallucination and relationship hallucination, which cause the generated text report to be unable to be used correctly, requiring manual re-audit and increasing labor costs. In addition, in the current practical application scenario, many large model systems that adapt to downstream text generation tasks use a single LoRA forward fine-tuning to improve factual output capability, which is not obvious. At the same time, current large models mostly use inference editing methods to adjust the latent representation, editing from hallucination to fact; but past research shows that there are many types of hallucinations in large models, and the binary hallucination classification has insufficient representation ability and the latent representation editing improvement is not obvious. SUMMARY

[0004] In view of the above problems existing in the prior art in the contract text generation large model, based on the large model, the LoRA fine-tuning technology, the contrast learning, the knowledge distillation and the representation editing technology, the application provides a system and method for large model generated content information security reinforcement, the LoRA fine-tuning technology in the scheme can more efficiently fine-tune the basic large model in a low-resource environment, starting from the perspective of hallucination samples, reduce resource waste, through the knowledge distillation technology, use the positive fact sample to improve the large model in the front and rear layer fact output ability, the improved representation editing technology can edit the potential representation of the basic large model in the inference in the direction of finer granularity from the perspective of the whole process, and optimize the direction from different kinds of hallucinations to facts.

[0005] The technical scheme adopted by the application is as follows:

[0006] A system for large model generated content information security reinforcement, comprising a bidirectional fine-tuning module, a fine-grained fact direction extraction module and an inference fact direction editing module;

[0007] The bidirectional fine-tuning module is unidirectionally connected to the fine-grained fact direction extraction module, and the fine-grained fact direction extraction module is unidirectionally connected to the inference fact direction editing module;

[0008] The bidirectional fine-tuning module is used for keyword extraction, and comprises fine-tuning training and self-distillation enhancement fact output on keyword extraction samples;

[0009] The fine-grained fact direction extraction module comprises fine-grained fact direction extraction for pre-processing of the fine-tuned model, wherein the fine-grained fact direction is a fine-grained division of hallucinations in the large model answer, and the hidden layer representation is extracted and mapped to the fact space;

[0010] The inference fact direction editing module differentially edits the inference process and corrects the fact expression.

[0011] A method for large model generated content information security reinforcement, comprising a large model internal parameter fact fine-tuning function and an inference fact editing function, the large model internal parameter fact fine-tuning function comprising the following steps:

[0012] Step A: classifying the large model answers in the data set, and constructing corresponding hallucination answer data sets and fact answer data sets;

[0013] Step B: LoRA fine-tuning on the hallucination level for data from hallucination answers, learning useful knowledge for fact expression from hallucination samples;

[0014] Step C: To address the inconsistent focus on factual information between earlier and more mature layers of the large model, a self-distillation method is used to stimulate the large model with factual response datasets, and the large model performs distillation between earlier and later layers, so that the factual representations of earlier and later layers tend to be consistent.

[0015] The fine-grained fact-direction extraction module includes the following steps:

[0016] Step D: Utilize the existing trained large model to perform more granular classification of hallucination samples, and extract the corresponding fact directions after stimulating the internal representation of the large model;

[0017] Step E: Based on the clustering of the hidden layer representations during the reasoning process, the fine-grained fact directions are mapped to different directions in the fact space.

[0018] Preferably, step A specifically includes the following steps:

[0019] Step A1: Ask a question x using the basic large model, organize the answers y of the corresponding questions in the large model, and obtain the dataset tuple (x, y);

[0020] Step A2: Manually classify y in the binary data set and add corresponding labels to the illusionary answer y, dividing it into D = {D true D hall Real samples and hallucinatory samples;

[0021] Preferably, step B specifically includes the following steps:

[0022] Step B1: Organize the hallucination samples into an instruction fine-tuning dataset, and use the LoRA module to perform negative fine-tuning to learn the factual knowledge contained in the hallucination samples or eliminate hallucination content.

[0023] Step B2: Learn from the illusion fine-tuning to obtain the incremental parameters θ of LoRA, which will be added to the overall network parameters later;

[0024] Preferably, step C specifically includes the following steps:

[0025] Step C1: The self-distillation of the front and back layers of the large model at the fact level is based on the real sample D. true Make fine adjustments to enhance the consistency of the generation trends of factual answers across different layers;

[0026] Preferably, step C1 specifically includes the following steps:

[0027] Step C101: The large model is derived from real samples D true The data in the middle is activated, and the internal representations of the earlier and last layers are extracted;

[0028] Step C102: Using the output of the last layer of the large model as supervision, the internal representations extracted from previous layers are used for prediction, and the KL divergence formula is used as its loss function. Specifically: Used to measure the difference between the probability distribution of the output of the previous layer and the output of the last layer.

[0029] Step C103: Set N to the total number of samples, i.e., D true The number of factual answers; calculation and These are the probability distributions from the last layer and the kth layer, respectively;

[0030] Step C104: To ensure consistency of features between preceding and following layers, a mean squared loss function is introduced. in, and These are the outputs of the hidden layer.

[0031] Step C2: Combining the two loss functions mentioned above, start self-distillation, allowing the larger model to learn the factual output of the last layer from the earlier layers;

[0032] Preferably, step C2 specifically includes the following steps:

[0033] Step C201: The loss functions obtained in step C1 are summed to calculate the comprehensive loss function, as shown in the following formula: L Combined =α·L Layer-wise +β·L KL , where α and β are the weights of the mean squared loss function and the fact consistency loss function, respectively, and can be changed according to the specific large model and the specific task;

[0034] Step C202: Train the large model using the comprehensive loss function to obtain a large model that maintains factual consistency between the front and back layers;

[0035] Preferably, step D specifically includes the following steps:

[0036] Step D1: The fine-grained fact orientation extraction method first processes the dataset to be used, labels the fine-grained hallucination answers, activates the large model by combining the question and hallucination answer pairs, and extracts their internal representations.

[0037] Preferably, step D1 specifically includes the following steps:

[0038] Step D101: The dataset is manually re-verified and the hallucination responses are finely divided into entity hallucinations, relational hallucinations, sentence hallucinations, and unverifiable hallucinations according to the type of hallucination;

[0039] Step D102: Use the partitioned dataset for activation of the large model for scheme selection, and extract internal representations at each layer of the large model;

[0040] Step D103: Map the internal representation to the fact space and semantic space respectively through the fact encoder and semantic encoder to avoid interference from semantic information during orientation extraction;

[0041] Step D2: Modify the contrastive learning loss function to maximize the difference between facts and fine-grained illusions, while avoiding excessive convergence between different types of illusions;

[0042] Preferably, step D2 specifically includes the following steps:

[0043] Step D201: The scheme maximizes the difference between fact and fine-grained illusion by maximizing the following formula The difference is calculated using sim similarity, X. + It is a set of fact samples. It is the sample set of the i-th type of hallucination;

[0044] Step D202: To prevent different types of hallucinations from converging too closely, the following function is used. Different types of hallucinations push each other away;

[0045] Step D203: Combining the two loss functions above, the overall fact loss function is L. truth =L true-hall +λL hall-hall , where λ is a hyperparameter, minimizing this function lays the foundation for subsequent extraction directions;

[0046] Step D204: By comparing the learning process and the set loss function, the training effectively distinguishes between hallucination types and facts in the fact space. This step calculates the average value of the same hallucination type in the space and subtracts the average values ​​to obtain the δ values ​​corresponding to different hallucination types. i Fine-grained fact extraction is complete;

[0047] Preferably, step E specifically includes the following steps:

[0048] Step E1: Using the large model that has been fine-tuned bidirectionally as described above, the large model responds to the user's input question Q, extracts internal representations at the same layer, and determines the type of potential hallucination.

[0049] Preferably, step E1 specifically includes the following steps:

[0050] Step E101: The user inputs question Q to activate the large model. After extracting the internal representation, the possible hallucination types or facts are simply determined by the clustering algorithm. If it is a specific hallucination type i, proceed to step E102. If it is a fact type, end step E without further editing.

[0051] Step E102: Add the corresponding δ based on the identified type of hallucination. i The specific incremental calculation formula is Δ=Dec(h sem +Attn(h sem ,h truth +δ))-Dec(h sem +Attn(h sem ,h truth -δ));

[0052] Step E2: Add the calculated incremental formula back into the internal representation, specifically the formula x. ' = x + α × Δ, where α is the incremental weight of directional editing, and x ' Continue with normal restoration within the larger model.

[0053] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0054] 1. This invention provides a system and method for enhancing the information security of content generated by large-scale models. Through the coordinated operation of multiple stages—a bidirectional fine-tuning module, a fine-grained fact-direction extraction module, and a fine-grained fact-direction editing module during fine-grained inference—the contract text extraction function based on the text generation large model can more effectively generate factual content, comprehensively considering and avoiding the generation of illusory information. By utilizing the bidirectional fine-tuning module and self-distillation technology, while effectively using illusory samples for training, the internal parameters of the large model are also strengthened, allowing earlier layers to learn the attention paid to factual content by mature layers, maintaining consistency.

[0055] 2. This invention addresses the problems of wasted hallucination samples generated by large models and the past perception of hallucinations and facts as two binary opposites. Our solution involves fine-grained classification of hallucinations to learn editing directions for different hallucination types and facts in the fact space, thereby optimizing the content generated by the large model towards the truth at the knowledge editing level. Specifically, during training, the direction from different types of hallucinations to facts is learned. In subsequent inference, a clustering algorithm is used to determine potential hallucination types and their corresponding directions are added to the original intrinsic representation, ultimately resulting in an edited output. Hallucination samples generated by large models were previously considered worthless. Our solution reuses these samples by first labeling them with hallucination types and then fine-tuning them. This method of fine-tuning is more effective than fine-tuning using only factual samples, improving training efficiency.

[0056] 3. This invention comprises two main stages: pre-application and application of a large-scale text generation model. Before application, pre-training is used to fine-tune the model's parameters, adapting it to downstream contract keyword extraction tasks while learning to output factual content from both illusion and factual dimensions. During application, fine-grained knowledge editing is used to correct potentially illusory answers. This invention mainly includes four functions: First, the utilization of illusionary information. Unlike previous methods that solely utilized facts or negative labels for illusions, this solution utilizes illusionary information by collecting an illusionary dataset from real user feedback after deployment. Fine-tuning using this illusionary approach enables iterative optimization of the large-scale model. Second, optimizing the latent representations of the large-scale model's earlier layers involves modifying the model's earlier parameters through self-distillation, achieving early attention to factual information. The fine-grained factual direction extraction function is an operation performed on the finely tuned large-scale model. By activating the model and learning fine-grained directions from its internal representations, it studies the model's own parameters. Finally, the selected model's factual editing direction is used during inference, effectively correcting potentially illusory answers and enhancing factual output capabilities. Attached Figure Description

[0057] Figure 1 This is a schematic diagram of the system structure of the present invention;

[0058] Figure 2 This is a flowchart of the method of the present invention;

[0059] Figure 3 Activity diagram for the dataset preparation phase in Example 2;

[0060] Figure 4 This is an activity diagram of the negative hallucination fine-tuning stage in Example 2;

[0061] Figure 5This is an activity diagram of the positive fact self-distillation stage in Example 2.

[0062] Figure 6 This is an activity diagram of the fine-grained fact direction extraction stage in Example 2.

[0063] Figure 7 This is an activity diagram of the fact-direction editing stage during reasoning in Example 2. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0065] Example 1

[0066] like Figure 1 As shown, this invention provides a system for enhancing the information security of content generated by large models, including a bidirectional fine-tuning module, a fine-grained fact direction extraction module, and a fact direction editing module during inference. The bidirectional fine-tuning module is used in the contract text keyword extraction task downstream of the basic large model adaptation. The adaptation task includes fine-tuning training and self-distillation of keyword extraction sample instances to enhance factual output and reduce hallucination representation. The fine-grained fact direction extraction module includes fine-grained fact direction extraction preprocessing of the model obtained from the previous fine-tuning step. The fine-grained fact direction is used to fine-grainedly segment hallucinations in the large model's response, extracting hidden layer representations and mapping them to facts. In the space; the fact-direction editing module during reasoning is responsible for differential editing of the reasoning process. Specifically, the differential editing scheme is to perform fine-grained fact-direction editing on the hidden layer feature representation in the clustering reasoning process to correct its factual statements. In Example 1, we assume that the application scenario is keyword extraction of asset contracts to generate a concise and clear report containing information such as amount, time, and contract parties. Considering that the initial input of this scenario is a detailed but excessively long contract document, and the pre-output is a concise report containing key information after being learned and organized by a large text model, the most important requirement in this scenario is that the generated report does not produce illusions, thus making it an excellent implementation scenario for this invention.

[0067] In Example 1, the present invention comprises two main stages: a pre-application stage involving a large text generation model, and a pre-application stage involving a pre-training method to fine-tune the parameters of the large model to adapt it to the downstream contract keyword extraction task, while learning to output factual content from both illusion and fact dimensions; and a fine-grained knowledge editing method to correct any illusory answers that might have occurred during the application process. This invention mainly includes four functions: First, to ensure the real-time updating of the subsequent large model and avoid hallucinations, the first function utilizes hallucination information. Unlike previous methods that solely utilize facts or negative labels for hallucinations, this solution utilizes hallucination information by collecting a hallucination dataset from real user feedback after deployment and operation. This hallucination fine-tuning allows for iterative optimization of the large model. Second, it optimizes the potential representations of the preceding and following layers of the large model by using self-distillation to modify the parameters of earlier layers, enabling early attention to factual information in the contract report generation task. The fine-grained fact direction extraction function is an operation performed on the already fine-tuned large model. It activates the large model through full-fact contract reports and hallucination-protecting contract reports, learning fine-grained directions from internal representations. This is a study of the model's parameters and its adaptation to the task. Finally, the function utilizes the selected model's fact editing directions during inference, effectively correcting potentially hallucinatory answers and enhancing factual output capabilities.

[0068] Example 2

[0069] like Figure 2 and Figure 3 As shown, this invention also provides a method for enhancing the security of content generated by large models, which is also applicable to knowledge question-and-answer tasks in vertical domains. The preset scenario for Example 2 is the asset contract management field. The specific task format is a dialogue with memory, including questions and answers on asset contract document interpretation, common dispute analysis, and asset contract-related knowledge. The initial input is a simple question from the user to the large model, and the pre-output is the corresponding answer generated by the large model. In this scenario, the large model is applied to a vertical domain, involving a narrower breadth of content but requiring high accuracy, thus making it an excellent implementation case for this invention. Example 2 includes the following steps:

[0070] Step A: Classify the large model responses in the dataset to construct corresponding hallucination response datasets and fact response datasets;

[0071] Step B: Perform LoRA fine-tuning at the hallucination level on the data from hallucination responses to learn knowledge useful for factual expression from the hallucination samples;

[0072] Step C: To address the inconsistent focus on factual information between earlier and mature layers of the large model, a self-distillation method is used to stimulate the large model with factual datasets and allow the large model to distill the factual representations of the earlier and later layers, so that the factual representations of the earlier and later layers tend to be consistent.

[0073] The fine-grained fact-direction extraction module includes the following steps:

[0074] Step D: Utilize the existing trained large model to perform more granular classification of hallucination samples, distinguishing between entity hallucinations, relational hallucinations, and unverifiable hallucinations, and extract the corresponding fact directions after stimulating the internal representation of the large model;

[0075] Step E: Based on the clustering of the hidden layer representations during the reasoning process, the fine-grained fact directions are mapped to different directions in the fact space.

[0076] like Figure 2 As shown, this embodiment includes the entire process of the large-scale model illusion reduction scheme for text generation, comprising a dataset preparation stage, a negative illusion fine-tuning stage, a positive fact self-distillation stage, a fine-grained fact direction extraction stage, and a fine-grained fact direction editing stage during inference. In the dataset preparation stage, this embodiment constructs a dataset using existing open-source low-carbon environmental protection question-and-answer datasets or by asking questions to the large model and then collecting illusion and fact answers, for subsequent fine-tuning and direction extraction editing. The negative illusion fine-tuning stage specifically uses the dataset containing illusion answers obtained in the previous stage to learn from illusion samples using the LoRA fine-tuning method, thereby acquiring factual knowledge. The positive fact self-distillation stage operates on the preceding and following layers of the large model, optimizing the representations of the preceding layers of the large model by using the representation of the last layer when outputting facts as guidance. The aim is to enhance the consistent attention of each layer of the large model to factual information. After this stage, the bidirectional fine-tuning stage is completed. The fine-grained fact direction extraction operates on the large model that has been fine-tuned in the previous step, extracting the directions from various illusion types to facts from the hidden layers of the large model. During reasoning, the potential illusions in the intermediate layers of the large model are added with the corresponding factual directions extracted in the previous step. By using this method, the problem of illusions can be corrected in a timely manner when in use.

[0077] Step A, which involves constructing the dataset for the implementation case, specifically includes the following steps:

[0078] Step A1: Find the initial question-and-answer dataset in the asset contract domain, collect the questions x for the basic large model, organize the answers y for the corresponding questions in the large model, and obtain the dataset tuple (x, y);

[0079] Step A2: Manually classify the y in the binary data set and add corresponding labels to the hallucination answer y, dividing it into D = {D true D hall Real samples and hallucinatory samples;

[0080] like Figure 3 As shown in this embodiment, the dataset preparation stage requires the participation of developers and a selected basic model. Developers obtain answers by asking questions to the basic model, and then reclassify the answers as either hallucinations or facts. Specifically, developers first search for low-carbon and environmentally friendly information, organize it, and then ask multiple rounds of questions to the model, collecting answers and determining whether the answers contain hallucinations. After the questioning, the dataset is organized into a negative hallucination dataset and a positive fact dataset.

[0081] Then proceed to step B, which includes the following:

[0082] Step B1: Organize the hallucination samples (x, y) into an instruction fine-tuning dataset, and use the LoRA module to perform negative fine-tuning to learn the factual knowledge contained in the hallucination samples or eliminate hallucination content.

[0083] Step B2: Learn from the illusion fine-tuning to obtain the incremental parameters θ of LoRA, which will be added to the overall network parameters later;

[0084] like Figure 4 As shown, in this embodiment, the negative hallucination fine-tuning stage involves operating on the hallucination sample dataset from the previous step to learn knowledge from the hallucination samples. Specifically, the hallucination dataset samples obtained in the previous step are additionally labeled, local labels are added to the answers, and annotations are added when hallucinations occur at specific locations. The prepared dataset is then fed into the large model for efficient LoRA fine-tuning training. The parameter adjustment method and dataset preparation process are simple but can significantly improve the factual output performance of the large model.

[0085] Step C specifically includes the following steps:

[0086] Step C1: The self-distillation of the front and back layers of the large model at the fact level is based on the real sample D. true Make fine adjustments to enhance the consistency of the generation trends of factual answers across different layers;

[0087] Step C1 specifically includes the following steps:

[0088] Step C101: The large model is derived from real samples D true The data in the middle is activated, and the internal representations of the earlier and last layers are extracted;

[0089] Step C102: Using the output of the last layer of the large model as supervision, the internal representations extracted from previous layers are used for prediction, and the KL divergence formula is used as its loss function. Specifically: Used to measure the difference between the probability distribution of the output of the previous layer and the output of the last layer.

[0090] Step C103: Set N to the total number of samples, i.e., D true The number of factual answers; calculation and These are the probability distributions from the last layer and the kth layer, respectively;

[0091] Step C104: To ensure consistency of features between preceding and following layers, a mean squared loss function is introduced. in, and These are the outputs of the hidden layer.

[0092] Step C2: Combining the two loss functions mentioned above, start self-distillation, allowing the larger model to learn the factual output of the last layer from the earlier layers;

[0093] Step C2 specifically includes the following steps:

[0094] Step C201: Add the loss functions obtained in the previous two steps to calculate the comprehensive loss function, as shown in the formula below L. Combined =α·L Layer-wise +β·L KL , where α and β are the weights of the mean squared loss function and the fact consistency loss function, respectively, and can be changed according to the specific large model and the specific task;

[0095] Step C202: Train the large model using the comprehensive loss function to obtain a large model that maintains factual consistency between the front and back layers;

[0096] like Figure 5 As shown, in this embodiment, the positive fact self-distillation stage involves learning the internal representations of the large model. This requires developers to set a loss function for self-distillation. Specifically, the large model is activated using fact-answering pairs, and its representations are extracted from the outputs of the early and last layers. Developers set two loss functions: one is a fact consistency function, with the last layer as supervision and the early layers for prediction; the other is a feature mean squared loss function to make the output features converge. The large model is then trained using these two loss functions and the weight hyperparameters set by the developers, resulting in a large model with positive fact self-distillation.

[0097] Step D specifically includes the following steps:

[0098] Step D1: The fine-grained fact extraction method first processes the dataset to be used, labels the fine-grained hallucination answers, activates the large model by combining the question and hallucination answer pairs, and extracts their internal representations.

[0099] Step D1 specifically includes the following steps:

[0100] Step D101: The dataset is manually reviewed again and the hallucination responses are further subdivided into fine-grained categories based on the type of hallucination: entity hallucinations, relational hallucinations, sentence hallucinations, and unverifiable hallucinations.

[0101] Step D102: Use the partitioned dataset for activation of the large model for scheme selection, and extract internal representations at each layer of the large model;

[0102] Step D103: Map the internal representation to the fact space and semantic space respectively through the fact encoder and semantic encoder to avoid interference from semantic information during orientation extraction;

[0103] Step D2: Modify the contrastive learning loss function to maximize the difference between facts and fine-grained illusions, while avoiding excessive convergence between different types of illusions;

[0104] Step D2 specifically includes the following steps:

[0105] Step D201: The scheme maximizes the difference between fact and fine-grained illusion by maximizing the following formula The difference is calculated using sim similarity, X. + It is a set of fact samples. It is the sample set of the i-th type of hallucination;

[0106] Step D202: To prevent different types of hallucinations from converging excessively, the following function is used. Different types of hallucinations push each other away;

[0107] Step D203: Combining the two loss functions above, the overall fact loss function is L. truth =L true-hall +λL hall-hall , where λ is a hyperparameter, minimizing this function lays the foundation for subsequent extraction directions;

[0108] Step D204: Through the above contrastive learning process and the set loss function, the training effectively distinguishes between hallucination types and facts in the fact space. This step calculates the average value of the same hallucination type in the space and subtracts the average values ​​to obtain the δ values ​​corresponding to different hallucination types. i Fine-grained fact extraction is complete;

[0109] like Figure 6As shown, in this embodiment, the fine-grained fact direction extraction stage requires developers to perform additional processing on the dataset containing illusions, activating the large model to extract internal representations in the fact space through an autoencoder. Specifically, in this stage, developers perform fine-grained labeling on the answers in the illusion dataset, including entity-type illusions, semantically contradictory illusions, whole-sentence illusions, and unverifiable illusions. These are then fed into the large model for activation. An autoencoder is used in the intermediate layer to map tokens to the fact space and semantic space, avoiding interference from semantic information. A contrastive learning loss function designed by the developers is then used, including calculating the distance between facts and illusions in the fact space to prevent illusions from becoming too close. After training, the distribution of fine-grained illusions and facts in the fact space is obtained, and the corresponding fact editing direction is calculated for the next editing step.

[0110] Step E specifically includes the following steps:

[0111] Step E1: Using the large model that has been fine-tuned bidirectionally as described above, the large model responds to the user's input question Q, extracts internal representations at the same layer, and determines the type of potential hallucination.

[0112] Step E1 specifically includes the following steps:

[0113] Step E101: The user inputs question Q to activate the large model. After extracting the internal representation, the possible hallucination types or facts are simply determined by the clustering algorithm. If it is a specific hallucination type i, proceed to step E102. If it is a fact type, end step E without further editing.

[0114] Step E102: Add the corresponding δ based on the identified type of hallucination. i The specific incremental calculation formula is Δ=Dec(h sem +Attn(h sem ,h truth +δ))-Dec(h sem +Attn(h sem ,h truth -δ));

[0115] Step E2: Add the calculated incremental formula back into the internal representation, specifically the formula x. ' = x + α × Δ, where α is the incremental weight of directional editing, and x ' Continue with normal restoration within the larger model.

[0116] like Figure 7As shown, in this embodiment, the fine-grained fact-direction editing stage during inference requires the participation of both the user and the finely tuned large model. Specifically, the user inputs relevant questions in the field of low-carbon environmental protection into the large model, which responds internally. The solution of this invention extracts and encodes the intermediate layer, mapping it to the fact space and semantic space, similar to the previous step. In the fact space, the internal representation is judged for potential illusion types and a direction mapped to the facts is added. Then, a decoder is used to restore it to the large model, continuing the large model generation process. Through multiple fine-grained fact-direction editing optimizations, the generated fact content is generated.

[0117] The embodiments described above merely illustrate specific implementation methods of this application, and while the descriptions are detailed and specific, they should not be construed as limiting the scope of protection of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the technical solution of this application, and these modifications and improvements all fall within the scope of protection of this application.

Claims

1. A method for large model generation content information security reinforcement, characterized in that: The application is applied to a system for generating content information security reinforcement for large models, comprising a bidirectional fine-tuning module, a fine-grained fact direction extraction module, and a reasoning fact direction editing module. The bidirectional fine-tuning module is unidirectionally connected to the fine-grained fact direction extraction module, and the fine-grained fact direction extraction module is unidirectionally connected to the reasoning fact direction editing module. The bidirectional fine-tuning module is used for keyword extraction, comprising fine-tuning training and self-distillation enhancement of fact output on keyword extraction samples. The fine-grained fact direction extraction module comprises fine-grained fact direction extraction for pre-processing of the fine-tuned model, which is a fine-grained division of hallucinations in the large model answers, and extraction of hidden layer representation mapping to the fact space. The reasoning fact direction editing module differentially edits the reasoning process and corrects the factual expression. The method comprises the following steps: Step A: classify the large model answers in the data set to construct corresponding hallucination answer data sets and fact answer data sets; Step B: LoRA fine-tuning on the hallucination level for data from hallucination answers to learn useful knowledge for factual expression from hallucination samples; Step C: for the inconsistent attention of the large model to the fact information in the previous layer and the mature layer, use the self-distillation method to stimulate the large model with the fact answer data set, and let the large model distill the previous and later layers, and the fact representations of the previous and later layers tend to be consistent; Step D: using the trained large model, more fine-grained classification is performed on the hallucination samples to stimulate the internal representation of the large model and extract the corresponding fact direction; Specifically, the data set is further checked by artificial and fine-grained divided into entity hallucination, relationship hallucination, sentence hallucination and unverifiable hallucination according to the hallucination type; The divided data set is used to activate the large model to extract internal representation at each layer of the large model; The internal representation is mapped to the fact space and the semantic space through the fact encoder and the semantic encoder respectively to avoid the interference of semantic information during direction extraction; Step E: clustering the fine-grained fact direction according to the representation of the hidden layer in the reasoning process and mapping it to different directions in the fact space. Maximize the difference between facts and fine-grained hallucinations by maximizing the following equation where the difference is computed using the similarity sim, X + is the set of fact samples, is the set of hallucination samples of the i-th class; avoid different hallucination classes to be too close to each other by using the following function, different hallucination classes to push away from each other; L true-hall and L hall-hall , the overall fact loss function is L truth = L true-hall + λL hall-hall , where λ is a hyperparameter, minimizing this function is the basis for subsequent extraction direction; Through the comparison learning process and the set loss function, the hallucination categories and the facts are effectively distinguished in the fact space after training, and the average amount of the same hallucination type in the calculation space is calculated, and the difference is obtained δ corresponding to different hallucination categories i , the fine-grained fact direction extraction is completed; Step A specifically comprises the following steps:

2. The method for large model oriented content information security reinforcement generation according to claim 1, characterized in that: Step A1: ask x through the basic large model, arrange the answer y of the large model corresponding to the question, and obtain the data set two-tuple (x, y); Step B specifically comprises the following steps: Step A2: Artificially classify y in the binary dataset and add corresponding labels to the hallucinated answers, dividing into D = {D true ,D hall} real samples and hallucinated samples.

3. The method for large model-oriented content information security reinforcement generation according to claim 2, characterized in that: Step B1: arrange the hallucination sample into an instruction fine-tuning data set, and use the LoRA module to fine-tune it in a negative direction to learn the factual knowledge contained in the hallucination sample or exclude the hallucination content; Step B2: learn from the hallucination fine-tuning to obtain the incremental parameter θ of LoRA, which is used for subsequent addition to the overall network parameter. Step C specifically comprises:

4. The method for large model oriented content information security reinforcement generation according to any one of claims 1-3, characterized in that: Step C2: combine the two loss functions, KL divergence loss function and mean square loss function, and start self-distillation to let the large model learn the factual output of the last layer. Step C1: Pre-post layer self-distillation of large model on factual level is to real sample D true Fine-tuning, strengthening the consistency of pre-post layer generation trends for factual answers; Step C1 specifically comprises the following steps:

5. The method for large model oriented content information security reinforcement generation according to claim 4, characterized in that: Step C2 specifically comprises the following steps: Step C101: The large model is activated by data from real samples D true and extracts internal representations of the previous layers and the last layer. Step C102: With the last layer output of the large model as supervision, the internal representation extracted at the previous level is used as prediction, and the KL divergence formula is used as the factual consistency loss function, which is to measure the difference between the probability distribution of the output of the previous level and the output of the last layer. Step C103: Set N as the total number of samples, i.e. D true Number of facts answers; compute and are the probability distributions from the last layer and the k-th layer, respectively; Step C104: Introduce mean square loss function for consistency of features of front and rear layers wherein, and are the outputs of the hidden layers, respectively.

6. The method for large model oriented content information security reinforcement generation according to claim 5, characterized in that: Step C202: use the comprehensive loss function to train the large model to obtain the large model with consistent facts in the previous and later layers. Step C201: The comprehensive loss function is calculated by adding the loss functions obtained in step C1, and the formula is as follows L Combined = a L Layer-wise + β L KL , wherein a and β are the weights of the mean square loss function and the fact consistency loss function, respectively. ​ 7. The method for large model oriented content information security reinforcement generation according to claim 1, characterized in that: Step D specifically comprises the following steps: Step D1: The fine-grained fact direction extraction method first processes the data set to be used, labels the fine-grained illusion answers, activates the big model for the question and illusion answer binary tuple, and extracts its internal representation; Step D2: Modify the contrastive learning loss function to maximize the difference between facts and fine-grained illusions, while avoiding excessive convergence of different illusion categories.

8. The method for large model-oriented content information security reinforcement generation according to claim 1, characterized in that: Step E specifically comprises the following steps: Step E1: Use the already bidirectionally fine-tuned large model to respond to user input question Q, extract internal representation at the same layer, and judge the potential illusion category; Step E101: After activating the large model by inputting the question Q from the user and extracting the internal representation, the possible illusion category or fact is simply distinguished by a clustering algorithm. If it is a specific illusion category i, it goes to step E102. If it is a fact type, it ends step E and does not perform additional editing; Step E102: Add the corresponding delta to the hallucination category according to the judgment i , and the specific incremental calculation formula is Δ = Dec(h sem + Attn(h sem , h truth + delta))-Dec(h sem + Attn(h sem , h truth -delta) ; Step E2: The computed delta formula is added back into the internal representation, specifically formula x ' = x + a x delta, where a is the delta weight for the direction edit, which updates x ' Continues normal reduction within the large model.

Images