Large model generation content information security enhancement-oriented system and method

Abstract

The invention discloses a system and a method for security enhancement of generated content information of a large model, and the method comprises the steps: firstly, carrying out the bidirectional LoRA fine tuning technology and knowledge distillation technology on the large model, and enabling the large model to adapt to a factual output task from the illusion dimension and the factual dimension; performing extraction of intermediate layer representation in a fact space on the fine-tuned large model, and obtaining different mappings from fine-grained illusion types to a fact direction through a comparative learning method; and finally, in the reasoning generation process of the large model, extracting the representation of the middle layer of the large model to judge the potential illusion type, then applying the learned mapping corresponding to different illusions to the middle representation of the reasoning process, correcting factual representation, reducing the illusion output probability, and in the pre-training stage and the reasoning generation stage, obtaining the illusion type of the large model. Parameters and generated contents of the large model are controlled, and the factuality and reliability of the generated contents of the large model are greatly improved while the generation capability of the large model is guaranteed.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and specifically provides a system and method for enhancing the security of content information generated by a large model. Background Art

[0002] Large Language Model (LLM) is a class of models based on deep learning, which usually use neural networks (especially variants such as Transformer models) to process and generate natural language. They learn the structure, patterns and contextual relationships of language by training on a large amount of text data, so that they can perform a variety of language-related tasks such as text generation, question answering, translation, dialogue systems, etc. LoRA fine-tuning (Low-Rank Adaptation) is a technique that decomposes the parameters of a large model into a low-rank matrix and fine-tunes on this basis. Its purpose is to maintain the performance of the model while reducing the training time and the number of parameters, and is particularly suitable for fine-tuning specific tasks in resource-constrained environments. Knowledge distillation is a process of transferring the knowledge of a large model to a smaller model. It uses the predictions of the large model as a supervisory signal to help the smaller model learn the most important features of the task, thereby achieving model compression without significantly reducing the model's performance. Direction editing technology is used in large models to change the specific direction of the model to generate text or images. By adjusting the potential representation of the model, researchers can guide the model to generate output that conforms to a specific style or attribute, so it has been widely used in generative tasks to adjust the output content of the large model. Contrastive learning is an unsupervised learning method that learns representations by constructing positive and negative sample pairs. In the field of large models, contrastive learning can be used to divide positive and negative sample pairs so that positive and negative samples are pushed away from each other, distinguish representations, and improve the performance of large models in classification tasks.

[0003] Through the use of the existing large model for contract text generation, it is found that the existing large model uses the LoRA fine-tuning scheme to adapt the downstream contract reading comprehension, keyword extraction, and report generation scheme on the basic large model. During the operation, the generated content will produce a large number of hallucination outputs, including entity extraction hallucinations, relationship hallucinations, etc., resulting in the inability to use the generated text report correctly, requiring manual review again, increasing labor costs. In addition, in the current actual application scenarios, many large model systems adapted to downstream text generation tasks use a single LoRA positive fine-tuning to improve the factual output capability, but the effect is not obvious; at the same time, the current large model uses more editing methods in reasoning to adjust the latent representation, editing the direction from hallucination to fact; but past studies have shown that large model hallucinations are of various types, binary hallucination classification has insufficient representation capabilities, and the latent representation editing is not significantly improved. Summary of the invention

[0004] In view of the above-mentioned problems existing in the prior art of contract text generation big model, the present invention provides a system and method for strengthening the security of content information generated by big model based on big model, LoRA fine-tuning technology, contrastive learning, knowledge distillation and representation editing technology. The LoRA fine-tuning technology in the scheme can perform more efficient bidirectional fine-tuning on the basic big model in a low-resource environment, reduce resource waste from the perspective of hallucination samples, and use positive factual samples through knowledge distillation technology to improve the factual output capability of the big model in the front and back layers; the improved representation editing technology, from the perspective of the whole process, can perform potential representation editing in reasoning on the basic big model in a finer-grained direction, and optimize the direction from different types of hallucinations to facts.

[0005] The technical solution adopted by the present invention is as follows:

[0006] A system and method for enhancing the security of content information generated by a large model, a system for enhancing the security of content information generated by a large model, including a bidirectional fine-tuning module, a fine-grained fact direction extraction module, and a fact direction editing module during reasoning;

[0007] The bidirectional fine-tuning module is unidirectionally connected to the fine-grained fact direction extraction module, and the fine-grained fact direction extraction module is unidirectionally connected to the reasoning fact direction editing module;

[0008] The bidirectional fine-tuning module is used for keyword extraction, including fine-tuning training of keyword extraction samples and self-distillation to enhance factual output;

[0009] The fine-grained fact direction extraction module includes a fine-grained fact direction extraction for preprocessing the fine-tuned model, wherein the fine-grained fact direction is a fine-grained division of the hallucinations in the large model answer, and the hidden layer representation is extracted and mapped to the fact space;

[0010] The fact direction editing module during reasoning performs differential editing on the reasoning process and corrects its factual expression.

[0011] A method for enhancing the security of content information generated by a large model includes a function of fine-tuning the factuality of internal parameters of the large model and a function of editing facts during reasoning. The function of fine-tuning the factuality of internal parameters of the large model includes the following steps:

[0012] Step A: Classify the answers of the large model in the data set and construct the corresponding hallucination answer data set and fact answer data set;

[0013] Step B: Fine-tune LoRA at the hallucination level for data from hallucinated answers, and learn useful knowledge for factual expression from hallucinated samples;

[0014] Step C: In view of the inconsistent attention paid by the big model to factual information compared with the previous and mature layers, a self-distillation method is used to stimulate the big model using the factual data set, and the big model is allowed to perform distillation of the front and back layers, so that the fact representations of the front and back layers tend to be consistent.

[0015] The fact editing function at inference time includes the following steps:

[0016] Step D: Use the existing training large model to perform more fine-grained classification of hallucination samples, distinguish entity hallucinations, relationship hallucinations, and unverifiable hallucinations, stimulate the internal representation of the large model, and extract the fact direction of the corresponding category;

[0017] Step E: The fact editor during reasoning maps the fine-grained fact directions to different directions in the fact space according to the representation clusters of the hidden layer during the reasoning process.

[0018] Preferably, step A specifically comprises the following steps:

[0019] Step A1: Ask a question x through the basic large model, sort out the answers y of the large model to the corresponding questions, and obtain the data set binary (x, y);

[0020] Step A2: Manually classify the y in the binary data set and add corresponding labels to the hallucinated answers y, dividing them into D = {D true , D hall}Real samples and hallucinated samples;

[0021] Preferably, step B specifically comprises the following steps:

[0022] Step B1: Organize the hallucination samples (x, y) into an instruction fine-tuning dataset, and use the LoRA module to negatively fine-tune it to learn the factual knowledge contained in the hallucination samples or exclude the hallucination content;

[0023] Step B2: Learn from the hallucination fine-tuning to obtain the incremental parameters θ of LoRA for subsequent addition to the overall network parameters;

[0024] Preferably, step C specifically comprises the following steps:

[0025] Step C1: The self-distillation of the front and back layers of the large model at the fact level is to extract the factual sample D true Fine-tune to strengthen the consistency of the generation trends of the previous and next layers for factual answers;

[0026] Preferably, step C1 specifically comprises the following steps:

[0027] Step C101: The large model is composed of real samples D true Activate the data in the network and extract the internal representations of the earlier and last layers;

[0028] Step C102: Take the output of the last layer of the large model as supervision, and the internal representation extracted from the previous layer as prediction. Use the KL divergence formula as its loss function, which is: It is used to measure the difference between the probability distribution of the output of the previous layer and the output of the last layer.

[0029] Step C103: Set N to the total number of samples, i.e. D true The number of factual answers in ; count and are the probability distributions from the last layer and the kth layer respectively;

[0030] Step C104: Introduce the mean square loss function for consistency of front and back layer features in, and are the outputs of the hidden layers respectively.

[0031] Step C2: Combining the above two loss functions, start self-distillation to let the large model learn the factual output of the last layer from the previous layer;

[0032] Preferably, step C2 specifically comprises the following steps:

[0033] Step C201: The loss functions obtained in the previous two steps are added together and calculated as follows: Combined =α·L Layer-wise +β·L KL , where α and β are the weights of the mean square loss function and the fact consistency loss function, respectively, which can be changed according to the specific large model and specific task;

[0034] Step C202: Use a comprehensive loss function to train the large model to obtain a large model in which the front and back layers maintain fact consistency;

[0035] Preferably, step D specifically comprises the following steps:

[0036] Step D1: The fine-grained fact extraction method first processes the dataset to be used, marks the fine-grained hallucination answers, activates the large model with the question and hallucination answer binary, and extracts its internal representation;

[0037] Preferably, step D1 specifically includes the following steps:

[0038] Step D101: The data set is manually checked again and the hallucination answers are finely divided into entity hallucinations, relationship hallucinations, sentence hallucinations and unverifiable hallucinations according to the types of hallucinations;

[0039] Step D102: Use the divided data set to activate the large model for scheme selection, and extract internal representations at each layer of the large model;

[0040] Step D103: Map the internal representation to the fact space and the semantic space respectively through the fact encoder and the semantic encoder to avoid interference of semantic information during direction extraction;

[0041] Step D2: Modify the contrastive learning loss function to maximize the difference between the truth and the fine-grained hallucinations while avoiding the different hallucination types from being too close;

[0042] Preferably, step D2 specifically comprises the following steps:

[0043] Step D201: The scheme maximizes the difference between the truth and the fine-grained hallucination by maximizing the following formula The difference is calculated using sim similarity, X + is a set of fact samples, is the i-th type of hallucination sample set;

[0044] Step D202: The method of avoiding different types of hallucinations from being too close together is implemented by the following function: Different kinds of hallucination push away from each other;

[0045] Step D203: Combining the above two loss functions, the overall fact loss function is L truth =L true-hall +λL hall-hall , where λ is a hyperparameter, minimizing this function is the basis for subsequent extraction directions;

[0046] Step D204: Through the above-mentioned comparative learning process and the set loss function, the types of hallucinations and facts are effectively distinguished in the fact space after training. This step calculates the average amount of the same type of hallucination in the space, and subtracts it to obtain the δ corresponding to different types of hallucinations. i ,Fine-grained editing direction extraction is completed;

[0047] Preferably, step E specifically comprises the following steps:

[0048] Step E1: Using the bidirectionally fine-tuned large model, the user inputs question Q, the large model responds, extracts internal representations at the same layer, and determines the potential hallucination type;

[0049] Preferably, step E1 specifically includes the following steps:

[0050] Step E101: The user inputs question Q to activate the large model. After extracting the internal representation, the clustering algorithm is used to simply identify the possible hallucination type or fact. If it is a specific hallucination type i, step E102 is entered. If it is a fact type, step E is terminated without additional editing.

[0051] Step E102: Add the corresponding delta according to the determined type of hallucination i The specific increment calculation formula is Δ=Dec(h sem +Attn(h sem ,h truth +δ))-Dec(h sem +Attn(h sem ,h truth -δ));

[0052] Step E2: Add the calculated incremental formula back to the internal representation. The specific formula is x′=x+α×Δ, where α is the incremental weight of the directional editing. x′ is then restored normally inside the large model.

[0053] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0054] 1. The present invention provides a system and method for strengthening the security of content information generated by a large model. Through the coordinated operation of the large model bidirectional fine-tuning module, the fine-grained fact direction extraction module, and the fine-grained fact direction editing module during reasoning, the contract text extraction function based on the text generation large model can more effectively generate factual content and comprehensively consider avoiding the generation of hallucination information. With the help of the bidirectional fine-tuning module and self-distillation technology, while effectively using hallucination samples for training, the internal parameters of the large model are also strengthened so that the earlier layers can also learn the degree of attention paid to factual content by the mature layer, and maintain consistency.

[0055] 2. The present invention solves the problems that hallucination samples generated by large models are wasted, and hallucinations and facts are considered as two binary opposing labels in the past. In our solution, hallucinations are divided into fine-grained parts for learning the editing directions of different types of hallucinations and facts in the fact space, thereby optimizing the content generated by the large model to move closer to the fact direction at the knowledge editing level; specifically, the directions from different types of hallucinations to facts are learned during the training process, and in the subsequent reasoning process, clustering algorithms are used to determine potential hallucination types and then their corresponding directions are added to the original intrinsic representation, and finally the edited output is obtained. The hallucination samples generated by large models were previously considered to have no value. In our solution, the hallucination samples are reused, first labeled with hallucination types and then fine-tuned. The effect of fine-tuning in this way is better than fine-tuning with only factual samples, which improves training efficiency.

[0056] 3. The present invention includes two main stages: the pre-application and mid-application stages of the large model for text generation. Before application, the parameters of the large model are fine-tuned using the pre-training method to adapt it to the downstream contract keyword extraction task, while learning to output factual content from the two dimensions of hallucination and fact; in application, the method of fine-grained knowledge editing is used to correct the hallucination answers that may have appeared. The present invention mainly includes four functions: first, the use of hallucination information. Unlike the previous single use of facts or negative labels, the use of hallucination information in this scheme can be designed to collect hallucination data sets from real user feedback after deployment and operation. The hallucination fine-tuning of this scheme can realize the optimization iteration of the large model; second, the problem of optimizing the potential representation of the front and back layers of the large model, using self-distillation to modify the parameters of the front layer of the large model, and realizing early attention to factual information; the fine-grained fact direction extraction function is an operation performed on the fine-tuned large model. By activating the large model and learning the fine-grained direction from the internal representation, it is a study of the parameters of the model itself; the last function is to use the fact editing direction of the selected model in reasoning, which can effectively correct the answers that may be hallucinated and enhance the fact output capability. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is a schematic diagram of the system structure of the present invention;

[0058] Figure 2 is a flow chart of the method of the present invention;

[0059] Figure 3 This is the activity diagram of the data set preparation phase in Example 2;

[0060] Figure 4 This is the activity diagram of the negative hallucination fine-tuning stage in Example 2;

[0061] Figure 5 This is the activity diagram of the positive fact self-distillation stage in Example 2;

[0062] Figure 6 This is the activity diagram of the fine-grained fact direction extraction stage in this embodiment 2;

[0063] Figure 7 This is an activity diagram of the fact direction editing stage during reasoning in Example 2. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0065] Example 1

[0066] like Figure 1 As shown, the present invention provides a system for enhancing the security of content information generated by a large model, including a bidirectional fine-tuning module, a fine-grained fact direction extraction module, and a fact direction editing module during reasoning; the bidirectional fine-tuning module is used in the contract text keyword extraction task downstream of the basic large model adaptation, and the adaptation task includes fine-tuning training and self-distillation of keyword extraction sample instances to enhance factual output and weaken hallucination performance; the fine-grained fact direction extraction module includes fine-grained fact direction extraction for preprocessing the model obtained by fine-tuning in the previous step, and the fine-grained fact direction is a fine-grained division of the hallucinations in the large model answer, extracting hidden layer representations and mapping them to facts space; the fact direction editing module during reasoning is responsible for differential editing of the reasoning process. The differential editing scheme specifically represents the hidden layer features in the clustering reasoning process and then performs fine-grained fact direction editing to correct its factual expression; in Example 1, we assume that the application scenario is keyword extraction of asset contracts to generate a report that briefly summarizes the amount, time, parties to the contract and other information; considering that the initial input of this scenario is a detailed but overly lengthy contract document, the pre-output is a brief report containing key information after learning and organizing the large text model. In this scenario, the most important demand is that the generated report does not have hallucinations, so it is an excellent implementation scenario for the invention.

[0067] In Example 1, the present invention includes two main stages in the system invention: two stages before and during the application of a large model for text generation. Before application, a pre-training method is used to fine-tune the parameters of the large model to adapt it to the downstream contract keyword extraction task, while learning to output factual content from two dimensions: hallucination and fact; in application, a fine-grained knowledge editing method is used to correct hallucination answers that may have appeared. The present invention mainly includes four functions: in order to ensure the real-time update of the subsequent large model and avoid hallucinations, the first is the use of hallucination information. Unlike the previous single use of facts or negative labels, the use of hallucination information in this solution can be designed to collect hallucination data sets from real user feedback after deployment and operation. The hallucination fine-tuning of this solution can realize the optimization iteration of the large model; the second is to optimize the problem of potential representation of the front and back layers of the large model, and use self-distillation to modify the parameters of the front layer of the large model to achieve early attention to factual information in the contract report generation task; the fine-grained fact direction extraction function is an operation performed on the fine-tuned large model. The large model is activated through the full-fact contract report and the contract report that protects the hallucination and learns the fine-grained direction from the internal representation. It is a study of the model's own parameters and the adaptation task; the last function is to use the fact editing direction of the selected model for reasoning, which can effectively correct the answers that may be hallucinated and enhance the fact output capability.

[0068] Example 2

[0069] like Figure 2 and Figure 3 As shown, the present invention also provides a method for enhancing the security of content information generated by a large model, which is also applicable to knowledge question-and-answer tasks in vertical fields. The preset scenario of Example 2 is the field of asset contract management. The specific task form is a dialogue with memory. The specific tasks include interpretation of asset contract documents, analysis of common disputes, and questions and answers about asset contract-related knowledge. The initial input is a simple question from the user to the large model, and the pre-output is the corresponding answer generated by the large model. In this scenario, the large model is applied to vertical fields, involving a narrow breadth of content but high precision requirements, so it is an excellent implementation case of the invention. The following steps are included for this Example 2:

[0070] Step A: Classify the answers of the large model in the data set and construct the corresponding hallucination answer data set and fact answer data set;

[0071] Step B: Fine-tune LoRA at the hallucination level for data from hallucinated answers, and learn useful knowledge for factual expression from hallucinated samples;

[0072] Step C: In view of the inconsistent attention paid by the big model to factual information compared with the previous and mature layers, a self-distillation method is used to stimulate the big model using the factual data set, and the big model is allowed to perform distillation of the front and back layers, so that the fact representations of the front and back layers tend to be consistent.

[0073] The fact editing function at inference time includes the following steps:

[0074] Step D: Use the existing training large model to perform more fine-grained classification of hallucination samples, distinguish entity hallucinations, relationship hallucinations, and unverifiable hallucinations, stimulate the internal representation of the large model, and extract the fact direction of the corresponding category;

[0075] Step E: The fact editor during reasoning maps the fine-grained fact directions to different directions in the fact space according to the representation clusters of the hidden layer during the reasoning process.

[0076] like Figure 2As shown, in this embodiment, the whole process of the hallucination reduction scheme of the text generation large model is included, including the data set preparation stage, the negative hallucination fine-tuning stage, the positive fact self-distillation stage, the fine-grained fact direction extraction stage, and the fine-grained fact direction editing stage during reasoning. In the data set preparation stage, this embodiment constructs a data set through an existing open source low-carbon environmental protection question-and-answer data set or by asking questions to the large model and then collecting hallucinations and fact answers for subsequent fine-tuning and editing direction extraction. The negative hallucination fine-tuning stage specifically uses the data set containing hallucination answers obtained in the previous stage to learn the hallucination samples through the LoRA fine-tuning method to obtain factual related knowledge. The positive fact self-distillation stage is an operation on the front and back layers of the large model. The representation of the last layer when the fact is output is used as a guide to optimize the representation of the front and back layers of the large model. The purpose is to enhance the consistent attention of each layer of the large model to the fact information. After this stage, the bidirectional fine-tuning stage is completed. Fine-grained fact direction extraction operates on the large model that has been fine-tuned in the previous step, and extracts the direction from each hallucination type to the fact from the hidden layer of the large model. During reasoning, the potential hallucinations in the middle layer of the large model are added with the corresponding fact directions extracted in the previous step. With this method, the hallucination problem can be corrected in time when used.

[0077] Step A: Constructing the data set for the implementation case includes the following steps:

[0078] Step A1: Find the initial asset contract field question-answering dataset, collect its questions x for the basic large model, sort out the answers y of the large model to the corresponding questions, and obtain the dataset binary (x, y);

[0079] Step A2: Manually classify the y in the binary data set and add corresponding labels to the hallucinated answers y, dividing them into D = {D true , D hall}Real samples and hallucinated samples;

[0080] like Figure 3 As shown, in this embodiment, the data set preparation stage: the data set preparation stage requires the participation of developers and the selection of basic large models. Developers obtain answers by asking questions to the basic large model, and then judge whether the answers are illusions or facts and reclassify them. Specifically, the developer first searches for low-carbon environmental protection related information, and after sorting, asks the large model multiple rounds of questions, collects answers and judges whether the answers contain illusions. After the questioning, the sorted data set is divided into a negative illusion data set and a positive fact data set.

[0081] Then proceed to step B, which specifically includes the following contents:

[0082] Step B1: Organize the hallucination samples (x, y) into an instruction fine-tuning dataset, and use the LoRA module to negatively fine-tune it to learn the factual knowledge contained in the hallucination samples or exclude the hallucination content;

[0083] Step B2: Learn from the hallucination fine-tuning to obtain the incremental parameters θ of LoRA for subsequent addition to the overall network parameters;

[0084] like Figure 4 As shown, in this embodiment, the negative hallucination fine-tuning stage: the negative hallucination fine-tuning stage is to operate the hallucination sample data set in the previous step and learn knowledge from the hallucination samples; specifically: the hallucination data set samples obtained in the previous step are additionally labeled, local labels are added to the answers, hallucinations are annotated at specific locations, and the prepared data set is then sent to the large model for LoRA efficient fine-tuning training. The parameter adjustment method and the data set preparation process are simple but can significantly improve the factual output performance of the large model.

[0085] The step C specifically comprises the following steps:

[0086] Step C1: The self-distillation of the front and back layers of the large model at the fact level is to extract the factual sample D true Fine-tune to strengthen the consistency of the generation trends of the previous and next layers for factual answers;

[0087] The step C1 specifically comprises the following steps:

[0088] Step C101: The large model is composed of real samples D true Activate the data in the network and extract the internal representations of the earlier and last layers;

[0089] Step C102: Take the output of the last layer of the large model as supervision, and the internal representation extracted from the previous layer as prediction. Use the KL divergence formula as its loss function, which is: It is used to measure the difference between the probability distribution of the output of the previous layer and the output of the last layer.

[0090] Step C103: Set N to the total number of samples, i.e. D true The number of factual answers in ; count and are the probability distributions from the last layer and the kth layer respectively;

[0091] Step C104: Introduce the mean square loss function for consistency of front and back layer features in, and are the outputs of the hidden layers respectively.

[0092] Step C2: Combining the above two loss functions, start self-distillation to let the large model learn the factual output of the last layer from the previous layer;

[0093] The step C2 specifically comprises the following steps:

[0094] Step C201: The loss functions obtained in the previous two steps are added together and calculated as follows: Combined =α·L Layer-wise +β·L KL , where α and β are the weights of the mean square loss function and the fact consistency loss function, respectively, which can be changed according to the specific large model and specific task;

[0095] Step C202: Use a comprehensive loss function to train the large model to obtain a large model in which the front and back layers maintain fact consistency;

[0096] like Figure 5 As shown, in this embodiment, the positive fact self-distillation stage: the positive fact self-distillation stage is to learn the internal representation of the large model, and the developer needs to set the loss function of its self-distillation. Specifically: use the fact question and answer pair to activate the large model, extract its representation at the output of the early layer and the last layer, and the developer sets two loss functions, one is the fact consistency function, with the last layer as supervision and the early layer for prediction; the other is the feature mean square loss function to make the output features converge. The large model begins to be trained using the above two loss functions and the weight hyperparameters set by the developer to obtain a large model of positive fact self-distillation.

[0097] The step D specifically comprises the following steps:

[0098] Step D1: The fine-grained fact extraction method first processes the dataset to be used, marks the fine-grained hallucination answers, activates the large model with the question and hallucination answer binary, and extracts its internal representation;

[0099] The step D1 specifically comprises the following steps:

[0100] Step D101: the data set is manually checked again and the hallucination answers are finely divided into entity hallucinations, relationship hallucinations, sentence hallucinations and unverifiable hallucinations according to the types of hallucinations;

[0101] Step D102: Use the divided data set to activate the large model for scheme selection, and extract internal representations at each layer of the large model;

[0102] Step D103: Map the internal representation to the fact space and the semantic space respectively through the fact encoder and the semantic encoder to avoid interference of semantic information during direction extraction;

[0103] Step D2: Modify the contrastive learning loss function to maximize the difference between the truth and the fine-grained hallucinations while avoiding the different hallucination types from being too close;

[0104] The step D2 specifically comprises the following steps:

[0105] Step D201: The scheme maximizes the difference between the truth and the fine-grained hallucination by maximizing the following formula The difference is calculated using sim similarity, X + is a set of fact samples, is the i-th type of hallucination sample set;

[0106] Step D202: The method of preventing different types of hallucinations from being too close together is implemented by the following function: Different kinds of hallucination push away from each other;

[0107] Step D203: Combining the above two loss functions, the overall fact loss function is L truth =L true-hall +λL hall-hall , where λ is a hyperparameter, minimizing this function is the basis for subsequent extraction directions;

[0108] Step D204: Through the above-mentioned comparative learning process and the set loss function, the types of hallucinations and facts are effectively distinguished in the fact space after training. This step calculates the average amount of the same hallucination type in the space, and subtracts it to obtain the δ corresponding to different hallucination types. i ,Fine-grained editing direction extraction is completed;

[0109] like Figure 6 As shown, in this embodiment, the fine-grained fact direction extraction stage: the fine-grained fact direction extraction stage requires developers to perform additional processing on the data set containing hallucinations again, and activate the large model to extract the internal representation in the fact space through the autoencoder; specifically: in this stage, the developer performs fine-grained marking on the answers in the hallucination data set, including entity hallucinations, semantic contradiction hallucinations, whole sentence hallucinations, and unverifiable hallucinations. Send it to the large model for activation, and use the autoencoder in the middle layer to map the token to the fact space and the semantic space to avoid interference from semantic information. Then use the contrastive learning loss function designed by the developer, including pushing the distance between facts and hallucinations in the fact space to avoid hallucinations being too close. After training, the distribution of fine-grained hallucinations and facts in the fact space is obtained, and the corresponding fact editing direction is calculated for the next step of editing.

[0110] The step E specifically comprises the following steps:

[0111] Step E1: Using the bidirectionally fine-tuned large model, the user inputs question Q, the large model responds, extracts internal representations at the same layer, and determines the potential hallucination type;

[0112] The step E1 specifically includes the following steps:

[0113] Step E101: The user inputs question Q to activate the large model. After extracting the internal representation, the clustering algorithm is used to simply identify the possible hallucination type or fact. If it is a specific hallucination type i, it proceeds to step E102. If it is a fact type, it ends step E without additional editing.

[0114] Step E102: Add the corresponding delta according to the determined type of hallucination i The specific increment calculation formula is Δ=Dec(h sem +Attn(h sem ,h truth +δ))-Dec(h sem +Attn(h sem ,h truth -δ));

[0115] Step E2: Add the calculated incremental formula back to the internal representation. The specific formula is x′=x+α×Δ, where α is the incremental weight of the directional editing. x′ is then restored normally inside the large model.

[0116] like Figure 7 As shown, in this embodiment, the fine-grained fact direction editing stage during reasoning: the fine-grained fact direction editing stage during reasoning requires the participation of users and the fine-tuned large model. Specifically: the user inputs relevant questions in the field of low-carbon environmental protection to the large model, and the large model responds internally. The solution of the present invention extracts and encodes its intermediate layer, and maps it to the fact space and semantic space similarly to the previous step. In the fact space, the internal representation is judged for the potential type of hallucination and the direction mapped to the fact is added to it, and then the decoder is used to restore it to the large model, and the large model generation process is continued. The fact content is generated through multiple fine-grained fact direction editing optimizations.

[0117] The above-mentioned embodiments only express the specific implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the protection scope of the present application. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the technical solution concept of the present application, and these all belong to the protection scope of the present application.

Claims

1. A system for enhancing the security of content information generated by a large model, characterized by: It includes a bidirectional fine-tuning module, a fine-grained fact direction extraction module, and a reasoning-time fact direction editing module; The bidirectional fine-tuning module is unidirectionally connected to the fine-grained fact direction extraction module, and the fine-grained fact direction extraction module is unidirectionally connected to the reasoning fact direction editing module; The bidirectional fine-tuning module is used for keyword extraction, including fine-tuning training of keyword extraction samples and self-distillation to enhance factual output; The fine-grained fact direction extraction module includes a fine-grained fact direction extraction for preprocessing the fine-tuned model, wherein the fine-grained fact direction is a fine-grained division of the hallucinations in the large model answer, and the hidden layer representation is extracted and mapped to the fact space; The fact direction editing module during reasoning performs differential editing on the reasoning process and corrects its factual expression.

2. A method for enhancing the security of content information generated by a large model based on claim 1, characterized in that: Includes the following step: Step A: Classify the answers of the large model in the data set and construct the corresponding hallucination answer data set and fact answer data set; Step B: Fine-tune LoRA at the hallucination level for data from hallucinated answers, and learn useful knowledge for factual expression from hallucinated samples; Step C: In view of the inconsistent attention paid by the big model to factual information compared with the front and mature layers, the self-distillation method is used to stimulate the big model with the factual data set, and the big model is allowed to perform distillation of the front and back layers, so that the fact representation of the front and back layers tends to be consistent; The fact editing function at inference time includes the following steps: Step D: Use the existing training large model to perform more fine-grained classification of hallucination samples, distinguish entity hallucinations, relationship hallucinations, and unverifiable hallucinations, stimulate the internal representation of the large model, and extract the fact direction of the corresponding category; Step E: The fact editor during reasoning maps the fine-grained fact directions to different directions in the fact space according to the representation clusters of the hidden layer during the reasoning process.

3. The method for enhancing the security of content information generated by a large model according to claim 2 is characterized in that: Step A specifically includes the following steps: Step A1: Ask a question x through the basic large model, sort out the answers y of the large model to the corresponding questions, and obtain the data set binary (x, y); Step A2: Manually classify the y in the binary data set and add corresponding labels to the hallucinated answers y, dividing them into D = {D true ,D hall }Real samples and hallucinated samples.

4. The method for enhancing security of content information generated by a large model according to claim 3 is characterized in that: Step B specifically includes the following steps: Step B1: organize the hallucination samples (x, y) into an instruction fine-tuning dataset, and use the LoRA module to negatively fine-tune it to learn the factual knowledge contained in the hallucination samples or exclude the hallucination content; Step B2: Learn from hallucination fine-tuning and obtain the incremental parameters θ of LoRA for subsequent addition to the overall network parameters.

5. A method for enhancing security of content information generated by a large model according to any one of claims 2 to 4, characterized in that: Step C specifically includes: Step C1: The self-distillation of the front and back layers of the large model at the fact level is to perform the fact sample D true Fine-tune to strengthen the consistency of the generation trends of the previous and next layers for factual answers; Step C2: Combining the above two loss functions, start self-distillation and let the large model learn the factual output of the last layer from the previous layer.

6. A method for enhancing security of content information generated by a large model according to claim 5, characterized in that: Step C1 specifically includes the following steps: Step C101: The large model is composed of real samples D true Activate the data in the network and extract the internal representations of the earlier and last layers; Step C102: Take the output of the last layer of the large model as supervision, and the internal representation extracted from the previous layer as prediction, and use the KL divergence formula as its loss function, which is specifically: Used to measure the difference between the probability distribution of the output of the previous layer and the output of the last layer; Step C103: Set N to the total number of samples, i.e. D true The number of factual answers in ; count and are the probability distributions from the last layer and the kth layer respectively; Step C104: Introduce the mean square loss function for the consistency of the front and back layer features in, and are the outputs of the hidden layers respectively.

7. A method for enhancing security of content information generated by a large model according to claim 6, characterized in that: Step C2 specifically includes the following steps: Step C201: The loss functions obtained in the previous two steps are added together and calculated as follows: Combined =α·L Layer-wise +β·L KL , where α and β are the weights of the mean square loss function and the fact consistency loss function, respectively, which can be changed according to the specific large model and specific task; Step C202: Use the comprehensive loss function to train the large model to obtain a large model in which the front and back layers maintain factual consistency.

8. The method for enhancing security of content information generated by a large model according to claim 2, characterized in that: Step D specifically includes the following steps: Step D1: The fine-grained fact extraction method first processes the dataset to be used, marks the fine-grained hallucination answers, activates the large model with the question and hallucination answer binary, and extracts its internal representation; Step D2: Modify the contrastive learning loss function to maximize the difference between the ground truth and fine-grained hallucinations while avoiding different hallucination types from being too close.

9. The method for enhancing security of content information generated by a large model according to claim 8, characterized in that: Step D1 specifically includes the following steps: Step D101: The data set is manually checked again and the hallucination answers are finely divided into entity hallucinations, relationship hallucinations, sentence hallucinations and unverifiable hallucinations according to the types of hallucinations; Step D102: Use the divided data set to activate the large model for scheme selection, and extract internal representations at each layer of the large model; Step D103: Map the internal representation to the fact space and the semantic space respectively through the fact encoder and the semantic encoder to avoid interference of semantic information during direction extraction; Step D2 specifically includes the following steps: Step D201: The scheme maximizes the difference between the truth and the fine-grained hallucination by maximizing the following formula The difference is calculated using sim similarity, X + is a set of fact samples, is the i-th type of hallucination sample set; Step D202: The method of avoiding different types of hallucinations from being too close together is implemented by the following function: Different kinds of hallucination push away from each other; Step D203: Combining the above two loss functions, the overall fact loss function is L truth =L true-hall +λL hall-hall , where λ is a hyperparameter, minimizing this function is the basis for subsequent extraction directions; Step D204: Through the above-mentioned comparative learning process and the set loss function, the types of hallucinations and facts are effectively distinguished in the fact space after training. This step calculates the average amount of the same type of hallucination in the space, and subtracts it to obtain the δ corresponding to different types of hallucinations. i ,Fine-grained editing direction extraction is completed.

10. The method for enhancing security of content information generated by a large model according to claim 2, characterized in that: Step E specifically includes the following steps: Step E1: Using the bidirectionally fine-tuned large model, the user inputs question Q, the large model responds, extracts internal representations at the same layer, and determines the potential hallucination type; Step E101: The user inputs question Q to activate the large model. After extracting the internal representation, the clustering algorithm is used to simply identify the possible hallucination type or fact. If it is a specific hallucination type i, step E102 is entered. If it is a fact type, step E is terminated without additional editing. Step E102: Add the corresponding delta according to the determined type of hallucination i The specific increment calculation formula is Δ=Dec(h sem +Attn(h sem ,h truth +δ))-Dec(h sem +Attn(h sem ,h truth -δ)); Step E2: Add the calculated incremental formula back to the internal representation. The specific formula is x′=x+α×Δ, where α is the incremental weight of the directional editing. x′ is restored normally inside the large model.

Images