Safe and credible federal learning method for large model illusion perception
By introducing an illusion constraint loss function and a differential privacy mechanism into federated learning, the problems of illusion in generated results and parameter leakage in large model training are solved, achieving a synergistic improvement in the reliability and privacy security of model generated results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HARBIN UNIV OF SCI & TECH
- Filing Date
- 2026-01-26
- Publication Date
- 2026-05-08
AI Technical Summary
Under the federated learning framework, large model training is prone to generating illusions due to data heterogeneity, and the interaction of model parameters can easily leak sensitive information. Existing methods are difficult to coordinate and balance the reliability of model generation and privacy security.
During local training on the client side, an illusion constraint mechanism is introduced. The model parameters are protected through an illusion constraint loss function and a differential privacy mechanism. The server performs credibility assessment and weighted aggregation to achieve constraints on the model generation results and secure protection of the parameters.
It effectively reduces the deviation between model-generated results and real, reliable information, prevents the leakage of sensitive information, and improves the reliability and credibility of federated large model training.
Smart Images
Figure CN121998129A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of federated learning technology, specifically to a secure and reliable federated learning method for large-scale illusion perception. Background Technology
[0002] With the rapid development of deep learning technology, large models, characterized by their massive parameter count and strong expressive power, have demonstrated significant advantages in tasks such as natural language processing and content generation. Meanwhile, federated learning, as a distributed collaborative training paradigm, provides an effective way to solve the data silo problem and protect data privacy by exchanging model parameters or gradient information among participants without directly sharing raw data. Therefore, combining large models with federated learning has become an important development direction for achieving multi-party collaborative modeling and privacy protection.
[0003] However, training large models within the federated learning framework still faces numerous technical challenges. On one hand, the data distribution among clients in a federated learning environment typically exhibits significant non-independent and identically distributed characteristics. Coupled with the complex structure and massive parameter scale of large models, this can easily lead to illusions during model generation—that is, the generated results deviate from truthful and credible information at the semantic or factual level, thus reducing the reliability and credibility of the model output. Existing methods primarily focus on illusion suppression in centralized training scenarios, making them difficult to directly apply to the distributed and heterogeneous data environments of federated learning. On the other hand, while federated learning avoids direct sharing of raw data, model parameters or gradients may still leak sensitive client information during multiple rounds of interaction, especially during the training of large models, where the amount of information carried by model parameters is even greater, further amplifying the risk of privacy leaks. Effectively protecting model parameters during the federated learning process while ensuring model performance and stability remains a critical issue that urgently needs to be addressed. Existing privacy protection methods often focus solely on privacy or model performance, lacking a synergistic consideration of the reliability and privacy of large model generation.
[0004] Therefore, suppressing the illusion generation behavior of large models during federated learning training and strictly protecting the privacy of model parameters to achieve secure and reliable federated training of large models has become a technical challenge in the current field. To address this, this invention proposes a secure and reliable federated learning method for large model illusion perception. Within the federated learning framework, this method introduces an illusion constraint mechanism during local training on the client side, constraining the model-generated results from both factual and fidelity dimensions, effectively reducing the deviation between the model-generated results and real, reliable information. Simultaneously, it combines a differential privacy mechanism to perturb and protect model parameters, preventing the leakage of sensitive client information during federated interactions, thereby improving the reliability and reliability of the federated large model training process while ensuring data privacy and security. Summary of the Invention
[0005] To address the technical challenges of large-scale model training in federated learning frameworks, such as the generation of illusions due to data heterogeneity, the leakage of sensitive information through model parameter interactions, and the difficulty of balancing model generation reliability and privacy security in existing methods, this invention provides a secure and reliable federated learning method for large-scale model illusion awareness, comprising:
[0006] S1, Configuration includes The federated learning system for the first client, the first Each client locally holds a private dataset. Each client shares the initial global large model parameters. ;
[0007] S2, Each client is based on the private dataset The large model is trained locally to obtain local model parameters. Constructing an illusion constraint loss function The illusion constraint loss function Used to measure the deviation between the results generated by a large model and reality;
[0008] S3, the illusion constraint loss function Introducing a local optimization objective for the local model parameters After optimization, a local training objective function is obtained, which is expressed as:
[0009]
[0010] in, This represents the task loss function. For global large model parameters, The weighting coefficient for hallucination suppression;
[0011] S4. Regarding the local model parameters Perform security protection processing to obtain local model parameters with security protection. And upload it to the server;
[0012] S5. The server-side protection of the local model parameters. Perform a credibility assessment and, based on the assessment results, execute a weighted federated aggregation to obtain the global model parameters:
[0013]
[0014] in, These are global model parameters. Represents the confidence weights of the client's local model parameters;
[0015] S6. After aggregating the global model parameters, distribute them to each client and repeat steps S2-S5 to obtain a trained large model with a low hallucination rate, thereby achieving hallucination suppression and security enhancement of the large model in a federated learning environment.
[0016] Furthermore, the illusion constraint loss function described in S2 Includes factual constraints based on the differences between model-generated results and credible factual information. And fidelity evaluation item based on consistency of multiple generation results The expression is:
[0017]
[0018] in, These are the weighting coefficients for consistency constraints.
[0019] Furthermore, the factual constraint term based on the difference between the model-generated results and credible factual information... The generation method is as follows:
[0020] Within the same round, for each input data ,repeat Multiple outputs are generated through a forward propagation, but only the first generation result of the large model is taken, and then the differences are calculated:
[0021]
[0022] in, This represents the initial generation result of the large model; This indicates the corresponding credible factual information obtained from the input manually labeled data; It is a cosine similarity function used to measure the difference between the generated result and the factual information; Indicates to the client All samples The expectation.
[0023] Furthermore, the fidelity evaluation item based on the consistency of multiple generation results... The generation method is as follows:
[0024] Within the same round, for each input data ,repeat Multiple forward propagations are used to generate multiple outputs, and then the difference between each generated result and the mean is calculated:
[0025]
[0026] in, Indicates the number of times it was generated; For the first Secondary generation result; for The mean of the generated results.
[0027] Furthermore, the security protection process described in S4 specifically involves: in the local model parameters Adding differential privacy perturbation, its expression is:
[0028]
[0029] in, Indicates Gaussian noise; Let be the variance of the Gaussian distribution; It is an identity matrix, which represents that the noise is independent in each dimension and the noise variance is consistent in each dimension.
[0030] Furthermore, the credibility assessment described in S5 specifically involves calculating the hallucination risk assessment value calculated by the client during local training. Its definition is:
[0031]
[0032] If the hallucination risk assessment value If the value exceeds the preset hallucination risk threshold, it is determined to be a parameter of the local model. The credibility weight is used to cancel the federated aggregation and resolve the anomaly. The degree of influence of the client-side model's hallucination risk on the global hallucination risk is determined by:
[0033]
[0034] in, These are hyperparameters used to adjust the degree of influence. This represents the hallucination risk assessment value for the kth client.
[0035] The beneficial effects of this invention are:
[0036] This invention provides a secure and reliable federated learning method for large-scale model illusion perception. By constructing factual constraints based on the difference between model-generated results and reliable factual information, and fidelity evaluation terms based on the consistency of multiple generation results, the generated content of the large model during federated learning training is jointly verified and constrained, thereby reducing the deviation between model-generated results and reliable factual information. By introducing a differential privacy mechanism to the client-side local model parameters and perturbing the model parameters with noise, sensitive client information is further prevented from being leaked during federated communication without sharing the original data. Furthermore, by introducing a model parameter reliability evaluation method in the aggregation phase of federated learning, the server measures the reliability of model parameters uploaded by each client and performs weighted aggregation or filtering based on the evaluation results, effectively reducing the negative impact of low-quality or abnormal models on the global model. Attached Figure Description
[0037] Figure 1 A flowchart of a safe and reliable federated learning method for hallucination perception in large models;
[0038] Figure 2 A framework diagram of a secure and trustworthy federated learning method for large-scale model illusion perception. Detailed Implementation
[0039] The technical solution of the present invention will be further described below with reference to embodiments, but it is not limited thereto. Any modifications or equivalent substitutions to the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention should be covered within the protection scope of the present invention. In the following embodiments, process equipment or devices not specifically specified are all conventional equipment or devices in the art. Unless specifically specified, the technical means used in the embodiments of the present invention are all conventional means well known to those skilled in the art.
[0040] Example 1, Reference Figures 1-2 This embodiment describes a secure and reliable federated learning method for large-scale hallucination perception, comprising:
[0041] S1, Configuration includes The federated learning system for the first client, the first Each client locally holds a private dataset. Each client shares the initial global large model parameters. ;
[0042] S2, Each client is based on the private dataset The large model is trained locally to obtain local model parameters. Constructing an illusion constraint loss function The illusion constraint loss function Used to measure the deviation between the results generated by a large model and reality;
[0043] S3, the illusion constraint loss function Introducing a local optimization objective for the local model parameters After optimization, a local training objective function is obtained, which is expressed as:
[0044]
[0045] in, This represents the task loss function. For global large model parameters, The weighting coefficient for hallucination suppression;
[0046] S4. Regarding the local model parameters Perform security protection processing to obtain local model parameters with security protection. And upload it to the server;
[0047] S5. The server-side protection of the local model parameters. Perform a credibility assessment and, based on the assessment results, execute a weighted federated aggregation to obtain the global model parameters:
[0048]
[0049] in, These are global model parameters. Represents the confidence weights of the client's local model parameters;
[0050] S6. After aggregating the global model parameters, distribute them to each client and repeat steps S2-S5 to obtain a trained large model with a low hallucination rate, thereby achieving hallucination suppression and security enhancement of the large model in a federated learning environment.
[0051] Specifically, the process of this invention is as follows: Figure 1 As shown, Figure 2 The paper presents a large-scale federated learning process involving N clients and servers: when the client trains the large model based on the local dataset, it introduces factual and fidelity constraints, calculates the hallucination risk assessment value, and uploads the model parameter update after differential privacy perturbation; the server issues the initial global parameters, performs credibility assessment on the uploaded content, and performs weighted federated aggregation to obtain new global parameters, thereby realizing large-scale federated learning that combines hallucination suppression and privacy protection.
[0052] Furthermore, the illusion constraint loss function described in S2 Includes factual constraints based on the differences between model-generated results and credible factual information. And fidelity evaluation item based on consistency of multiple generation results The expression is:
[0053]
[0054] in, These are the weighting coefficients for consistency constraints.
[0055] Furthermore, the factual constraint term based on the difference between the model-generated results and credible factual information... The generation method is as follows:
[0056] Within the same round, for each input data ,repeat Multiple outputs are generated through a forward propagation, but only the first generation result of the large model is taken, and then the differences are calculated:
[0057]
[0058] in, This represents the initial generation result of the large model; This indicates the corresponding credible factual information obtained from the input manually labeled data; It is a cosine similarity function used to measure the difference between the generated result and the factual information; Indicates to the client All samples The expectation.
[0059] Specifically, this step quantifies the deviation between the large model's generated results and real information by repeatedly forward propagating single input data within the same training round, taking only the first generated result, measuring its difference from manually labeled credible facts using cosine similarity, and taking the expectation of the full sample from the client.
[0060] Furthermore, the fidelity evaluation item based on the consistency of multiple generation results... The generation method is as follows:
[0061] Within the same round, for each input data ,repeat Multiple forward propagations are used to generate multiple outputs, and then the difference between each generated result and the mean is calculated:
[0062]
[0063] in, Indicates the number of times it was generated; For the first Secondary generation result; for The mean of the generated results.
[0064] Specifically, this step generates multiple outputs by repeating forward propagation multiple times for each input data, and quantifies the consistency of the model's generated results by calculating the difference between each generated result and the mean of all generated results.
[0065] Furthermore, the security protection process described in S4 specifically involves: in the local model parameters Adding differential privacy perturbation, its expression is:
[0066]
[0067] in, Indicates Gaussian noise; Let be the variance of the Gaussian distribution; It is an identity matrix, which represents that the noise is independent in each dimension and the noise variance is consistent in each dimension.
[0068] Furthermore, the credibility assessment described in S5 specifically involves calculating the hallucination risk assessment value calculated by the client during local training. Its definition is:
[0069]
[0070] If the hallucination risk assessment value If the value exceeds the preset hallucination risk threshold, it is determined to be a parameter of the local model. The credibility weight is used to cancel the federated aggregation and resolve the anomaly. The degree of influence of the client-side model's hallucination risk on the global hallucination risk is determined by:
[0071]
[0072] in, These are hyperparameters used to adjust the degree of influence. This represents the hallucination risk assessment value for the kth client.
[0073] Specifically, hallucination risk assessment value A value greater than the preset hallucination risk threshold is represented as:
[0074]
[0075] in, The preset hallucination risk threshold;
[0076] In each communication round, if the hallucination risk assessment value corresponding to the client's local model parameters meets the conditions, the client uploads the hallucination risk assessment value to the server. The server calculates the credibility weight of each client model based on the hallucination risk assessment value, and then completes the weighted federated aggregation, which is represented as:
[0077]
[0078] Although the invention has been described with reference to a limited number of embodiments, those skilled in the art will understand from the foregoing description that other embodiments are conceivable within the scope of the invention described herein. Furthermore, it should be noted that the language used in this specification has been chosen primarily for readability and instructional purposes, and not for the purpose of interpreting or limiting the subject matter of the invention. Therefore, many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the appended claims. The disclosure of the invention is illustrative and not restrictive, and the scope of the invention is defined by the appended claims.
Claims
1. A secure and reliable federated learning method for hallucination perception in large models, characterized in that, include: S1, Configuration includes The federated learning system for the first client, the first Each client locally holds a private dataset. Each client shares the initial global large model parameters. ; S2, Each client is based on the private dataset The large model is trained locally to obtain local model parameters. Constructing an illusion constraint loss function The illusion constraint loss function Used to measure the deviation between the results generated by a large model and reality; S3, the illusion constraint loss function Introducing a local optimization objective for the local model parameters After optimization, a local training objective function is obtained, which is expressed as: ； in, This represents the task loss function. For global large model parameters, The weighting coefficient for hallucination suppression; S4. Regarding the local model parameters Perform security protection processing to obtain local model parameters with security protection. And upload it to the server; S5. The server-side protection of the local model parameters. Perform a credibility assessment and, based on the assessment results, execute a weighted federated aggregation to obtain the global model parameters: ； in, These are global model parameters. Represents the confidence weights of the client's local model parameters; S6. After aggregating the global model parameters, distribute them to each client. Repeat steps S2-S5 until the set number of communication rounds is reached to obtain a large model with a low hallucination rate after training, thereby realizing hallucination suppression and security enhancement of the large model in a federated learning environment.
2. The secure and reliable federated learning method for large-scale hallucination perception according to claim 1, characterized in that, The illusion constraint loss function described in S2 Includes factual constraints based on the differences between model-generated results and credible factual information. And fidelity evaluation item based on consistency of multiple generation results The expression is: ； in, These are the weighting coefficients for consistency constraints.
3. The secure and reliable federated learning method for large-scale hallucination perception according to claim 2, characterized in that, The factual constraint term based on the difference between the model-generated results and credible factual information. The generation method is as follows: Within the same round, for each input data ,repeat Multiple outputs are generated through a forward propagation, but only the first generation result of the large model is taken, and then the differences are calculated: ； in, This represents the initial generation result of the large model; This indicates the corresponding credible factual information obtained from the input manually labeled data; It is a cosine similarity function used to measure the difference between the generated result and the factual information; Indicates to the client All samples The expectation.
4. A secure and reliable federated learning method for large-scale hallucination perception according to claim 3, characterized in that, The fidelity evaluation item based on the consistency of multiple generation results The generation method is as follows: Within the same round, for each input data ,repeat Multiple forward propagations are used to generate multiple outputs, and then the difference between each generated result and the mean is calculated: ； in, Indicates the number of times it was generated; For the first Secondary generation result; for The mean of the generated results.
5. A secure and reliable federated learning method for large-scale hallucination perception according to claim 1, characterized in that, The security protection process described in S4 specifically involves: in the local model parameters Adding differential privacy perturbation, its expression is: ； in, Indicates Gaussian noise; Let be the variance of the Gaussian distribution; It is an identity matrix, which represents that the noise is independent in each dimension and the noise variance is consistent in each dimension.
6. A secure and reliable federated learning method for large-scale hallucination perception according to claim 1, characterized in that, The credibility assessment described in S5 specifically involves calculating the hallucination risk assessment value calculated by the client during local training. Its definition is: ； If the hallucination risk assessment value If the value exceeds the preset hallucination risk threshold, it is determined to be a parameter of the local model. The credibility weight is used to cancel the federated aggregation and resolve the anomaly. The degree of influence of the client-side model's hallucination risk on the global hallucination risk is determined by: ； in, These are hyperparameters used to adjust the degree of influence. This represents the hallucination risk assessment value for the kth client.