A secure alignment method based on neural network parameter subspace repair

Abstract

The application provides a secure alignment method based on a neural network parameter subspace repair, which is applied to a federal learning server side, and private training data of each participant is not accessed in a method execution process. In each round of distributed collaborative training, the following steps are performed: based on a current round global neural network model parameter, a secure parameter increment vector and an unsafe parameter increment vector are respectively obtained; a difference parameter vector is calculated and decomposed to obtain an orthogonal basis matrix of a dangerous parameter subspace; based on the orthogonal basis matrix, a to-be-repaired global parameter update vector is projected to an orthogonal complement space of the dangerous parameter subspace to remove a projection component of the to-be-repaired global parameter update vector on the dangerous subspace; the weighted secure parameter increment vector and the projected parameter update vector are fused to obtain a repaired global parameter update vector to update a parameter of the global neural network model.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and neural network security technology, specifically relating to a secure alignment method based on neural network parameter subspace repair. Background Technology

[0002] With the rapid development of Large Language Models (LLMs), instruction fine-tuning has become a key technology for improving the model's intent understanding ability and task generalization performance. To utilize distributed private data and protect user privacy, Federated Learning (FL) is widely used in the fine-tuning process of Large Language Models. However, the Federated Learning architecture inherently has security vulnerabilities: because the server cannot directly access the client's private data, malicious clients can compromise the model's security through data poisoning or backdoor attacks.

[0003] Especially in FedInstruction Fine-tuning (FedIT) scenarios, attackers can inject malicious behavior into local model updates by constructing malicious command-response pairs. Once these updates are aggregated into the global model, the global model will output detailed and harmful responses to harmful commands. When the model outputs detailed and actionable responses to harmful commands, the harm is not limited to the level of a single interaction, but also exhibits a cascading effect from the individual to the system: at the individual level, fine-grained steps and authoritative statements significantly lower the threshold for implementing harmful behavior, increasing the risk of physical harm, self-harm, and psychological manipulation; at the social level, such outputs may amplify hate narratives, promote radicalization, and exacerbate group antagonism; at the technical and institutional levels, it weakens the model's security alignment boundaries, promotes the generalization of jailbreak strategies, erodes user trust, and causes the attack scope to expand and attack attribution to become difficult in distributed collaborative training scenarios, thus leading to long-term systemic degradation of the model's security boundaries.

[0004] Existing robust defense methods for federated learning, such as mainstream robust aggregation algorithms based on statistical outlier detection like Krum, Trimmed Mean, and Median, all rely on statistical principles, assuming that the update vector of a malicious attacker is an outlier in Euclidean space. However, these existing defense methods fail to effectively detect data poisoning attacks aimed at security alignment in federated instruction fine-tuning scenarios. The core reason for this is that the optimization goals of malicious attacks targeting harmful instructions are highly similar to the optimization goals of benign fine-tuning of normal business instructions. Both require the model to output detailed and direct responses to input instructions rather than rejections. This similarity in optimization goals leads to a high degree of convergence in statistical characteristics between the local model updates trained by benign and malicious clients, rendering a series of existing defense methods that rely on model-level statistical differences ineffective. Furthermore, the coarse-grained update elimination strategies used in existing defense methods are prone to inadvertently penalizing benign clients containing high-quality data, resulting in a significant decrease in the model's effectiveness on general tasks.

[0005] Currently, there is a lack of technical solutions that can accurately separate unsafe update directions at the neural network parameter level, while completing model safety repair and alignment without significantly impairing the model's general performance. Summary of the Invention

[0006] To address the shortcomings and deficiencies of existing technologies, this invention provides a secure alignment method based on neural network parameter subspace repair. It mainly solves the core problems in the context of federated instruction fine-tuning: traditional statistical robust defense methods cannot identify hidden semantically guided data poisoning attacks that converge with the statistical characteristics of benign updates, and coarse-grained defense strategies are prone to inadvertently damaging benign data and significantly impairing the performance of the model in general tasks.

[0007] This invention first constructs a sample set of instruction input-safe response-unsafe response triplets with the same instruction consistency constraint. It then uses a hard example mining mechanism to filter samples near the safety boundary, constructing a unified semantic reference system for comparing the parameter directions of safe and unsafe behaviors. Subsequently, based on the global neural network model benchmark of the current round, it employs an efficient parameter fine-tuning method, performing dual-path fine-tuning on both the safe and unsafe sample sets to obtain the safety parameter increment vector representing the safety alignment direction and the unsafe parameter increment vector representing the unsafe behavior offset direction. Simultaneously, it adaptively determines the fine-tuning rank through a dual criterion of reconstruction error and cross-round stability, balancing the incremental representation accuracy and training stability. Finally, it calculates... The difference between the two types of parameter increments is calculated to obtain the difference parameter vector. Noise interference from general semantic features is eliminated. After decomposing the difference parameter vector, a dangerous parameter subspace is constructed by combining the cumulative energy threshold and cross-round stability constraints. This accurately describes the core parameter direction of the model's shift from safe alignment to unsafe behavior. Finally, the parameter update vector to be repaired after local updates from each participant in the federated learning is obtained. It is first mapped to the orthogonal complement space of the dangerous subspace through orthogonal projection to filter out the unsafe parameter components. Then, weighted safe parameter increments are adaptively injected based on the risk level of the update to be repaired to obtain the repaired parameter update vector and complete the iterative update of the global model.

[0008] This invention is executed entirely on the federated learning server side, without accessing the private training data of each participant or modifying the existing client training protocol. It can effectively suppress hidden data poisoning attacks and backdoor injections while preserving the model's general language capabilities and benign instruction compliance to the greatest extent. It has the advantages of being lightweight and easy to deploy, having high defense accuracy, and low model performance loss. At the same time, it is suitable for the security alignment requirements of neural network models in multiple scenarios such as centralized training and distributed collaborative training.

[0009] The specific technical solution adopted by this invention to solve its technical problem is as follows:

[0010] A secure alignment method based on neural network parameter subspace repair is applied to the federated learning server side. The method does not access the private training data of each participant during execution. In each round of distributed collaborative training, the following steps are performed:

[0011] Based on the global neural network model parameters of the current round, for the safe and unsafe responses corresponding to the same instruction input, obtain the safety parameter increment vector representing the direction of safety alignment and the unsafe parameter increment vector representing the direction of unsafe behavior offset, respectively.

[0012] The difference between the unsafe parameter increment vector and the safe parameter increment vector is calculated to obtain the difference parameter vector. The difference parameter vector is decomposed to obtain the orthogonal basis matrix of the hazard parameter subspace. The hazard parameter subspace is used to characterize the parameter direction of the global neural network model shifting from safe alignment to unsafe behavior.

[0013] Obtain the global parameter update vector to be repaired obtained by aggregating the local parameter updates of each participant in the current round. Based on the orthogonal basis matrix, project the global parameter update vector to be repaired onto the orthogonal complement space of the dangerous parameter subspace to remove the projection component of the global parameter update vector to be repaired on the dangerous subspace, and obtain the projected parameter update vector. Merge the weighted safety parameter increment vector with the projected parameter update vector to obtain the repaired global parameter update vector.

[0014] The parameters of the global neural network model are updated using the repaired global parameter update vector.

[0015] Furthermore, a sample set containing multiple sets of triplet samples is pre-constructed. Each set of triplet samples includes the same instruction input, a safe response to the corresponding instruction input, and an unsafe response. The sample set is divided into a safe sample set and an unsafe sample set. The construction of triplet samples satisfies the same instruction consistency constraint. For the same instruction input, the constructed safe and unsafe responses are completely identical in all task-related content except for the content related to the safety compliance boundary. The content related to the safety compliance boundary includes whether to reject malicious instructions, whether to provide harmful operation details, and whether to provide safety guidance. At the same time, a hard example mining mechanism is adopted. Based on the output distribution, rejection probability, or safety score of the instruction input by the global neural network model in the current round, samples near the safety boundary are selected as hard examples and added to the sample set.

[0016] Furthermore, the safety parameter increment vector and the insecurity parameter increment vector are obtained by fine-tuning the parameters of the global neural network model using a low-rank adapter (LoRA). A first loss function is optimized on the safe sample set to obtain the safety parameter increment vector. The first loss function includes a cross-entropy loss term to increase the probability of generating a safe answer and a KL divergence constraint term to constrain the model distribution shift. A second loss function is optimized on the insecurity sample set to obtain the insecurity parameter increment vector. The second loss function includes a cross-entropy loss term to increase the probability of generating an insecurity answer or decrease the probability of rejecting an answer and a KL divergence constraint term to constrain the model distribution shift.

[0017] Furthermore, the rank r of the low-rank adapter LoRA is determined by a trade-off between reconstruction error and stability: the reconstruction error of the safety parameter increment vector, the insecurity parameter increment vector, and the difference parameter vector under the low-rank subspace approximation of rank r is calculated, and the minimum rank r that satisfies the reconstruction error being less than or equal to a preset maximum reconstruction error threshold is selected; and the rank r is corrected based on the directional stability of the cross-round safety parameter increment vector and the difference parameter vector. When the directional stability is lower than a preset stability threshold, the rank r is increased until the stability constraint is met or the preset maximum rank upper limit is reached.

[0018] Furthermore, singular value decomposition is performed on the difference parameter vector to extract the singular vectors corresponding to the first k principal components and construct the orthogonal basis matrix of the hazard parameter subspace. The dimension k of the hazard parameter subspace is determined according to the cumulative energy threshold, and the smallest k value that satisfies the cumulative energy ratio being greater than or equal to the preset energy threshold is selected. The cumulative energy ratio is the ratio of the sum of squares of the first k singular values ​​to the sum of squares of all available singular values. The dimension k is corrected by the stability constraint of the hazard subspace across rounds. The subspace similarity index based on the Frobenius norm is calculated for the orthogonal basis matrix of the hazard subspace of adjacent rounds. When the similarity index is less than the preset minimum similarity threshold, the dimension k is increased until the stability constraint is satisfied or the preset maximum dimension upper limit is reached.

[0019] Furthermore, an orthogonal complementary projection matrix is ​​constructed based on the orthogonal basis matrix of the hazardous parameter subspace. The orthogonal complementary projection matrix is ​​the difference between the product of the identity matrix and the orthogonal basis matrix and its transpose. The global parameter update vector to be repaired is mapped through the orthogonal complementary projection matrix to remove the projection component of the global parameter update vector to be repaired on the hazardous subspace, thereby obtaining the projected parameter update vector.

[0020] Furthermore, the weight of the security parameter increment vector is a security injection weight coefficient, which is determined using an adaptive method based on the projection energy ratio. The projection energy ratio is the ratio of the L2 norm of the projection component of the global parameter update vector to be repaired on the dangerous subspace to the L2 norm of the global parameter update vector to be repaired. The security injection weight coefficient is a monotonically non-decreasing function of the projection energy ratio, which increases with the increase of the projection energy ratio and decreases with the decrease of the projection energy ratio. Moreover, the security injection weight coefficient is between a preset minimum weight coefficient and a preset maximum weight coefficient.

[0021] Furthermore, the difference parameter vectors from multiple rounds are stacked to form a difference matrix, and the difference matrix is ​​decomposed to construct an orthogonal basis matrix for the dangerous parameter subspace. The parameters of the global neural network model are iteratively updated using the repaired global parameter update vector according to the gradient descent update rule, and the corresponding update step size is set during the update process.

[0022] Furthermore, the method is also adapted to centralized neural network training scenarios, and the neural network parameter update vector to be repaired is derived from the single-round parameter update vector under the centralized neural network training scenario.

[0023] Furthermore, a secure alignment system based on neural network parameter subspace repair, deployed on a federated learning server, which does not access the private training data of each participant in the federated learning process during operation, the system comprising:

[0024] The parameter increment extraction module is used to obtain, based on the parameters of the global neural network model trained in the current round of distributed collaborative training, the safe parameter increment vector representing the safe alignment direction and the unsafe parameter increment vector representing the unsafe behavior offset direction for the safe and unsafe responses corresponding to the same instruction input.

[0025] The hazardous subspace construction module is used to calculate the difference between the unsafe parameter increment vector and the safe parameter increment vector to obtain the difference parameter vector, and to decompose the difference parameter vector to obtain the orthogonal basis matrix of the hazardous parameter subspace. The hazardous parameter subspace is used to characterize the parameter direction of the global neural network model shifting from safe alignment to unsafe behavior.

[0026] The parameter repair module is used to obtain the global parameter update vector to be repaired obtained by aggregating the local parameter updates of each participant in the current round. Based on the orthogonal basis matrix, the global parameter update vector to be repaired is projected onto the orthogonal complement space of the dangerous parameter subspace to remove the projection component of the global parameter update vector to be repaired on the dangerous subspace, so as to obtain the projected parameter update vector. The weighted safety parameter increment vector is fused with the projected parameter update vector to obtain the repaired global parameter update vector.

[0027] The model update module is used to update the parameters of the global neural network model using the repaired global parameter update vector.

[0028] And a computer device including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the method described above.

[0029] A non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described above.

[0030] Compared with the prior art, the present invention and its preferred embodiments have at least the following beneficial effects:

[0031] First, this invention overcomes the technical limitations of existing robust defense methods based on statistical outliers. It can effectively identify and suppress unsafe behaviors such as hidden semantically guided data poisoning attacks and backdoor injections that converge with the statistical characteristics of benign model updates in federated instruction fine-tuning scenarios. This solves the core technical problem of existing defense schemes being difficult to detect and prone to failure in non-outlier malicious updates. Second, this invention adopts a parameter repair mechanism that combines directional projection depoisoning with adaptive secure injection. This mechanism can accurately filter out unsafe components in model parameter updates while preserving the model's general language capabilities and benign instruction compliance capabilities to the greatest extent. This avoids the shortcomings of existing coarse-grained removal strategies, which are prone to accidentally damaging benign data and causing a significant decrease in the model's general task utility. This achieves an effective balance between model security and practical performance. Third, this invention can be executed entirely on the federated learning server side without accessing the private training data of each participant or modifying the existing client's training protocol and optimization process. It also relies on an efficient parameter fine-tuning scheme to achieve core functions, with low computational and resource overhead and good deployment compatibility and feasibility. Furthermore, through optimized designs such as instruction consistency constraints, hard example mining mechanisms, and cross-cycle stability correction, this invention effectively improves the modeling accuracy of the dangerous parameter subspace and the defensive stability of cross-cycle training. It also possesses stronger identification and defense capabilities against covert attacks near the security boundary, ensuring that the model's security boundary does not systematically degrade during long-term iterative training. Finally, this invention is not only suitable for distributed collaborative training scenarios but also compatible with the model security alignment requirements in centralized neural network training scenarios, thus possessing a wide range of applications. Attached Figure Description

[0032] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0033] Figure 1 This is a flowchart illustrating the secure alignment method based on neural network parameter subspace repair according to an embodiment of the present invention.

[0034] Figure 2 This is a schematic diagram of the systematic implementation structure of the hazardous parameter subspace construction and parameter projection repair in an embodiment of the present invention. Detailed Implementation

[0035] To make the features and advantages of the present invention more apparent and understandable, specific embodiments are described below in detail:

[0036] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0037] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0038] This invention aims to provide a secure alignment method that can accurately separate unsafe directions at the neural network parameter level and perform repair and enhancement without significantly impairing model utility. Specifically, it is a secure alignment method based on neural network parameter subspace repair, addressing the problems of insufficient identification of hidden unsafe parameter updates in existing defense technologies and the vulnerability of the defense process to compromise the model's general capabilities. This invention focuses on neural network parameter space orientation modeling and employs a technical route of: comparing secure and unsafe parameter increments, constructing a dangerous subspace, performing orthogonal projection detoxification, and injecting secure increments for enhancement. This achieves targeted suppression of unsafe parameter components and strengthens security capabilities. By comparing and modeling secure and unsafe parameter increments, a dangerous parameter subspace is constructed, and projection repair and secure injection are performed on neural network parameter updates. This method is suitable for centralized training or distributed collaborative training scenarios.

[0039] The implementation process of this invention includes:

[0040] 1. Data Construction and Preprocessing Steps. Construct a triplet dataset containing instruction inputs, safe responses, and unsafe responses. The triplet dataset includes both benign and malicious datasets.

[0041] 2. Parameter Increment Calculation Steps. Based on the trainable parameter set of the current round of the neural network model, efficient fine-tuning of the parameters is performed on both benign and malicious datasets to obtain the security parameter increment vector representing the security alignment direction. and the unsafe parameter increment vector characterizing the direction of unsafe behavior .

[0042] 3. Hazardous subspace construction steps. Based on the obtained safety parameter increment vector... and unsafe parameter increment vector Construct the difference parameter vector and the difference parameter vector Decomposition yields the orthogonal basis matrix of the dangerous subspace. .

[0043] 4. Projection Repair Steps. Obtain the parameter update vector of the neural network to be repaired. Update the neural network parameter vector Projecting onto the orthogonal complement space of the dangerous subspace yields the parameter update vector after projection. It is used to filter and exclude parameter components along unsafe directions.

[0044] 5. Safe Injection and Parameter Update Steps. Update the parameter vector after projection. Inject weighted security parameter increment vector Obtain the repaired neural network parameter update vector And update the vector using the repaired neural network parameters. Update the neural network model parameters.

[0045] The trainable parameter set of the above neural network model includes weight parameters, bias parameters, and parameters of the parameter-efficient fine-tuning module; the trainable parameter set is flattened and concatenated to form a parameter vector. And the neural network parameter update vector is represented as a parameter increment vector. .

[0046] In step 1, input each instruction. Match a safe answer to each And an unsafe answer To form triplet samples Among them, a safe response is used to reject a malicious request or provide safe guidance, while an insecure response is used to indicate compliance with a malicious request.

[0047] In step 2, efficient parameter fine-tuning is implemented using a low-rank adapter LoRA, optimizing the first loss function on a benign dataset to obtain the safety parameter increment vector. The first loss function includes a cross-entropy loss term to increase the probability of generating safe answers and a KL divergence term to constrain the fine-tuning of the model's distribution shift. The second loss function is optimized on a malicious dataset to obtain an increment vector of insecure parameters. The second loss function is used to increase the probability of generating an unsafe answer or reduce the probability of refusing an answer, and includes a KL divergence constraint term.

[0048] In step 3, the difference parameter vector Determined by the following formula: and the difference parameter vector Perform singular value decomposition and extract the previous values. The singular vectors corresponding to the principal components construct the orthogonal basis matrix of the dangerous subspace. .

[0049] In step 4, the projection matrix is ​​constructed. I is the identity matrix; and the parameters of the neural network to be repaired are updated in the vector. Mapped to To remove the neural network parameter update vector Projected components in the dangerous subspace.

[0050] In step 5, the security injection module obtains the repaired neural network parameter update vector in the following manner: ,in, For a safe incremental vector, Inject weighting coefficients for security.

[0051] Preferably, in the present invention, the formula for updating the neural network model parameters using the repaired neural network parameter update vector is: ,in, These are the parameters for the current round of the model. For the parameter vector of the next round, To update the step size.

[0052] Neural network parameter update vector to be repaired The method originates from the aggregated parameter update vector in the distributed collaborative training scenario. Without accessing the private training data of the participants, the method performs dangerous subspace orthogonal complementary projection repair and safe parameter incremental injection on the aggregated parameter update vector to suppress the transfer of hidden unsafe behaviors and maintain the generality of the model.

[0053] Dangerous subspace dimension Based on the cumulative energy threshold, select the minimum value that satisfies the following conditions. : ,in, For the difference parameter vector The sequence of singular values ​​obtained by performing singular value decomposition. For elements in this sequence, The number of available singular values. The energy threshold is used; and further, stability constraints are applied to the cross-cycle dangerous subspace. Correction is performed by adjusting the orthogonal basis matrices of the dangerous subspaces of adjacent rounds. and Calculate similarity ,when At that time, adjust the dimensions upwards. Until the stability constraints are satisfied Or reach the preset limit ,||·|| F For Frobenius norm, s min This represents the minimum similarity requirement for the dangerous subspaces of adjacent rounds, used to constrain the stability of the subspace across rounds, and its value ranges from 0 to 1.

[0054] Safe injection weight coefficient An adaptive determination method based on the proportion of projected energy is adopted: the parameter update vector of the neural network to be repaired is calculated. Projected energy percentage in the dangerous subspace And based on the proportion of projected energy Sure For about A monotonically non-decreasing function such that when Increase when it increases To enhance the strength of safe alignment injection, when Decrease To reduce perturbations to the components of the general capability parameter, the monotonically non-decreasing function is one of the following: a piecewise linear function, a sigmoid function, or an equivalent function, and satisfies... .

[0055] The triplet dataset is constructed to satisfy the same instruction consistency constraint and employs a hard example mining mechanism: for each instruction input Construct corresponding security answers with unsafe answers The safe and unsafe responses are highly similar in task-related content, differing only at the safety boundary to reduce parameter increment noise caused by general semantic factors. Furthermore, the instruction input is based on the current round model. The output distribution, rejection probability, or safety score are used to select samples near the safety boundary as difficult examples and add them to the triplet dataset, so that the safety parameter increment vector and the unsafe parameter increment vector can more centrally represent the difference between the safety alignment direction and the unsafe direction.

[0056] Rank of the low-rank adapter LoRA Determined based on a trade-off between reconstruction error and stability: Calculate the incremental vector of safety parameters. Incremental vector of unsafe parameters and difference parameter vector In rank Reconstruction error under low-rank subspace approximation and select the one that satisfies minimum rank Furthermore, based on the cross-round safety parameter incremental vector and difference parameter vector Directional stability with respect to rank Correction is performed, and the adjustment is increased when the directional stability is below the threshold. Until the stability constraint is met or the preset upper limit is reached. .

[0057] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific implementations. This invention addresses the problem that existing robust defense methods for federated learning, such as Krum, Trimmed Mean, and Median algorithms, are ineffective in defending against data poisoning attacks by FedIT aimed at security alignment. It employs a proactive defense strategy by simulating the attack and defense process on the server side, extracting dangerous subspaces that characterize attack features, and using projection repair techniques to accurately depoison and enhance the security of the global model.

[0058] like Figure 2 As shown, the defense system of this invention can be divided into five functional modules according to the processing flow: a data construction and preprocessing module, a parameter incremental calculation module, a dangerous parameter subspace construction module, a projection detoxification module, and a security injection and parameter update module. The data construction and preprocessing module is deployed on the server side and pre-constructs a data structure containing command inputs without accessing the client's private data. Safe answer and unsafe answers The triplet dataset performs preprocessing on instructions and responses, including word segmentation, length truncation, and consistency constraints with the same instruction. When necessary, hard example mining is introduced to focus on samples near the safety boundary, thus forming a unified semantic reference system for safe and unsafe behaviors. The parameter incremental calculation module is based on the global model parameters of the current round. A low-rank adapter (LoRA) is used to construct secure and insecure losses with KL regularization on secure and insecure data subsets, respectively. The LoRA module is then fine-tuned to obtain secure parameter increments representing the secure alignment direction and insecure parameter increments representing the attack direction. The LoRA rank is adaptively determined using a dual criterion of reconstruction error and cross-round stability. The hazardous parameter subspace construction module subtracts and stacks the secure and insecure parameter increments to form a difference matrix. Using truncated singular value decomposition (SVD) under the constraints of cumulative energy threshold and subspace similarity, it extracts the main differential principal components, constructing a hazardous parameter subspace shifted from secure to insecure, and generating the corresponding orthogonal projection operator. The projection depoisoning module receives local model updates uploaded by each client during the federated learning aggregation phase and performs weighted aggregation to obtain a global update vector. This update vector is then multiplied left by the orthogonal projection operator of the hazardous subspace, mapping it to the orthogonal complement space of the hazardous subspace, thus explicitly filtering out parameter components aligned with the insecure direction. The secure injection and parameter update module calculates the risk index based on the projection energy ratio of the global update vector on the hazardous subspace. Through a preset Sigmoid type mapping function from Adaptive determination of safe injection coefficient We inject weighted security parameter increments into the update vector after projection depoisoning to obtain the repaired update amount, and iterate and update the global model parameters accordingly. This suppresses hidden backdoors and data poisoning attacks while maintaining the model's general utility and instruction compliance as much as possible.

[0059] As a further preferred embodiment, all five functional modules of the present invention are uniformly deployed on the federated learning server side. The execution process of all modules does not require access to the local private training data of the federated learning participants, and does not require any modification to the local training process, training protocol, or optimizer configuration of the client.

[0060] like Figure 1 As shown, the specific implementation of the method in this embodiment of the invention refers to the following steps:

[0061] Step S1: Data Construction and Preprocessing. Training data for incremental parameter extraction is pre-constructed on the federated learning server side or training control side, without needing access to the private training samples of each participant. For each instruction input... We construct triplet samples that simultaneously contain safe and insecure answers to characterize the semantic difference direction from safe to insecure.

[0062] Furthermore, the specific steps of step one are as follows:

[0063] 1-1. Data Source Preparation. Prepare a benign question-answering dataset and a malicious question-answering dataset on the server side. The benign dataset can use publicly available safe dialogue corpora or be generated using a large language model. The answers it contains should meet general task requirements and conform to human values ​​and safety standards. The malicious dataset contains misleading and dangerous instructions along with their corresponding unsafe answers. All of the above data is built offline on the server side without accessing the client's private data.

[0064] 1-2, Triple Construction. To capture the semantic differences between secure and insecure pairs, triple samples are constructed. .in, Input the command; The corresponding safe response is to refuse to execute malicious instructions or to provide harmless guiding advice; The corresponding unsafe response is to comply with malicious instructions and output harmful content. A benign dataset is formed from triplet samples. With malicious datasets It can perform word segmentation, length truncation, and template alignment on the input, and perform tag consistency verification on the output.

[0065] 1-3, Consistency constraints with the same instructions. To ensure the difference parameter vector... To more comprehensively represent the semantic direction of parameters from safe to unsafe, a consistency constraint with the same instruction is introduced in the construction of triples: for the same instruction input Constructing a secure answer with unsafe answers The tasks are highly similar in terms of relevant information, with differences only at the safety boundaries. For example, there are differences in whether to provide actionable steps, whether to provide harmful details, and whether to provide refusal and safety guidance, in order to reduce the interference of common language ability factors on incremental direction extraction.

[0066] 1-4, Difficult Example Mining. A difficult example mining mechanism is introduced to enhance sensitivity to subspace risk directions: based on the current round of the model, the input... The output distribution, rejection probability, or safety score are used to select samples near the safety boundary and add them to the triplet dataset. This makes the comparison between the increment of safety parameters and the increment of unsafe parameters more focused on the safety boundary flip region, thereby improving the ability of the dangerous subspace to characterize hidden unsafe directions.

[0067] As a further preferred implementation, the selection criteria for samples near the security boundary are: selecting samples whose model's rejection probability for the instruction input is in the range of 0.2 to 0.8, or samples whose security score is between a preset security threshold and a preset unsafe threshold; the security score is the compliance score of the model's output content, which can be obtained through a large language model security assessment tool commonly used in this field.

[0068] Step S2: Parameter Increment Calculation. Based on the neural network parameter vector of the current round. The parameter increments for safe and unsafe directions are calculated separately using an efficient fine-tuning method. To reduce catastrophic forgetting and control distribution shift, a KL divergence constraint can be added to the loss function.

[0069] Furthermore, the specific steps of step two are as follows:

[0070] 2-1, Incremental Calculation of Safety Parameters. Construct a first loss function that includes cross-entropy loss and KL divergence constraints. ,in, Indicates the security dataset Calculate the expectation of the samples in the dataset; This represents the negative log-likelihood of supervised learning, encouraging the model to prioritize probabilistic quality over safe answers; This represents the weighting coefficient, which controls the trade-off between the degree of safe alignment and the ability to maintain the original model; Used to penalize the current updated distribution. Parameter snapshot relative to this round The deviation limits the update amplitude in a single round and stabilizes training. The LoRA module is fine-tuned under the constraints of the above loss function, and the parameter vector obtained after optimization convergence is denoted as... Define the security parameter increment vector as .

[0071] 2-2, Incremental Calculation of Insecure Parameters. Construct a second loss function that incorporates negative cross-entropy loss and KL divergence constraints. ,in, This represents the expectation of samples on an insecure dataset; This indicates a supervision term that encourages the model to produce unsafe outputs with a higher probability. This represents the weighting coefficient, which controls the trade-off between fitting an unsafe objective and maintaining the ability to preserve the original model; Used to penalize the current updated distribution. Parameter snapshot relative to this round The deviation limits the update amplitude in a single round and stabilizes training. The LoRA module is fine-tuned under this loss function constraint, and the converged parameter vector is denoted as... Define the unsafe parameter increment vector as .

[0072] 2-3, The dual-criteria joint rank selection mechanism for LoRA rank. To ensure that the safe parameter increments and unsafe parameter increments obtained through efficient parameter fine-tuning can fully characterize the corresponding directions while maintaining stability across rounds, this invention employs a dual-criteria joint rank selection mechanism for LoRA rank. An adaptive selection method combining reconstruction error and stability constraints is adopted. Specifically, the increment vectors of multiple batches or multiple layers are stacked column-wise to form an increment matrix. ,right Perform singular value decomposition: Given rank Lower construction truncated SVD approximation ,in , , Take the first one respectively The submatrix obtained from singular vectors. Define the rank. The reconstruction error at time is Select the one that satisfies The smallest ,in A preset error threshold is set. To further ensure stability across rounds, the incremental matrix obtained in adjacent rounds is used. and The above are constructed with rank as subspace basis , Define the stability index as .when At that time, it is assumed that the incremental direction remains stable across rounds; when At that time, by raising the order The rank is corrected until both the reconstruction error and stability constraints are satisfied, or a preset upper limit is reached. The LoRA rank obtained in this way It ensures that the incremental direction is fully expressed while suppressing the introduction of noise and unstable directions.

[0073] Step S3: Construction of the hazardous parameter subspace, which is one of the core steps of this invention. By comparing the two increments obtained in Step 2, the essential features of the attack behavior in the parameter space are extracted, and a hazardous subspace representing the shift from safe to unsafe is constructed for subsequent projection depoisoning of the collaborative training parameter updates.

[0074] Furthermore, the specific steps of step three are as follows:

[0075] 3-1, Constructing the difference parameter vector. Calculate the difference between the insecurity increment and the safety increment: The difference vectors from multiple rounds are then stacked to form a difference matrix to enhance statistical stability. This difference vector mathematically eliminates the underlying general language capability features because basic grammatical and semantic capabilities are required to generate both safe and insecure answers, thus purely preserving the semantic direction of the shift from safe to insecure.

[0076] As a further preferred implementation, the stacking method of the multi-round difference vectors is column stacking, that is, the difference parameter vector calculated in each round is used as a column of the difference matrix to form a multi-column difference matrix; the stacking strategy can adopt cumulative full stacking or sliding window stacking, where cumulative full stacking retains the difference vectors of all rounds after training starts and stacks them, and sliding window stacking retains the difference vectors of consecutive rounds within a preset window size before the current round and stacks them, so as to balance statistical stability and computational cost.

[0077] 3-2, Principal Component Extraction and Energy Threshold Determination. For the differential parameter vector... Singular Value Decomposition (SVD) is performed using the following formula: ,in, and These are the left singular vector matrix and the right singular vector matrix, respectively. This is a singular value diagonal matrix. Since attacks typically focus on certain specific parameter dimensions, we extract the matrix... The middle corresponds to the front A column vector of maximum singular values The orthogonal basis matrix of the dangerous subspace is constructed using the principal component directions. , where column vectors Let be the corresponding left singular vector. Let the set of singular values ​​be... The initial dimension is determined based on the cumulative energy threshold. Select the one that satisfies The smallest ,in The number of available singular values. A preset threshold for the proportion of singular value cumulative energy is used to determine the initial dimension k of the dangerous subspace, with a value ranging from 0 to 1, preferably 0.80-0.98. This yields the initial orthogonal basis matrix. .

[0078] 3-3, Cross-round stability correction. To improve the stability of the hazardous subspace during multi-round collaborative training, cross-round stability constraints are introduced. This involves adjusting the orthogonal basis matrices obtained from adjacent rounds. and Calculate the subspace similarity index ,when At that time, the dangerous subspace is considered stable, and the final dimension is taken. ;when At that time, by adjusting the dimensions Until the stability constraint is met or the preset upper limit is reached. By selecting the energy threshold and performing stability correction as described above, the hazardous subspace can cover the main unsafe directional components and maintain directional consistency in multiple iterations.

[0079] Step S4: Projection Detoxification. After the hazardous parameter subspace is constructed, the update vector of the parameters to be repaired obtained from co-training is orthogonally projected to remove its components in the hazardous subspace, thereby suppressing the transfer of unsafe behaviors.

[0080] Furthermore, the specific steps of step four are as follows:

[0081] 4-1, Construction of parameter updates to be repaired. In federated or distributed collaborative training scenarios, obtain the parameter update vector after this round of aggregation. Using weighted average aggregation, it can be expressed as: ,in, For the first Local parameter update vectors for each participant. For the corresponding weights.

[0082] 4-2, Construction of the projection matrix. Based on the orthogonal basis matrix of the hazardous subspace. Constructing the projection matrix ,in It is an identity matrix with the same dimensions as the parameters.

[0083] 4-3, Projection Detoxification Update. Update the vector with the parameters to be repaired. Multiplying by the projection matrix on the left yields the parameter update vector after projection. This allows for explicit removal in the parameter space. The components in the hazardous subspace retain only the parameter update components related to general capabilities and safety orientation.

[0084] Step S5: Safety Injection and Parameter Update. After completing the projection detoxification, safety parameter increments are injected and adaptive intensity adjustments are made to actively repair unsafe directions and update the neural network parameters.

[0085] Furthermore, the specific steps of step five are as follows:

[0086] 5-1, Safe Injection. Safe injection is performed on the projected parameter update vector to obtain the repaired parameter update vector: ,in, Inject weighting coefficients for security.

[0087] 5-2, Adaptive Weight Determination Based on Projected Energy Ratio. To ensure that the safety injection strength matches the risk level of the update to be repaired in the current round, this invention adaptively determines the weights based on the projected energy ratio of the update vector to be repaired in the dangerous subspace. Specifically, calculation ,in, Let be the orthogonal basis matrix of the dangerous subspace. When A larger value indicates a stronger component of the update to be repaired in the unsafe direction, and the strength of the safe injection should be increased. When the value is relatively small, it indicates a weaker risk component, and the safety injection strength should be reduced to minimize disturbances to the general capability parameter components. Therefore, let... in It is a monotonically non-decreasing function and satisfies . The Sigmoid function is used to achieve smooth and controllable safe injection adjustment in different risk ranges.

[0088] As a further preferred embodiment, the specific interval mapping form of the above Sigmoid function is as follows:

[0089]

[0090] in, The hyperparameter for adjusting the slope of the function is preferably set to a value of 5 to 15; Let be the center point of the function, preferably with a value of 0.5; through the above mapping, the proportion of projected energy r with a value in the range [0,1] can be smoothly mapped to . Within the range, the safe injection intensity is adaptively and monotonically increased according to the risk level.

[0091] 5-3, Parameter Update. Finally, update the neural network parameters according to the following formula: ,in To update the step size, the above-mentioned repair update, which combines projection depollution and safe injection, effectively suppresses the transfer of unsafe behaviors during collaborative training without accessing the private data of the participants, while maintaining the model's generality as much as possible.

[0092] This invention addresses the vulnerability of large models to covert data poisoning attacks disguised as secure alignment during instruction fine-tuning. It proposes a secure alignment method based on neural network parameter subspace repair. Existing defense strategies, such as statistically robust aggregation algorithms like Krum, Trimmed Mean, and Median, typically assume that malicious updates exhibit significant outlier characteristics in their statistical distribution. However, in secure alignment scenarios where the common optimization goal is to generate sufficient information without rejecting answers, benign and malicious client-generated local updates share highly similar statistical characteristics. This makes it difficult for traditional defense methods to promptly identify and filter such semantically driven data poisoning. This invention departs from anomaly detection in update distribution. Instead, it starts from the semantic differences between secure and insecure answers, explicitly constructing a dangerous parameter subspace on the server side, and performing projection depoisoning and secure injection repair on aggregated updates, thereby achieving effective defense against covert attacks. Its innovative design and advantages include:

[0093] 1. This invention proposes a defense and remediation mechanism based on a dangerous parameter subspace. It constructs a triplet dataset of safe and unsafe answers on the server side, employs a low-rank adapter to efficiently fine-tune the parameters of the neural network with safe and unsafe increments, utilizes truncated SVD to extract the difference directions between the two, and constructs a dangerous parameter subspace by combining reconstruction error and cross-round stability criteria. Compared to existing defense methods that rely solely on model output or statistical features, this invention directly characterizes the directional information from safe to unsafe in the parameter space, enabling the identification and suppression of covert backdoor attacks and data poisoning attacks that are statistically insignificant but semantically biased.

[0094] 2. Achieving a balance between security and model utility under parameter subspace repair. This invention utilizes a dangerous parameter subspace to construct an orthogonal projection operator, updating and mapping the aggregated parameters obtained from co-training to a safe subspace orthogonal to the dangerous direction, explicitly removing components aligned with the attack direction. Simultaneously, an adaptive security injection mechanism based on projection energy ratio is introduced. This mechanism dynamically determines the injection weight of the security increment through a Sigmoid mapping function, injecting a stable safe alignment direction into the model while removing risky components. This significantly reduces the probability of harmful responses while maximizing the preservation of general language capabilities and benign instruction compliance, avoiding performance degradation caused by coarse-grained defenses such as overall scaling and one-size-fits-all filtering.

[0095] 3. Low computational overhead and high deployment compatibility. This invention uniformly employs a low-rank adapter (LoRA) on the server side for incremental extraction and dangerous subspace construction. It can characterize the safe and insecure directions using only a small amount of publicly available benchmark data, without accessing the private data of each client or modifying existing client training protocols and optimizers. Compared to full-parameter fine-tuning schemes, this invention significantly reduces memory and computational resource overhead, facilitating rapid integration and deployment in resource-constrained federated learning servers, centralized large model training platforms, and privacy-sensitive high-risk application scenarios. It provides a general, lightweight, and scalable defense path for secure alignment of large models in untrusted environments.

[0096] As a further preferred embodiment, the present invention can be adapted to centralized neural network training scenarios. In centralized training scenarios, the neural network parameter update vector to be repaired comes from the model gradient update vector obtained from a single round of centralized training, without the need to perform weighted aggregation operations of local updates from multiple clients. The implementation methods of the remaining steps (parameter increment calculation, dangerous subspace construction, projection repair, security injection and parameter update) are completely consistent with those of distributed collaborative training scenarios, which can achieve secure alignment of the model during centralized training and suppress the security alignment boundary degradation problem caused by malicious training data during training.

[0097] Based on the same inventive concept, this invention also provides a computer device, comprising: one or more processors, and a memory for storing one or more computer programs; the programs include program instructions, and the processor executes the program instructions stored in the memory. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, used to implement one or more instructions, specifically for loading and executing one or more instructions stored in a computer storage medium to implement the above-described method.

[0098] It should be further explained that, based on the same inventive concept, the present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, performs the above-described method. This storage medium can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0099] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0100] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

[0101] This invention is not limited to the preferred embodiment described above. Anyone inspired by this invention can derive other forms of a secure alignment method based on neural network parameter subspace repair. All equivalent variations and modifications made within the scope of the claims of this invention shall fall within the scope of this invention.

Claims

1. A secure alignment method based on neural network parameter subspace repair, applied to a federated learning server, wherein the method does not access the private training data of each participant during execution, characterized in that, In each round of distributed collaborative training, the following steps are performed: Based on the global neural network model parameters of the current round, for the safe and unsafe responses corresponding to the same instruction input, obtain the safety parameter increment vector representing the direction of safety alignment and the unsafe parameter increment vector representing the direction of unsafe behavior offset, respectively. The difference between the unsafe parameter increment vector and the safe parameter increment vector is calculated to obtain the difference parameter vector. The difference parameter vector is decomposed to obtain the orthogonal basis matrix of the hazard parameter subspace. The hazard parameter subspace is used to characterize the parameter direction of the global neural network model shifting from safe alignment to unsafe behavior. Obtain the global parameter update vector to be repaired obtained by aggregating the local parameter updates of each participant in the current round. Based on the orthogonal basis matrix, project the global parameter update vector to be repaired onto the orthogonal complement space of the dangerous parameter subspace to remove the projection component of the global parameter update vector to be repaired on the dangerous subspace, and obtain the projected parameter update vector. The weighted security parameter increment vector is fused with the projected parameter update vector to obtain the repaired global parameter update vector. The parameters of the global neural network model are updated using the repaired global parameter update vector; Singular value decomposition is performed on the difference parameter vector to extract the singular vectors corresponding to the first k principal components and construct an orthogonal basis matrix for the hazard parameter subspace; the dimension k of the hazard parameter subspace is determined according to the cumulative energy threshold, and the smallest k value that satisfies the cumulative energy ratio being greater than or equal to the preset energy threshold is selected, where the cumulative energy ratio is the ratio of the sum of squares of the first k singular values ​​to the sum of squares of all available singular values; The dimension k is corrected by the stability constraint of the cross-round danger subspace. The subspace similarity index based on the Frobenius norm is calculated for the orthogonal basis matrix of the danger subspace of adjacent rounds. When the similarity index is less than the preset minimum similarity threshold, the dimension k is increased until the stability constraint is met or the preset maximum dimension upper limit is reached. The weight of the incremental safety parameter vector is the safety injection weight coefficient, which is determined using an adaptive method based on the projection energy ratio. The projection energy ratio is the ratio of the L2 norm of the projection component of the global parameter update vector to be repaired on the dangerous subspace to the L2 norm of the global parameter update vector to be repaired. The safety injection weight coefficient is a monotonically non-decreasing function of the projection energy ratio, which increases with the increase of the projection energy ratio and decreases with the decrease of the projection energy ratio. The safety injection weight coefficient is between a preset minimum weight coefficient and a preset maximum weight coefficient.

2. The secure alignment method based on neural network parameter subspace repair according to claim 1, characterized in that: A sample set containing multiple sets of triplet samples is pre-constructed. Each triplet sample includes the same instruction input, a safe response to the corresponding instruction input, and an unsafe response. The sample set is divided into a safe sample set and an unsafe sample set. The construction of triplet samples satisfies the same instruction consistency constraint. For the same instruction input, the constructed safe and unsafe responses are completely identical in all task-related content except for the content related to the safety compliance boundary. The content related to the safety compliance boundary includes whether to reject malicious instructions, whether to provide harmful operation details, and whether to provide safety guidance. At the same time, a hard example mining mechanism is adopted. Based on the output distribution of the instruction input, the rejection probability, or the safety score of the global neural network model in the current round, samples near the safety boundary are selected as hard examples and added to the sample set.

3. The secure alignment method based on neural network parameter subspace repair according to claim 2, characterized in that: The incremental vectors of safe and unsafe parameters are obtained by fine-tuning the parameters of the global neural network model using a low-rank adapter (LoRA). A first loss function is optimized on the safe sample set to obtain the incremental vector of safe parameters. The first loss function includes a cross-entropy loss term to increase the probability of generating safe answers and a KL divergence constraint term to constrain the model distribution shift. A second loss function is optimized on the unsafe sample set to obtain the incremental vector of unsafe parameters. The second loss function includes a cross-entropy loss term to increase the probability of generating unsafe answers or decrease the probability of rejecting answers and a KL divergence constraint term to constrain the model distribution shift.

4. The secure alignment method based on neural network parameter subspace repair according to claim 3, characterized in that: The rank r of the low-rank adapter LoRA is determined by a trade-off between reconstruction error and stability: the reconstruction error of the safety parameter increment vector, the insecurity parameter increment vector, and the difference parameter vector under the low-rank subspace approximation of rank r is calculated, and the minimum rank r that satisfies the reconstruction error being less than or equal to a preset maximum reconstruction error threshold is selected; and the rank r is corrected based on the directional stability of the cross-round safety parameter increment vector and the difference parameter vector. When the directional stability is lower than a preset stability threshold, the rank r is increased until the stability constraint is met or the preset maximum rank upper limit is reached.

5. The secure alignment method based on neural network parameter subspace repair according to claim 1, characterized in that: An orthogonal complementary projection matrix is ​​constructed based on the orthogonal basis matrix of the hazard parameter subspace. The orthogonal complementary projection matrix is ​​the difference between the product of the identity matrix and the orthogonal basis matrix and its transpose. The global parameter update vector to be repaired is mapped using an orthogonal complementary projection matrix to remove the projection component of the global parameter update vector on the dangerous subspace, resulting in the projected parameter update vector.

6. The secure alignment method based on neural network parameter subspace repair according to claim 1, characterized in that: The difference parameter vectors from multiple rounds are stacked to form a difference matrix. The difference matrix is ​​then decomposed to construct an orthogonal basis matrix for the dangerous parameter subspace. The parameters of the global neural network model are iteratively updated using the repaired global parameter update vector according to the gradient descent update rule, with a corresponding update step size set during the update process.

7. The secure alignment method based on neural network parameter subspace repair according to claim 1, characterized in that: The method is also compatible with centralized neural network training scenarios, and the parameter update vector of the neural network to be repaired comes from the single-round parameter update vector under the centralized neural network training scenario.

8. A secure alignment system based on neural network parameter subspace repair, used to implement the method as described in claim 1, characterized in that, Deployed on the federated learning server side, and without accessing the private training data of the participants in the federated learning process during system operation, the system includes: The parameter increment extraction module is used to obtain, based on the parameters of the global neural network model trained in the current round of distributed collaborative training, the safe parameter increment vector representing the safe alignment direction and the unsafe parameter increment vector representing the unsafe behavior offset direction for the safe and unsafe responses corresponding to the same instruction input. The hazardous subspace construction module is used to calculate the difference between the unsafe parameter increment vector and the safe parameter increment vector to obtain the difference parameter vector, and to decompose the difference parameter vector to obtain the orthogonal basis matrix of the hazardous parameter subspace. The hazardous parameter subspace is used to characterize the parameter direction of the global neural network model shifting from safe alignment to unsafe behavior. The parameter repair module is used to obtain the global parameter update vector to be repaired obtained by aggregating the local parameter updates of each participant in the current round. Based on the orthogonal basis matrix, the global parameter update vector to be repaired is projected onto the orthogonal complement space of the dangerous parameter subspace to remove the projection component of the global parameter update vector to be repaired on the dangerous subspace, so as to obtain the projected parameter update vector. The weighted safety parameter increment vector is fused with the projected parameter update vector to obtain the repaired global parameter update vector. The model update module is used to update the parameters of the global neural network model using the repaired global parameter update vector.

Images