A neural network model fusion method based on adversarial robust interpolation and bayesian optimization

Abstract

This invention discloses a neural network model fusion method based on adversarial robust interpolation and Bayesian optimization, belonging to the field of artificial intelligence and deep learning model security protection technology. Addressing the difficulty in simultaneously achieving robustness and accuracy of deep neural networks under white-box adversarial attacks, and the shortcomings of existing methods in interpolation parameter selection, Fisher information estimation, and iterative optimization, this invention achieves efficient fusion of neural network models through data partitioning and adversarial example generation, normal and adversarial training of the baseline model, interpolation model generation and empirical Fisher information matrix calculation, dynamic search for optimal interpolation parameters using Bayesian optimization, and backpropagation retraining and iterative optimization. This method significantly improves adversarial robustness while maintaining the standard accuracy of the model, and can be widely applied to security-critical scenarios such as facial recognition access control systems, financial voiceprint authentication, and IoT malware detection.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and deep learning model security protection technology, and in particular relates to a neural network model fusion method based on adversarial robust interpolation and Bayesian optimization. Background Technology

[0002] With the rapid development of artificial intelligence technology, deep neural networks have been widely applied in fields such as image recognition, speech recognition, and natural language processing. However, these models face severe security challenges during practical deployment, particularly the threat of adversarial attacks. Attackers can easily mislead deep neural networks into producing erroneous outputs by adding minute perturbations to input data in a white-box environment, which seriously affects the reliability of the model in security-critical scenarios. To address this threat, adversarial training techniques improve model robustness by incorporating adversarial examples into the training process. However, this approach often sacrifices the model's standard accuracy on clean data, making it difficult to balance robustness and accuracy in practical applications.

[0003] In existing technologies, robustness enhancement methods based on model ensemble and parameter interpolation attempt to balance standard accuracy and adversarial robustness by integrating multiple model parameters obtained under different training strategies. For example, model fusion is achieved through linear interpolation, weighted averaging, or knowledge distillation, or the fusion process is guided by Fisher information matrices. However, these methods have significant limitations: interpolation coefficients are often determined using fixed weights or simple grid search, making it difficult to efficiently optimize in complex parameter spaces; the estimation of Fisher information matrices is typically performed offline in a single calculation, failing to dynamically reflect the latest changes in model parameters during training; and existing fusion strategies lack organic integration with retraining steps, resulting in limited and easily stagnant model performance improvements. These technical shortcomings are particularly prominent in practical applications such as facial recognition access control systems, financial voiceprint authentication, and IoT malware detection, where existing methods, due to these limitations, cannot provide effective dynamic protection.

[0004] Therefore, there is an urgent need for a technical solution to address the aforementioned problems. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention proposes a neural network model fusion method based on adversarial robust interpolation and Bayesian optimization, thereby resolving the issues present in the prior art.

[0006] Firstly, to achieve the above objectives, this invention provides a method for fusing neural network models based on adversarial robust interpolation and Bayesian optimization, comprising the following steps:

[0007] S1. Divide the dataset into a base set, a validation set, and a test set, and generate corresponding adversarial examples for each set;

[0008] S2. Use the original samples of the base set to perform normal training to obtain normal model parameters, and use the adversarial samples of the base set to perform adversarial training to obtain adversarial model parameters.

[0009] S3. Perform linear interpolation on the normal model parameters and the adversarial model parameters to generate an interpolation model, and calculate the empirical Fisher information matrix of the interpolation model.

[0010] S4. Using the adversarial test accuracy of the fusion model on the validation set adversarial examples as the objective function, a Bayesian optimization framework is used to search for the optimal interpolation parameters, wherein the empirical Fisher information matrix is ​​dynamically recalculated and the fusion model parameters are updated in each iteration.

[0011] S5. The optimal fusion parameters are fed back to the normal training model and the adversarial training model for retraining, and S3 and S4 are repeated until the preset number of iterations is reached to obtain the final fusion model.

[0012] S6. Evaluate the standard accuracy and robust accuracy of the final fusion model using the test set and adversarial test set.

[0013] Optionally, in S1, the process of dataset partitioning and adversarial example generation includes:

[0014] The training set is divided into a base set and a validation set. The base set consists of the main part of the training set samples, and the validation set consists of the remaining part of the training set samples. The test set is independent of the training set.

[0015] For each sample in the base set, validation set, and test set, a white-box adversarial attack is applied to generate adversarial samples, where the adversarial attack uses a homogeneous perturbation method.

[0016] Optionally, in S2, the normal training and adversarial training processes include:

[0017] Based on shared initialization parameters, training is performed using the original samples and adversarial samples of the base set, respectively;

[0018] Normal training obtains normal model parameters by minimizing the supervised loss function on the original samples;

[0019] Adversarial training obtains the parameters of the adversarial model by solving a minimization-maximization problem and maintaining classification accuracy while adding perturbations.

[0020] Optionally, in S3, the process of generating the interpolation model and calculating the empirical Fisher information matrix includes:

[0021] Linear interpolation is performed on the parameters of the normal model and the adversarial model to generate a first interpolation model and a second interpolation model;

[0022] For each interpolation model, the diagonal approximation of the empirical Fisher information matrix is ​​calculated on the base set and obtained by calculating the outer product expectation of the log-likelihood gradients of the model output and retaining the diagonal elements.

[0023] Optionally, in S4, the process of using Bayesian optimization to search for the optimal interpolation parameters includes:

[0024] Define the task vectors for the normal model and the adversarial model relative to the initialization parameters;

[0025] The parameters of the fusion model are calculated based on the interpolation coefficients and the empirical Fisher information matrix.

[0026] With the accuracy of the fusion model on the adversarial examples of the validation set as the objective function, a Bayesian optimization framework is used to search for interpolation coefficients. The Bayesian optimization uses a Gaussian process as a surrogate model and the expected improvement as the acquisition function, and dynamically recalculates the empirical Fisher information matrix in each iteration.

[0027] Optionally, in S5, the backhaul retraining process includes:

[0028] The optimal fusion parameters are passed as new initialization parameters to both the normal training model and the adversarial training model.

[0029] Perform a small number of rounds of normal training and adversarial training respectively to obtain updated normal model parameters and adversarial model parameters;

[0030] S3 and S4 are repeated based on the updated model parameters to form an iterative optimization process.

[0031] Secondly, the present invention also provides a neural network model fusion system based on adversarial robust interpolation and Bayesian optimization, for implementing a neural network model fusion method based on adversarial robust interpolation and Bayesian optimization, the system comprising:

[0032] The data partitioning and adversarial example generation module is used to partition the dataset into a base set, a validation set, and a test set, and generate corresponding adversarial examples for each set.

[0033] The baseline model training module is used to perform normal training using the original samples of the base set to obtain normal model parameters, and to perform adversarial training using adversarial samples of the base set to obtain adversarial model parameters.

[0034] The interpolation model and Fisher information calculation module is used to perform linear interpolation on the parameters of the normal model and the adversarial model, generate the interpolation model, and calculate the empirical Fisher information matrix of the interpolation model.

[0035] The Bayesian optimization search module is used to search for the optimal interpolation parameters using the adversarial test accuracy of the fusion model on the validation set adversarial examples as the objective function. In each iteration, the empirical Fisher information matrix is ​​dynamically recalculated and the fusion model parameters are updated.

[0036] The back-to-training module is used to back-to-train the optimal fusion parameters to the normal training model and the adversarial training model for retraining, and to repeatedly execute the functions of the interpolation model, Fisher information calculation module and Bayesian optimization search module until the preset number of iterations is reached to obtain the final fusion model.

[0037] The model evaluation module is used to evaluate the standard accuracy and robust accuracy of the final fusion model using test sets and adversarial test sets.

[0038] Thirdly, the present invention also provides a computer terminal device, comprising:

[0039] One or more processors;

[0040] A memory, coupled to the processor, for storing one or more programs;

[0041] When the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the neural network model fusion method based on adversarial robust interpolation and Bayesian optimization in the first aspect described above.

[0042] Fourthly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the steps of the neural network model fusion method based on adversarial robust interpolation and Bayesian optimization described in the first aspect above.

[0043] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the neural network model fusion method based on adversarial robust interpolation and Bayesian optimization described in the first aspect above.

[0044] Compared with the prior art, the present invention has the following advantages and technical effects:

[0045] This invention provides a neural network model fusion method based on adversarial robust interpolation and Bayesian optimization. This method effectively fuses complementary information in the parameter space of normally trained and adversarially trained models through adversarial robust interpolation, significantly improving robustness under adversarial perturbations while maintaining standard accuracy on clean data. It employs an empirical Fisher information matrix to guide interpolation and combines it with Bayesian optimization to dynamically search for optimal interpolation coefficients, overcoming the limitations of manually set or grid search parameter selection, and greatly improving fusion efficiency and scientific rigor. By backfeeding the optimal fusion parameters to the baseline model for iterative retraining and fine-tuning, a closed-loop optimization process is formed, avoiding performance stagnation or degradation after a single fusion. This method solves the problems in existing technologies where it is difficult to balance standard model accuracy and adversarial robustness, interpolation parameter selection relies on experience, and fusion strategies lack dynamic optimization. In practical security scenarios such as facial recognition access control systems, financial voiceprint authentication, and IoT malware detection, it can significantly enhance the stability and protection capabilities of models against white-box adversarial attacks, reduce security risks, and improve system reliability. Attached Figure Description

[0046] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0047] Figure 1 This is a flowchart of a method according to an embodiment of the present invention. Detailed Implementation

[0048] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0049] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0050] Example 1

[0051] This embodiment provides a method for fusing neural network models based on adversarial robust interpolation and Bayesian optimization, including:

[0052] S1. Divide the dataset into a base set, a validation set, and a test set, and generate corresponding adversarial examples for each set;

[0053] S2. Use the original samples of the base set to perform normal training to obtain normal model parameters, and use the adversarial samples of the base set to perform adversarial training to obtain adversarial model parameters.

[0054] S3. Perform linear interpolation on the normal model parameters and the adversarial model parameters to generate an interpolation model, and calculate the empirical Fisher information matrix of the interpolation model.

[0055] S4. Using the adversarial test accuracy of the fusion model on the validation set adversarial examples as the objective function, a Bayesian optimization framework is used to search for the optimal interpolation parameters, wherein the empirical Fisher information matrix is ​​dynamically recalculated and the fusion model parameters are updated in each iteration.

[0056] S5. The optimal fusion parameters are fed back to the normal training model and the adversarial training model for retraining, and S3 and S4 are repeated until the preset number of iterations is reached to obtain the final fusion model.

[0057] S6. Evaluate the standard accuracy and robust accuracy of the final fusion model using the test set and adversarial test set.

[0058] Specifically, it includes the following:

[0059] S1: Given the RML2016.10a dataset and the CNN1D convolutional neural network model, the dataset is divided into a base set, a test set, and a validation set. The training set consists of a large batch of labeled data for supervised training, the test set consists of a small batch of labeled data for performance evaluation, and the validation set is used for parameter optimization. Then, isomorphic PGD white-box adversarial attacks are performed on all three sets to obtain corresponding adversarial examples.

[0060] In this embodiment, given the dataset RML2016.10a, the neural network model is determined to be a one-dimensional convolutional neural network CNN1D, and the implementation field is radio modulation recognition.

[0061] The RML2016.10a dataset contains 11 modulation categories: 8BPSK, QPSK, 8PSK, 16QAM, 64QAM, BFSK, CPFSK, and PAM4, as well as 3 analog modulations: WB-FM, AM-SSB, and AM-DSB. Each sample is an IQ dual-channel dataset, and the modulation categories are uniformly distributed within the range of -20dB to 18dB. The dataset is divided into training and test sets in a 4:1 ratio, with the training set containing 176,000 signals and the test set containing 44,000 signals, both with the same number of modulation categories. The CNN1D model consists of 6 residual blocks. Each residual block contains 5 convolutional layers and 1 pooling layer. For RML2016.10a, this is split into training and test sets. For each signal-to-noise ratio in each category, 90% of the data (158,400 samples) is randomly selected as the base set. The remaining 10% (17,600 samples) is used as the validation set, and the test set consists of 44,000 samples. Then, for the samples in the base, validation, and test sets, a homogeneous PGD white-box adversarial attack is used to generate corresponding adversarial example sets. This ultimately yields the reference... Figure 1 On the left side of the middle.

[0062] S2: Baseline model normal training and adversarial training. Using the original samples and adversarial samples from the training set obtained in S1, normal training and adversarial training are performed respectively. Normal training uses only the original samples for conventional supervised training to obtain normal model parameters; adversarial training uses adversarial samples for adversarial training to obtain adversarial model parameters.

[0063] In this embodiment, based on shared initialization parameters, normal training and adversarial training are performed using original samples and adversarial samples respectively in the normal training and adversarial training cycles to obtain normal training model parameters and adversarial training model parameters.

[0064] The specific optimization methods for normal training and adversarial training are obtained as shown in formulas (1) and (2). Through the above training process, the parameters of the normal model and the adversarial model after one round of training are obtained respectively, which prepares for subsequent interpolation and Fisher information matrix calculation. Finally, the reference model is obtained. Figure 1 The first part of the process involves obtaining and saving the parameters for both the normal and adversarial models.

[0065] S3: Generate the interpolation model and calculate the empirical Fisher information matrix. Linear interpolation is performed on the normal model parameters and adversarial model parameters obtained in S2 to obtain the first interpolation model and the second interpolation model. The log-likelihood gradient of each interpolation model is calculated, and then the outer product operation is performed on the gradients and the expectation is calculated to obtain a complete Fisher information matrix. The diagonal elements of this matrix are retained, finally yielding the diagonal form of the empirical Fisher information matrix.

[0066] In this embodiment, using the obtained normal training model parameters and adversarial training model parameters, the interpolation coefficients are first initialized. Then, linear interpolation is performed using the normal training model parameters and adversarial training model parameters to obtain the first interpolation model and the second interpolation model. Subsequently, for each interpolation model, the diagonal elements of its empirical Fisher information matrix are estimated on the original samples and adversarial samples of the base set, respectively. In actual computation, to reduce computational complexity and storage overhead, the diagonal approximation of the matrix is ​​usually taken, and the model's prediction distribution is reused to obtain the empirical Fisher diagonal vector.

[0067] The specific methods for obtaining the first and second interpolation models are shown in formulas (3) and (4). The specific methods for obtaining the empirical Fisher diagonal vector are shown in formulas (5) to (7). The formulas are referenced. Figure 1 The second part involves obtaining and saving the empirical Fisher information matrix and interpolation parameters.

[0068] S4: Model fusion and Bayesian optimization search for optimal interpolation parameters. The fusion parameters are calculated using the empirical Fisher information matrix obtained in S3 to obtain the fusion model. Using the adversarial test accuracy of the fusion model on the validation set obtained in S1 as the objective function, a Bayesian optimization framework is employed to search for the optimal interpolation parameters. During the Bayesian optimization process, the expected improvement is selected as the acquisition function. In each iteration, the empirical Fisher information matrix is ​​dynamically recalculated at the candidate interpolation parameter values, and the fusion model parameters are updated.

[0069] In this embodiment, after obtaining the empirical Fisher information matrix, task vectors for the normal training model and the adversarial training model relative to the initial parameters are first defined. For the interpolation coefficient vector, each element corresponds to an interpolation coefficient of a model. Given the coefficients, the diagonal Fisher information of the corresponding interpolation model is estimated at the interpolation position. Then, the model is fused using the diagonal Fisher information and the task vector. Furthermore, the empirical Fisher information needs to be re-estimated and the fused model parameters updated each time at the candidate interpolation coefficients. The optimal interpolation coefficients are searched using Bayesian optimization with the goal of maximizing the adversarial robustness of the fused model, i.e., the accuracy of the fused model is evaluated on the adversarial examples of the validation set. In Bayesian optimization, sampling is first performed... For each of the randomly assigned interpolation parameters, interpolation point construction, diagonal empirical Fisher estimation, model fusion computation, and black-box objective function evaluation on adversarial examples in the validation set are performed. Subsequently, a Gaussian process is used to fit the interpolation parameters and objective function to obtain the posterior mean and variance. The acquisition function uses Expected Improvement (EI), which selects the currently observed optimal objective value. The next evaluation point is selected to maximize the expected improvement relative to the current best value in the posterior distribution. At each new point, the diagonal empirical Fisher at the corresponding interpolation point is dynamically re-estimated, and the fusion model and its objective function for the next selected evaluation point are calculated. New observations are added to the Gaussian process, and this iteration is repeated until the maximum number of iterations is reached.

[0070] The specific method for obtaining the fusion model is shown in formula (8), and the specific method for searching the optimal interpolation point using Bayesian optimization is shown in formulas (9) and (10). The formulas are referenced for further information. Figure 1 The third part involves obtaining and saving the fusion model parameters and optimal interpolation parameters.

[0071] S5: Retraining and Performance Evaluation. The fusion model parameters obtained in S4 are fed back to the normal training model and the adversarial training model, respectively, and used as initial parameters for further training. S3 and S4 are repeated until the preset maximum number of iterations is reached, finally outputting the fusion model with the best adversarial robustness. Then, the accuracy of the final fusion model is evaluated using an unused test set and adversarial examples, and the robust accuracy and standard accuracy are recorded as model performance metrics.

[0072] In this embodiment, the obtained optimal fusion parameters are used as new initialization points and passed to both the normally trained model and the adversarially trained model. Then, a few rounds of normal training are performed using the optimal fusion parameters to obtain new normal model parameters, and a few rounds of adversarial training are performed using the same optimal fusion parameters to obtain new adversarial model parameters. This completes one round of backtracking to fine-tuning. After each round of fine-tuning, based on the new normal model parameters and adversarial model parameters, the interpolation model is re-generated, the empirical Fisher information matrix is ​​calculated, and the model is fused. Bayesian optimization is then used to search for the optimal interpolation parameters, forming an iterative optimization process. The detection results in this embodiment are measured by the standard accuracy and robust accuracy calculated by the final fusion model on unseen test sets and adversarial test sets, respectively.

[0073] This embodiment presents a neural network model fusion method based on adversarial robust interpolation and Bayesian optimization. Compared to existing neural network model fusion methods, this invention introduces interpolation and Fisher information constraints in adversarial scenarios, enabling the model to fully utilize the complementary information of the normally trained model and the adversarial trained model during the fusion process. Simultaneously, Bayesian optimization replaces the manual selection of interpolation coefficients, automatically searching for optimal fusion parameters, significantly improving the robustness and generalization ability of the fusion model on adversarial examples. These two aspects allow model parameters under different training strategies to be effectively coordinated and complemented in the feature space, thereby obtaining a fusion model with superior performance.

[0074] Subsequently, the optimal fusion parameters are fed back to the normal training model and the adversarial training model through a backpropagation retraining strategy. In each iteration, the robustness and accuracy of the two baseline models are gradually improved, effectively avoiding the problems of manual parameter tuning and training instability. Therefore, the fusion model can maintain high accuracy and robustness when facing unknown adversarial perturbations, ultimately resulting in a neural network fusion model that performs well in adversarial environments.

[0075] As one implementation method in this embodiment, in S1, the process of dataset partitioning and adversarial example generation includes:

[0076] The training set is divided into a base set and a validation set. The base set consists of the main part of the training set samples, and the validation set consists of the remaining part of the training set samples. The test set is independent of the training set.

[0077] For each sample in the base set, validation set, and test set, a white-box adversarial attack is applied to generate adversarial samples, where the adversarial attack uses a homogeneous perturbation method.

[0078] Furthermore, in step S1, a dataset is typically split into a training set and a test set, represented as follows: and ,in Then the training set 90% as the base set , represented as Use the remaining 10% of the training set As a verification set , represented as test set , represented as ,in Indicates a sample, This represents the label. Then, for the sample... , and In other words, using isomorphic white-box adversarial attacks to generate the corresponding set of adversarial examples is represented as follows: , and ,in This indicates resistance to disturbances. , Indicates the batch size.

[0079] As one implementation method in this embodiment, in S2, the process of normal training and adversarial training includes:

[0080] Based on shared initialization parameters, training is performed using the original samples and adversarial samples of the base set, respectively;

[0081] Normal training obtains normal model parameters by minimizing the supervised loss function on the original samples;

[0082] Adversarial training obtains the parameters of the adversarial model by solving a minimization-maximization problem and maintaining classification accuracy while adding perturbations.

[0083] Furthermore, in step S2, the process of normal training and adversarial training of the baseline model is as follows:

[0084] In the normal training and adversarial training cycles, the shared initialization parameters are set as follows: Based on these initialization parameters, the original samples were used respectively. and adversarial examples Perform normal training and adversarial training to obtain the parameters of the normal training model. and the parameters of the adversarial training model The objective of normal training is to minimize the base set. The supervised loss is used, therefore, the optimization formula for normal training is as follows:

[0085] (1)

[0086] In the formula, This represents the number of samples in the base set. It is the cross-entropy loss function. The goal of adversarial training is a minimization-maximization problem, maintaining classification accuracy even when perturbations are added to the input. Therefore, the optimization formula for adversarial training is as follows:

[0087] (2)

[0088] In the formula, For norm type, To constrain the amplitude of the disturbance, It indicates resistance to disturbances.

[0089] As one implementation method in this embodiment, in S3, the process of generating the interpolation model and calculating the empirical Fisher information matrix includes:

[0090] Linear interpolation is performed on the parameters of the normal model and the adversarial model to generate a first interpolation model and a second interpolation model;

[0091] For each interpolation model, the diagonal approximation of the empirical Fisher information matrix is ​​calculated on the base set and obtained by calculating the outer product expectation of the log-likelihood gradients of the model output and retaining the diagonal elements.

[0092] Furthermore, in step S3, the process of generating the interpolation model and calculating the empirical Fisher information matrix is ​​as follows:

[0093] Obtain the normal training model parameters in S2 and the parameters of the adversarial training model Then, first initialize the interpolation coefficients. ,use and Linear interpolation is performed to obtain the first interpolation model and the second interpolation model, as shown in the following formulas:

[0094] (3)

[0095] (4)

[0096] In the formula, and These are the interpolation model parameters. Then, for each interpolation model, on the base set... and The diagonal elements of the empirical Fisher information matrix are estimated. The theoretical formula for the empirical Fisher information matrix is ​​as follows:

[0097] (5)

[0098] In the formula, This represents the number of samples in the base set. This represents the class probability distribution of the model output. This is the cross-entropy loss function. In practical calculations, to reduce computational complexity and storage overhead, a diagonal approximation of this matrix is ​​usually taken, and the model's predicted distribution is reused, resulting in the following formula for the empirical Fisher diagonal vector:

[0099] (6)

[0100] In the formula, It is a diagonal function. Its first... The formula for calculating each component is as follows:

[0101] (7)

[0102] In the formula, This represents the cross-entropy loss gradient corresponding to the sample. Predict labels for the model.

[0103] As one implementation method in this embodiment, in S4, the process of searching for the optimal interpolation parameters using Bayesian optimization includes:

[0104] Define the task vectors for the normal model and the adversarial model relative to the initialization parameters;

[0105] The parameters of the fusion model are calculated based on the interpolation coefficients and the empirical Fisher information matrix.

[0106] With the accuracy of the fusion model on the adversarial examples of the validation set as the objective function, a Bayesian optimization framework is used to search for interpolation coefficients. The Bayesian optimization uses a Gaussian process as a surrogate model and the expected improvement as the acquisition function, and dynamically recalculates the empirical Fisher information matrix in each iteration.

[0107] In step S4, the process of model fusion and Bayesian optimization to search for the optimal interpolation parameters is as follows:

[0108] After obtaining the empirical Fisher information matrix in S3, define the normal training model and the adversarial training model relative to the initial parameters. The task vector is and For the interpolation coefficient vector Each element corresponds to an interpolation coefficient of a model. Given coefficients... Under these conditions, the diagonal Fisher information of the corresponding interpolation model is first estimated at the interpolation position. Therefore, the formula for model fusion is as follows:

[0109] (8)

[0110] The empirical Fisher information needs to be re-estimated at each candidate interpolation coefficient. And update the fusion model parameters The optimal interpolation coefficients are searched using Bayesian optimization to maximize the adversarial robustness of the fusion model. Therefore, the formula for the black-box objective function in the Bayesian optimization framework is as follows:

[0111] (9)

[0112] That is, to evaluate the accuracy of the fusion model on adversarial examples in the validation set. In Bayesian optimization, sampling is first performed. A random For each Perform interpolation point construction, diagonal empirical Fisher estimation, model fusion computation, and in Upper assessment Subsequently, a Gaussian process was used for fitting. And obtain the posterior mean With variance The acquisition function uses Expected Improvement (EI), and the EI formula is as follows:

[0113] (10)

[0114] In the formula, This represents the currently observed optimal target value. The next evaluation point will be selected as... At each new point, it is necessary to dynamically re-estimate the diagonal empirical Fisher at the corresponding interpolation point and calculate the fusion model and its... New observations are added to the Gaussian process, and the iterations are repeated until the maximum number of iterations is reached.

[0115] As one implementation method in this embodiment, in S5, the backhaul retraining process includes:

[0116] The optimal fusion parameters are passed as new initialization parameters to both the normal training model and the adversarial training model.

[0117] Perform a small number of rounds of normal training and adversarial training respectively to obtain updated normal model parameters and adversarial model parameters;

[0118] S3 and S4 are repeated based on the updated model parameters to form an iterative optimization process.

[0119] In step S5, the process of backhauling for retraining and performance evaluation is as follows:

[0120] During the retraining phase, the optimal fusion parameters obtained in S4 are used. As new initialization points, they are passed to the normally trained model and the adversarial trained model respectively, and assigned values ​​to them. and Subsequently Normal training with a small number of epochs yields the following results. ,by Obtained through a small number of epochs of adversarial training This completes one round of feedback to the fine-tuning round. After each round of fine-tuning, based on the new... and Repeat steps S3 and S4 to form an iterative optimization process. Iterate continuously until the preset maximum number of iterations is reached.

[0121] In the final performance evaluation phase, the final fusion model is used. In the unseen test set and adversarial test set The standard accuracy and robust accuracy are calculated above. The formula for standard accuracy is as follows:

[0122] (11)

[0123] The formula for robust accuracy is as follows:

[0124] (12)

[0125] in Generated using white-box attack methods. This should be logged and reported throughout the entire training and fusion process. and Two types of metrics, and the Fisher information matrix for each iteration. interpolation coefficients This is so that a comprehensive analysis and comparison can be conducted.

[0126] Based on this, the present invention provides a method for fusing neural network models based on adversarial robust interpolation and Bayesian optimization. The beneficial effects of the present invention are as follows:

[0127] By introducing adversarial robust interpolation, this invention fully integrates the complementary information of the normally trained model and the adversarial trained model in the parameter space. This preserves the standard accuracy on clean data while significantly improving robustness under adversarial perturbations. Instead of manually setting or grid-searching interpolation parameters, it uses an empirical Fisher information matrix to guide interpolation and dynamically searches for the optimal interpolation coefficients within a Bayesian optimization framework, greatly improving the scientific rigor and efficiency of parameter selection in the fusion model. Simultaneously, the optimal fusion parameters are fed back to the baseline model for retraining and fine-tuning, forming a closed-loop iterative optimization that effectively avoids performance stagnation or degradation after a single fusion. This invention effectively solves the technical challenges of existing models, such as the difficulty in balancing standard accuracy and adversarial robustness, the reliance on experience in interpolation parameter selection, and the difficulty in dynamically optimizing fusion strategies. This makes the proposed neural network model fusion method more robust and practical in the face of adversarial attacks. Furthermore, this invention has significant industrial application value. In neural network systems deployed in real-world security environments such as access control, voiceprint authentication, and malware detection, the proposed method can significantly improve the stability and robustness of the model when facing white-box adversarial attacks, ensuring that the system can maintain high recognition accuracy under attack conditions and reducing security risks caused by model misleading, demonstrating outstanding technological progress and engineering practicality.

[0128] Example 2

[0129] In this embodiment, a computer terminal device is provided, including:

[0130] One or more processors;

[0131] A memory, coupled to the processor, for storing one or more programs;

[0132] When the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the above-described method for fusing neural network models based on adversarial robust interpolation and Bayesian optimization.

[0133] In this embodiment, a computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the above-described method for fusing neural network models based on adversarial robust interpolation and Bayesian optimization.

[0134] In this embodiment, an electronic device is also provided, including a memory and a processor. The memory stores a computer program, and the processor is configured to run the computer program to perform the steps of the above-described neural network model fusion method based on adversarial robust interpolation and Bayesian optimization.

[0135] In this embodiment, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps of the above-described method for fusing neural network models based on adversarial robust interpolation and Bayesian optimization.

[0136] The aforementioned program can run on a processor or be stored in memory (or a computer-readable medium). Computer-readable media includes both permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0137] These computer programs may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes can be implemented by different modules for different steps.

[0138] This embodiment provides such a device or system. The system, referred to as a neural network model fusion system based on adversarial robust interpolation and Bayesian optimization, includes:

[0139] The data partitioning and adversarial example generation module is used to partition the dataset into a base set, a validation set, and a test set, and generate corresponding adversarial examples for each set.

[0140] The baseline model training module is used to perform normal training using the original samples of the base set to obtain normal model parameters, and to perform adversarial training using adversarial samples of the base set to obtain adversarial model parameters.

[0141] The interpolation model and Fisher information calculation module is used to perform linear interpolation on the parameters of the normal model and the adversarial model, generate the interpolation model, and calculate the empirical Fisher information matrix of the interpolation model.

[0142] The Bayesian optimization search module is used to search for the optimal interpolation parameters using the adversarial test accuracy of the fusion model on the validation set adversarial examples as the objective function. In each iteration, the empirical Fisher information matrix is ​​dynamically recalculated and the fusion model parameters are updated.

[0143] The back-to-training module is used to back-to-train the optimal fusion parameters to the normal training model and the adversarial training model for retraining, and to repeatedly execute the functions of the interpolation model, Fisher information calculation module and Bayesian optimization search module until the preset number of iterations is reached to obtain the final fusion model.

[0144] The model evaluation module is used to evaluate the standard accuracy and robust accuracy of the final fusion model using test sets and adversarial test sets.

[0145] As one implementation method in this embodiment, the data partitioning and adversarial example generation module includes:

[0146] The data partitioning unit is used to divide the training set into a base set and a validation set. The base set consists of the main part of the training set samples, the validation set consists of the remaining part of the training set samples, and the test set is independent of the training set.

[0147] The adversarial example generation unit is used to apply white-box adversarial attacks to each sample in the base set, validation set, and test set to generate adversarial examples, where the adversarial attacks adopt a homogeneous perturbation method.

[0148] As one implementation method in this embodiment, the baseline model training module includes:

[0149] The normal training unit is used to perform training using the original samples of the base set based on shared initialization parameters, and obtains normal model parameters by minimizing the supervised loss function on the original samples;

[0150] The adversarial training unit is used to perform training using adversarial examples from the base set on the basis of shared initialization parameters. By solving a minimization-maximization problem, it maintains classification accuracy while adding perturbations and obtains the parameters of the adversarial model.

[0151] As one implementation method in this embodiment, the interpolation model and Fisher information calculation module include:

[0152] The interpolation model generation unit is used to perform linear interpolation on the parameters of the normal model and the adversarial model to generate a first interpolation model and a second interpolation model.

[0153] Fisher information computation unit is used to compute the diagonal approximation of the empirical Fisher information matrix on the base set for each interpolation model, obtained by computing the outer product expectation of the log-likelihood gradients of the model output and retaining the diagonal elements.

[0154] As one implementation method in this embodiment, the Bayesian optimization search module includes:

[0155] The task vector definition unit is used to define the task vectors of the normal model and the adversarial model relative to the initialization parameters;

[0156] The fusion model calculation unit is used to calculate the fusion model parameters based on the interpolation coefficients and the empirical Fisher information matrix;

[0157] An optimized search unit is used to search for interpolation coefficients with the accuracy of the fusion model on the adversarial examples of the validation set as the objective function. The Bayesian optimization framework is used to search for interpolation coefficients, where the Bayesian optimization uses a Gaussian process as a surrogate model and the expected improvement as the acquisition function, and the empirical Fisher information matrix is ​​dynamically recalculated in each iteration.

[0158] As one implementation method in this embodiment, the backhaul retraining module includes:

[0159] The parameter feedback unit is used to pass the optimal fusion parameters as new initialization parameters to the normal training model and the adversarial training model;

[0160] Fine-tuning the training unit is used to perform a small number of rounds of normal training and adversarial training to obtain updated normal model parameters and adversarial model parameters.

[0161] The iterative control unit is used to repeatedly execute the functions of the interpolation model, Fisher information calculation module, and Bayesian optimization search module based on the updated model parameters, forming an iterative optimization process.

[0162] The system or apparatus is used to implement the functions of the methods in the above embodiments. Each module in the system or apparatus corresponds to each step in the method, as has been described in the method and will not be repeated here.

[0163] The above implementation method solves the problem of integrating neural network models based on adversarial robust interpolation and Bayesian optimization in related technologies, thereby ensuring that the problems existing in the prior art are resolved.

[0164] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A neural network model fusion method based on adversarial robust interpolation and Bayesian optimization, characterized in that, Includes the following steps: S1. Divide the dataset into a base set, a validation set, and a test set, and generate corresponding adversarial examples for each set; S2. Use the original samples of the base set to perform normal training to obtain normal model parameters, and use the adversarial samples of the base set to perform adversarial training to obtain adversarial model parameters. S3. Perform linear interpolation on the normal model parameters and the adversarial model parameters to generate an interpolation model, and calculate the empirical Fisher information matrix of the interpolation model. S4. Using the adversarial test accuracy of the fusion model on the validation set adversarial examples as the objective function, a Bayesian optimization framework is used to search for the optimal interpolation parameters, wherein the empirical Fisher information matrix is ​​dynamically recalculated and the fusion model parameters are updated in each iteration. S5. The optimal fusion parameters are fed back to the normal training model and the adversarial training model for retraining, and S3 and S4 are repeated until the preset number of iterations is reached to obtain the final fusion model. S6. Evaluate the standard accuracy and robust accuracy of the final fusion model using the test set and adversarial test set; In S5, the backhaul retraining process includes: The optimal fusion parameters are passed as new initialization parameters to both the normal training model and the adversarial training model. Perform a small number of rounds of normal training and adversarial training respectively to obtain updated normal model parameters and adversarial model parameters; S3 and S4 are repeated based on the updated model parameters to form an iterative optimization process.

2. The method of claim 1, wherein, In S1, the process of dataset partitioning and adversarial example generation includes: The training set is divided into a base set and a validation set. The base set consists of the main part of the training set samples, and the validation set consists of the remaining part of the training set samples. The test set is independent of the training set. For each sample in the base set, validation set, and test set, a white-box adversarial attack is applied to generate adversarial samples, where the adversarial attack uses a homogeneous perturbation method.

3. The method of claim 1, wherein, In S2, the normal training and adversarial training processes include: Based on shared initialization parameters, training is performed using the original samples and adversarial samples of the base set, respectively; Normal training obtains normal model parameters by minimizing the supervised loss function on the original samples; Adversarial training obtains the parameters of the adversarial model by solving a minimization-maximization problem and maintaining classification accuracy while adding perturbations.

4. The method of claim 1, wherein, In S3, the process of generating the interpolation model and calculating the empirical Fisher information matrix includes: Linear interpolation is performed on the parameters of the normal model and the adversarial model to generate a first interpolation model and a second interpolation model; For each interpolation model, the diagonal approximation of the empirical Fisher information matrix is ​​calculated on the base set and obtained by calculating the outer product expectation of the log-likelihood gradients of the model output and retaining the diagonal elements.

5. The method of claim 1, wherein, In S4, the process of searching for the optimal interpolation parameters using Bayesian optimization includes: Define the task vectors for the normal model and the adversarial model relative to the initialization parameters; The parameters of the fusion model are calculated based on the interpolation coefficients and the empirical Fisher information matrix. With the accuracy of the fusion model on the adversarial examples of the validation set as the objective function, a Bayesian optimization framework is used to search for interpolation coefficients. The Bayesian optimization uses a Gaussian process as a surrogate model and the expected improvement as the acquisition function, and dynamically recalculates the empirical Fisher information matrix in each iteration.

6. A neural network model fusion system based on adversarial robust interpolation and Bayesian optimization, characterized in that, The system for implementing the method of any one of claims 1-5 comprises: The data partitioning and adversarial example generation module is used to partition the dataset into a base set, a validation set, and a test set, and generate corresponding adversarial examples for each set. The baseline model training module is used to perform normal training using the original samples of the base set to obtain normal model parameters, and to perform adversarial training using adversarial samples of the base set to obtain adversarial model parameters. The interpolation model and Fisher information calculation module is used to perform linear interpolation on the parameters of the normal model and the adversarial model, generate the interpolation model, and calculate the empirical Fisher information matrix of the interpolation model. The Bayesian optimization search module is used to search for the optimal interpolation parameters using the adversarial test accuracy of the fusion model on the validation set adversarial examples as the objective function. In each iteration, the empirical Fisher information matrix is ​​dynamically recalculated and the fusion model parameters are updated. The back-to-training module is used to back-to-train the optimal fusion parameters to the normal training model and the adversarial training model for retraining, and to repeatedly execute the functions of the interpolation model, Fisher information calculation module and Bayesian optimization search module until the preset number of iterations is reached to obtain the final fusion model. The model evaluation module is used to evaluate the standard accuracy and robust accuracy of the final fusion model using test sets and adversarial test sets.

7. A computer terminal device, characterized by include: One or more processors; A memory, coupled to the processor, for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors perform the steps of the method as described in any one of claims 1-5.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-5.

9. A computer program product comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-5.

Images