Picture data deep learning model robustness enhancement method based on mutation operator

Abstract

The invention discloses a picture data deep learning model robustness enhancement method based on a mutation operator, and the method comprises the following steps: applying disturbance to an input image through a data layer mutation operator, and generating mutation data containing natural scene disturbance; applying structure or parameter disturbance to the original deep learning model by using a model layer mutation operator to generate a plurality of mutation models; respectively inputting the variation data into the original model and each variation model for cross validation, and screening out error-prone samples based on loss differences output by the models; adding the screened error-prone samples into a training set, and retraining the original model to obtain a robustness-enhanced deep learning model; according to the method, the model robustness is systematically improved through a collaborative disturbance mechanism of the data layer and the model layer, and meanwhile, the training resource consumption is reduced by adopting a dynamic screening strategy.

Description

Technical Field

[0001] This invention relates to the field of structural vulnerability technology, specifically to a method for enhancing the robustness of deep learning models for image data based on mutation operators. Background Technology

[0002] With the widespread application of deep learning technology in fields such as image recognition and object detection, image processing neural networks have demonstrated powerful learning capabilities and practical application potential. However, in practical applications, neural network models often face the problem of small perturbations from natural scenes, such as geometric transformations like rotation, translation, and size changes. These perturbations can lead to unstable prediction results or even misidentification, thus seriously affecting the robustness and reliability of the system.

[0003] Traditional solutions primarily rely on single-dimensional data augmentation methods, such as rotating, flipping, and cropping images. While these methods can expand the training dataset to some extent, they often only capture changes at the data level and fail to fully reflect the complex disturbances encountered by the model in real-world environments. Furthermore, existing technologies do not adequately address the inherent variability of the model itself, making it difficult to simulate the vulnerability caused by parameter fine-tuning or changes in network structure. Therefore, simply relying on a single strategy in data augmentation or model perturbation is insufficient to fully reveal and address the robustness issues of neural networks in real-world scenarios. Summary of the Invention

[0004] Purpose of the invention: The purpose of this invention is to provide a robustness enhancement method for deep learning models of image data based on mutation operators, and to solve the vulnerability of image processing neural networks when faced with natural perturbations.

[0005] Technical solution: The robustness enhancement method for deep learning models of image data based on mutation operators described in this invention includes the following steps:

[0006] (1) The data layer mutation operator is used to apply perturbation to the input image to generate mutated data containing natural scene perturbation;

[0007] (2) Apply structural or parameter perturbations to the original deep learning model using the model layer mutation operator to generate multiple mutated models;

[0008] (3) Input the variant data into the original model and each variant model for cross-validation, and screen out error-prone samples based on the difference in loss output by the model;

[0009] (4) Add the selected error-prone samples to the training set and retrain the original model to obtain a robust deep learning model.

[0010] Furthermore, in step (1), the data layer mutation operators are as follows: data duplication, data missing, noise injection, label replacement, rotation transformation, random translation, and size adjustment.

[0011] Furthermore, in step (2), the model layer mutation operators are as follows: random removal of layers, dynamic adjustment of batch size, random perturbation of learning rate, and change of training period.

[0012] Furthermore, in step (3), the screening of error-prone samples is as follows: for each sample, calculate the prediction loss under all variant data and variant model combinations; calculate the original loss of the sample on the original model and the unperturbed data; based on the difference between the average loss under all variant conditions and the original loss, determine whether the sample is an error-prone sample.

[0013] Furthermore, in step (4), the retraining is specifically as follows: error-prone samples are added to the training set with enhanced weights or secondary enhancements, and the model is optimized in a targeted manner to improve its stability under perturbation.

[0014] The robustness enhancement system for deep learning models of image data based on mutation operators described in this invention includes:

[0015] Data module: Used to apply perturbation to the input image using data layer mutation operators to generate mutated data containing perturbations of the natural scene;

[0016] Mutation module: Used to apply structural or parameter perturbations to the original deep learning model using model layer mutation operators to generate multiple mutated models;

[0017] Validation module: Used to cross-validate the mutated data with the original model and each mutated model, and to screen out error-prone samples based on the differences in the loss output of the models;

[0018] Training module: Used to add the selected error-prone samples to the training set, retrain the original model, and obtain a robust deep learning model.

[0019] Furthermore, in the data module, the specific data layer mutation operators are as follows: data duplication, data missing, noise injection, label replacement, rotation transformation, random translation, and size adjustment.

[0020] Furthermore, in the mutation module, the specific mutation operators for the model layer are as follows: random removal of layers, dynamic adjustment of batch size, random perturbation of learning rate, and change of training period.

[0021] Furthermore, in the verification module, the screening of error-prone samples is as follows: For each sample, the prediction loss under all mutated data and mutated model combinations is calculated; the original loss of the sample on the original model and the unperturbed data is calculated; based on the difference between the average loss under all mutated conditions and the original loss, it is determined whether the sample is an error-prone sample.

[0022] Furthermore, in the training module, retraining is specifically performed as follows: error-prone samples are added to the training set with enhanced weights or secondary enhancements to optimize the model in a targeted manner, thereby improving its stability under perturbation environments.

[0023] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: (1) Dual perturbation mechanism: The present invention not only uses a variety of mutation operators such as rotation, translation, size adjustment, and noise injection to generate small perturbation data in real scenes at the data level, but also simulates the vulnerability of the model itself at the model level through parameter random perturbation and structural mutation. This collaborative perturbation method of data and model can more comprehensively reveal the weak links of the model under natural perturbation than the traditional single data augmentation method. (2) Precise sample selection strategy: By comparing the loss function output of the original model and the mutated model under different perturbation conditions, the present invention can accurately identify those samples that are extremely sensitive to perturbation and are prone to incorrect prediction. This quantitative selection method based on loss difference ensures that error-prone samples are used preferentially for optimization during subsequent iterative training, further improving the robustness of the model. (3) Systematic robustness enhancement: Through the combination of data perturbation, model perturbation, sample selection and iterative enhancement, the present invention constructs a full-process robustness enhancement system. This systematic approach can maintain the high accuracy and stability of deep neural networks when faced with various perturbations in natural scenes, thereby significantly improving the reliability and adaptability of the model in practical applications. Attached Figure Description

[0024] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0025] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0026] like Figure 1 As shown, this embodiment of the invention introduces a mutation mechanism at both the data and model levels, forming a dual perturbation system. This systematically improves the robustness of the model in real-world application scenarios while reducing training resource consumption. The main technical solution and implementation steps are as follows:

[0027] Step 1: Perturb the input image using data layer mutation operators, including operations such as data duplication, data missing, noise perturbation, label error correction, rotation transformation, random translation, and size adjustment; including the following steps:

[0028] (11) Data duplication: By copying local areas of the image to generate artifacts, the duplication interference in the actual scene is simulated. If the original image is Then at a random position Insert a copy block at:

[0029] ;

[0030] in, , For the size of the copy block, It is an image of repeated data;

[0031] (12) Data missing: Randomly delete local areas of the image to simulate sensor failure or occlusion, with probability. Generate mask matrix ,according to:

[0032] ;

[0033] Generate an image with missing data, where It is Gaussian noise;

[0034] (13) Noise perturbation: Gaussian noise is used to superimpose normal distribution noise on the pixel values:

[0035] ;

[0036] in, Indicates the original image at position Pixel value at that location, This indicates that the mean is 0 and the variance is . Gaussian noise, This represents the standard deviation of noise, used to control the intensity of disturbances. This represents the new pixel value after adding noise;

[0037] (14) Labeling error: based on probability Replace the classification labels and test the model's robustness to mislabeling on the label set. Generate disturbance labels:

[0038] ;

[0039] in, The generated new label, Indicates uniformly distributed random sampling. Indicates the true label of the sample;

[0040] (15) Random translation: The position of the image on the two-dimensional plane is changed by generating random offsets and applying an affine transformation matrix. First, the horizontal translation amount is randomly selected within a specified range. and vertical translation Then generate the transformation matrix:

[0041] ;

[0042] This matrix applies to each pixel coordinate. Generate new coordinates Areas outside the original image area are filled with edge pixels;

[0043] (16) Rotation transformation: Based on the image center, generate random angles and calculate the size of the rotated image. First, randomly select the rotation angle. Then generate based on the rotation angle. The transformation matrix is ​​the center of rotation.

[0044] ;in, These are the coordinates of the center point of the original image.

[0045] Then, the new boundary is calculated for the rotated image:

[0046] ;

[0047] in, It is the width of the rotated image. It is the height of the rotated image. It is the width of the original image. It is the height of the original image;

[0048] (17) Random size adjustment: Change the image resolution by scaling factor and randomly select the scaling factor. The image is scaled proportionally, and the new image size is... .

[0049] Step 2: Adjust the parameters of the deep neural network using model layer mutation operators. These mutation operators include layer removal, batch size adjustment, dynamic learning rate adjustment, and training period modification. Specifically, this includes:

[0050] (21) Layer Removal: Randomly discard intermediate layers in the model to test the redundancy of the model structure. Layered neural networks, proportionally Randomly frozen layers:

[0051] ;

[0052] in, This represents the set of dynamically selected active layers in a neural network. Indicates from 1 to Random selection One unique element Indicates the actual number of activated layers;

[0053] (22) Batch size adjustment: Dynamically adjust the batch size during the training cycle. ,in Indicates the original batch size. This indicates the new batch size after adjustment. Indicates the adjustment factor. ;

[0054] (23) Dynamic adjustment of learning rate: During the model training process, the learning rate is adjusted periodically. Adjusting the learning rate :

[0055] ;

[0056] in, This represents the initial learning rate. Represents a uniformly distributed random number. Indicates the learning rate after perturbation;

[0057] (24) Training cycle changes: extending or shortening the training cycle to ,in Indicates the original training period. This indicates the revised learning period. Indicates the change factor.

[0058] Step 3: Input the mutated data into the original model and perform cross-validation with the mutated model, and filter error-prone samples based on the output difference. Detailed process:

[0059] (31) For each sample Predictions are made using all variable data and combinations of variable models, and the corresponding loss function values ​​are calculated. Let the loss function be... For the sample In the The first variant model and the second The loss under each data disturbance is:

[0060] ;

[0061] in, Representing the mutation model, Indicates the true label;

[0062] (32) Calculate the loss of the original model on the undisturbed data: ;

[0063] (33) Define the sample The average loss difference is used to characterize the performance changes of the sample under different perturbation conditions: ;

[0064] in, This indicates the number of data-level mutation operators. Indicates the number of model-level mutation operators, if A larger value indicates that the prediction result of the sample changes significantly when it is disturbed by data or model, that is, the original model has a high degree of instability or error susceptibility to the sample.

[0065] (34) The samples are screened based on the difference in average loss. First, a threshold is set. ,when At that time, it was considered that the sample The corresponding prediction results are relatively unstable and are marked as error-prone samples.

[0066] Step 4: Add the selected samples to the training set to retrain the original model, generating a robust enhanced model. Specifically, the selected error-prone samples are added to the training set for subsequent retraining or model fine-tuning. By strategically increasing the weights of these samples or performing additional data augmentation, the model can become more robust to natural perturbations, thereby improving overall model performance.

Claims

1. A method for enhancing the robustness of deep learning models for image data based on mutation operators, characterized in that, Includes the following steps: (1) The data layer mutation operator is used to apply perturbation to the input image to generate mutated data containing natural scene perturbation; (2) Apply structural or parameter perturbations to the original deep learning model using the model layer mutation operator to generate multiple mutated models; (3) Input the variant data into the original model and each variant model for cross-validation, and screen out error-prone samples based on the difference in loss output by the model; (4) Add the selected error-prone samples to the training set and retrain the original model to obtain a robust deep learning model.

2. The robustness enhancement method for deep learning models of image data based on mutation operators according to claim 1, characterized in that, In step (1), the data layer mutation operators are as follows: data duplication, data missing, noise injection, label replacement, rotation transformation, random translation, and size adjustment.

3. The robustness enhancement method for deep learning models of image data based on mutation operators according to claim 1, characterized in that, In step (2), the specific mutation operators of the model layer are as follows: random removal of layers, dynamic adjustment of batch size, random perturbation of learning rate, and change of training period.

4. The robustness enhancement method for deep learning models of image data based on mutation operators according to claim 1, characterized in that, In step (3), the specific steps for screening error-prone samples are as follows: For each sample, calculate the prediction loss under all variable data and variable model combinations; calculate the original loss of the sample on the original model and the unperturbed data. Based on the difference between the average loss under all variation conditions and the original loss, determine whether the sample is a fallible sample.

5. The robustness enhancement method for deep learning models of image data based on mutation operators according to claim 1, characterized in that, In step (4), the retraining is specifically as follows: error-prone samples are added to the training set with enhanced weights or secondary enhancements to optimize the model in a targeted manner.

6. A robustness enhancement system for deep learning models of image data based on mutation operators, characterized in that, include: Data module: Used to apply perturbation to the input image using data layer mutation operators to generate mutated data containing perturbations of the natural scene; Mutation module: Used to apply structural or parameter perturbations to the original deep learning model using model layer mutation operators to generate multiple mutated models; Validation module: Used to cross-validate the mutated data with the original model and each mutated model, and to screen out error-prone samples based on the differences in the loss output of the models; Training module: Used to add the selected error-prone samples to the training set, retrain the original model, and obtain a robust deep learning model.

7. The robustness enhancement system for deep learning models of image data based on mutation operators according to claim 6, characterized in that, In the data module, the specific data layer mutation operators are as follows: data duplication, data missing, noise injection, label permutation, rotation transformation, random translation, and size adjustment.

8. The robustness enhancement system for deep learning models of image data based on mutation operators according to claim 6, characterized in that, In the mutation module, the specific mutation operators for the model layer are as follows: random removal of layers, dynamic adjustment of batch size, random perturbation of learning rate, and change of training period.

9. The robustness enhancement system for deep learning models of image data based on mutation operators according to claim 6, characterized in that, In the validation module, the screening of error-prone samples is carried out as follows: For each sample, the prediction loss is calculated under all variant data and variant model combinations; the original loss of the sample on the original model and the unperturbed data is calculated. Based on the difference between the average loss under all variation conditions and the original loss, determine whether the sample is a fallible sample.

10. The robustness enhancement system for deep learning models of image data based on mutation operators according to claim 6, characterized in that, In the training module, retraining is performed as follows: error-prone samples are added to the training set with enhanced weights or secondary enhancements to optimize the model.

Images