A neural network collaborative robust training method based on joint perturbation optimization

Abstract

The application discloses a neural network collaborative robust training method based on joint disturbance optimization, and belongs to the technical field of artificial intelligence. The original sample, the natural environment disturbance sample, the adversarial disturbance sample and the joint disturbance sample of the two are jointly input into the neural network; a total loss function containing a main task loss, a statistical stability loss, a geometric stability loss and a discriminative consistency loss is calculated, and the model parameters are updated through back propagation. Among them, the statistical stability loss restricts the mean and covariance deviation of the feature distribution before and after disturbance, the geometric stability loss restricts the norm of the Jacobian matrix of the model at the disturbance input, and the discriminative consistency loss restricts the consistency of the prediction distribution before and after disturbance. The application simultaneously enhances the collaborative robustness of the model to natural disturbance and adversarial disturbance under a unified optimization framework, can significantly improve the classification accuracy under joint disturbance in image classification and text classification tasks, and is suitable for high-reliability scenes such as automatic driving and security monitoring.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a collaborative robust training method for neural networks based on joint perturbation optimization. Specifically, this application discloses a collaborative training method that enhances the stability of model feature representations through a joint perturbation optimization strategy under the combined effects of complex environmental perturbations and adversarial perturbations. This method is applicable to artificial intelligence applications requiring highly reliable perception and decision-making, such as computer vision and natural language processing. Background Technology

[0002] In recent years, deep learning models have made significant progress in tasks such as image recognition, object detection, and semantic understanding. However, when deployed in the real physical world, model performance is severely limited by two typical perturbations: first, input data distribution shifts caused by complex natural environmental factors such as low light, fog, and nighttime imaging, i.e., natural perturbations; and second, adversarial perturbations, which are artificially constructed by malicious attackers and add tiny but worst-case adversarial noise to the input. Under single or combined perturbations, models often exhibit problems such as disordered feature representations, blurred discrimination boundaries, and decreased prediction confidence, severely restricting the reliability of artificial intelligence systems in high-risk scenarios such as autonomous driving, security and control, and intelligent interaction.

[0003] To address natural perturbations, existing technologies primarily employ methods such as image enhancement, image restoration, domain adaptation, or end-to-end joint optimization. For example, histogram equalization or dehazing algorithms can improve input quality, or adversarial domain alignment techniques can reduce the distributional differences between clean and degraded data. These methods can recover input information and improve task accuracy to some extent, but their design goals are mostly focused on input-level recovery or performance optimization for specific types of degradation. They do not systematically analyze the fundamental impact of natural perturbations on the internal feature representation structure of the model, nor do they consider it in conjunction with the representation instability problem in adversarial perturbation scenarios.

[0004] To address adversarial perturbations, current mainstream techniques include adversarial attacks (such as the fast gradient sign method and projective gradient descent) and adversarial training. Adversarial training, by dynamically adding worst-case perturbations to the training data, forces the model to learn resilience and is currently one of the most effective defense methods. In addition, there are methods based on regularization constraints, robust optimization, and authentication defense. However, these methods primarily focus on improving the model's robustness to specific attacks, with insufficient research into the intrinsic mechanisms by which adversarial perturbations disrupt the local set continuity of features, cross-layer propagation stability, and classification margins. Furthermore, most of these methods are independent of natural robustness enhancement techniques.

[0005] In addition, some studies have attempted to improve robustness from the perspective of representation learning, such as through mutual information constraints, feature decorrelation, or robust prototype learning. However, existing technologies still have the following significant drawbacks: The divide-and-conquer approach leads to an imbalance in robustness. Current solutions typically treat robustness to natural perturbations and robustness to adversarial perturbations as two separate problems, lacking a unified technical framework capable of simultaneously modeling both types of perturbations. This makes it difficult for models to balance both types of robustness in real-world scenarios where environmental degradation and malicious attacks coexist, and may even result in one type of robustness being enhanced while the other is weakened.

[0006] There is a lack of a unified characterization of the degradation mechanism of feature representations. Whether it is the distribution shift caused by natural perturbations or the worst-case direction sensitive amplification caused by adversarial perturbations, the ultimate consequence is the statistical distribution distortion and geometric instability of the feature representations within the model. However, existing techniques have failed to establish a unified degradation metric and constraint mechanism from the two dimensions of statistical stability (mean and covariance shift) and geometric stability (local Jacobian variation and feature space continuity). As a result, robust enhancement measures lack theoretical guidance and rely heavily on empirical parameter tuning.

[0007] Training data and optimization objectives are isolated. Existing methods, during training, either use only clean data and augmented data, or only clean data and adversarial data, rarely incorporating clean samples, single-perturbation samples, and joint-perturbation samples into a unified training data stream and designing a co-optimization loss function. Therefore, the model struggles to learn the ability to maintain representation consistency under different perturbation sources.

[0008] Therefore, existing technologies have not yet provided a training method that can effectively cope with both complex natural environmental disturbances and adversarial disturbances, and can still maintain the stability of feature representations and the reliability of discrimination when the two types of disturbances act together. Summary of the Invention

[0009] The purpose of this invention is to provide a neural network cooperative robust training method based on joint perturbation optimization, which addresses the aforementioned problems.

[0010] The technical solution of the present invention is as follows: A collaborative robust training method for neural networks based on joint perturbation optimization includes the following steps: Obtain the original sample set ,in For the input sample, For the corresponding tags, This represents the total number of samples in the original sample set. For the original sample Applying natural environmental disturbances Generate natural environmental disturbance samples This is used to simulate input degradation under low-light, foggy, or nighttime imaging conditions. For environmental disturbance parameters, The transformation function is the natural environment disturbance. For the first The perturbed sample is obtained by perturbing the original sample with natural environmental disturbance. superscript The labeled samples are naturally perturbed samples; For the original sample Apply worst-case direction counter-perturbation Generate adversarial perturbation samples This is used to simulate the worst-case input under a malicious attack; among them, superscript The samples are labeled as adversarial perturbation samples; Samples of natural environmental disturbance Further superimposed worst-case direction counter-perturbation Generate joint perturbation samples This is used to simulate a complex risk scenario where the environment degrades and is then subjected to malicious attacks; among them, superscript This represents a sample under joint perturbation conditions; Original sample Natural environmental disturbance samples Adversarial perturbation samples and joint perturbation samples Common Input Neural Network Model ; Calculate total loss ,in The statistical stability loss is used to constrain the degree of shift in the feature distribution before and after the perturbation. Geometric stability loss is used to constrain the local smoothness of the model at perturbation inputs. To determine the consistency loss, it is used to constrain the stability of the predicted distribution before and after the perturbation. Preset weighting coefficients; Based on total loss The model parameters are updated through backpropagation so that the model maintains the statistical and geometric stability of the feature representation under joint perturbations.

[0011] By training with four types of samples, the model learns resistance to both natural and adversarial perturbations simultaneously during the same optimization process, overcoming the problem in existing technologies where both types of robustness cannot be simultaneously achieved. Through three stability constraints in the total loss function, direct control over the model's feature representation structure is realized, giving the training process a clear optimization direction and reducing reliance on empirical parameter tuning.

[0012] Furthermore, the geometric stability loss The calculation includes the following steps: According to the formula Calculation, where For the model in joint perturbation samples Jacobian matrix at the location, Frobenius norm; geometric stability loss This is used to constrain the magnitude of local geometric changes in the model at joint perturbation inputs, in order to suppress non-smooth shifts in feature representations caused by adversarial perturbations.

[0013] The above method directly constrains the magnitude of local gradient changes in the model at the joint perturbation input, suppressing the non-smooth shift in feature representation caused by adversarial perturbations. This loss calculation, based on the Jacobian matrix of the model's intermediate layers, can be backpropagated end-to-end, achieving explicit regularization of local geometry, which is superior to existing methods that rely solely on input gradient constraints.

[0014] Furthermore, the statistical stability loss The calculation includes the following steps: According to the formula Calculation, where These are the mean vector and covariance matrix of the original samples in the specified layer of the model, respectively. The mean vector and covariance matrix of the feature representations of the jointly perturbed samples at the same layer; statistical stability loss. The statistical offset used to constrain the characteristic distribution before and after the joint perturbation, in order to suppress the degradation of the characteristic distribution caused by natural perturbation.

[0015] By directly constraining the shifts in the first moment (mean) and second moment (covariance) of the feature representations before and after the joint perturbation, the statistical consistency of the feature distribution is maintained. Compared to methods that only constrain the mean shift, this loss also considers the correlation (covariance) between feature dimensions, thus more comprehensively preserving the stability of the feature structure.

[0016] Furthermore, the discriminative consistency loss The calculation includes the following steps: According to the formula Calculation, where This represents the model's predicted probability distribution for the original samples. This represents the predicted probability distribution of the model for the corresponding joint perturbation samples. Given the batch size; determine the consistency loss. This is used to constrain the model to maintain consistent discriminative output before and after perturbation, thereby enhancing the stability of the classification boundary.

[0017] By using the above method, the model is forced to output similar prediction distributions for different perturbation versions of the same input, thereby stabilizing the decision boundary.

[0018] Furthermore, the method for generating joint perturbation samples is as follows: First, the original sample Applying natural environmental disturbances Obtain intermediate samples Then, using this intermediate sample as a benchmark, the worst-case direction anti-perturbation solution is obtained. Then, the adversarial perturbation is superimposed on the intermediate sample to obtain a joint perturbation sample; wherein Indicates the true label, Represents the model mapping function. Represents the loss function. This indicates a disturbance strength constraint. For vectors of Norm.

[0019] Using the above method, the generated joint perturbation samples more closely resemble the combined risk of "environmental degradation followed by malicious attack" in real-world scenarios, improving the scene realism of the training data. This generation sequence utilizes the signal-to-noise ratio decrease caused by environmental perturbations, making subsequent adversarial perturbations more destructive in vulnerable areas, thereby forcing the model to learn more robust features.

[0020] Furthermore, the worst-case direction counteracts the disturbance. The single-step fast gradient sign method is used for approximation, and its approximate expression is: ,in For disturbance strength constraints, For symbolic functions, This is the gradient of the loss function with respect to the input samples.

[0021] The above method employs a single-step Fast Gradient Sign Method (FGSM) to approximate the worst-case direction perturbation. This approach is computationally efficient and suitable for real-time generation of adversarial examples during training without significantly increasing training time. Compared to more complex multi-step attacks (such as PGD), the perturbations generated by FGSM are easier for the model to learn, making it suitable for scenarios where training speed is critical.

[0022] Furthermore, the natural environment disturbance transformation It is at least one of low-light simulation transformation, haze simulation transformation, or nighttime imaging simulation transformation; environmental disturbance parameters These include brightness reduction factor, visibility distance, or signal-to-noise ratio, used to control the severity of environmental degradation.

[0023] By selecting common environmental degradation types such as low light, haze, and nighttime imaging, and quantifiable parameters such as brightness reduction coefficient, visibility distance, and signal-to-noise ratio, this method can cover a variety of real-world environmental scenarios and has good engineering adaptability. Parameterized control of environmental degradation intensity facilitates the use of multi-intensity perturbation enhancements during training, improving the model's generalization ability to different degrees of degradation.

[0024] Furthermore, the weighting coefficients satisfy This is used to balance the relative importance of the three constraints—statistical stability, geometric stability, and discriminant consistency—in the total loss.

[0025] By setting the sum of the weighting coefficients to 1, it is ensured that the contribution of each constraint in the total loss will not be overly dominant or ignored, making it easy to adjust quickly in different tasks.

[0026] Furthermore, the method can be applied to computer vision tasks or natural language understanding tasks; when applied to vision tasks, the input samples... For image data; when applied to language tasks, the input samples It is a text sequence data.

[0027] Furthermore, after the method is executed on visual and language tasks respectively, the cross-modal consistency index is calculated using the following formula: , in, This is a collaborative robustness evaluation value measured on a visual task. This is the collaborative robustness evaluation value measured on language tasks; this index is used to assess the consistency of the method's robustness performance across different modalities.

[0028] Compared with existing technologies, the advantages of this invention are: 1. Achieving synergistic improvement of natural robustness and adversarial robustness: This application incorporates natural environment perturbation samples, adversarial perturbation samples, and joint perturbation samples of the two into the same training data stream, and designs a joint loss function that integrates statistical stability, geometric stability, and discriminant consistency, which can simultaneously enhance the model's resistance to complex environmental degradation and malicious adversarial attacks under a unified training framework. 2. Establish a quantifiable feature representation stability constraint mechanism; addressing the problem that existing technologies cannot uniformly characterize the impact of natural and adversarial perturbations on the feature representation structure, this application proposes a statistical stability loss function. With geometric stability loss function .in, Directly constrain the mean and covariance shifts of clean samples and jointly perturbed samples in the feature space. Constrain the Jacobian matrix norm of the model at the joint perturbation input, thereby forcing the feature representation learned by the model to maintain the consistency of the statistical distribution and the smoothness of the local tangent space under different perturbation conditions; 3. To optimize the training data construction method and improve sample utilization efficiency, this application uses four types of data simultaneously for training: clean samples, naturally disturbed samples, adversarially disturbed samples, and jointly disturbed samples, and designs a discriminant consistency loss. This approach enables the model to output a consistent predicted distribution for different perturbation versions of the same original sample. Compared to existing techniques that use only a single type of perturbation sample or simply concatenate different perturbation datasets, the joint training strategy in this application allows the model to achieve better joint perturbation robustness in fewer training epochs and exhibits better generalization ability to changes in perturbation intensity. Furthermore, the joint perturbation sample generation method provided in this application, compared to directly applying adversarial perturbations to clean samples, better simulates the combined risk of "environmental degradation followed by malicious attack" in real-world scenarios, thus improving the scene realism of the training data.

[0029] 4. It possesses cross-task and cross-modal versatility. The training method of this application does not depend on a specific network structure or task type. Validation results on computer vision tasks (such as image classification and object detection) and natural language understanding tasks (such as sentiment classification and textual entailment) show that the method of this application achieves consistent collaborative robustness improvement in both modalities. This indicates that the joint perturbation optimization strategy proposed in this application has good transferability and can provide unified technical support for the deployment of different types of artificial intelligence systems in scenarios with high reliability requirements. Attached Figure Description

[0030] Figure 1 This is a flowchart of a neural network collaborative robust training method based on joint perturbation optimization. Detailed Implementation

[0031] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0032] The features and performance of the present invention will be further described in detail below with reference to embodiments.

[0033] Please see Figure 1In view of the problem that existing technologies handle complex natural environmental disturbances and adversarial disturbances separately, making it difficult for models to resist both types of disturbances at the same time, the purpose of this application is to provide a neural network collaborative robust training method based on joint perturbation optimization.

[0034] To achieve the above objectives, the present invention adopts the following technical approach in its specific implementation: Complex environmental disturbances are abstracted as structured distribution shifts in the input data, and adversarial disturbances are abstracted as worst-case disturbances applied along the sensitive directions of the model. Based on this, the impact of these two types of disturbances on the model's internal feature representations and discrimination boundaries is analyzed from two dimensions: the statistical distribution of feature representations and the local geometric structure.

[0035] A collaborative training framework that balances statistical and geometric stability is constructed. This framework incorporates the enhancement objectives of natural robustness and adversarial robustness into the same optimization system by designing statistical stability loss functions, geometric stability loss functions, and discriminant consistency loss functions. This allows the model to simultaneously acquire collaborative resistance to both types of perturbations during the same training process.

[0036] Through experimental verification on computer vision tasks (such as image classification) and natural language understanding tasks (such as sentiment classification), the effectiveness and cross-modal applicability of the training method proposed in this application are evaluated, thereby providing a reproducible technical solution for the reliable deployment of artificial intelligence systems in high-risk scenarios such as autonomous driving and security monitoring.

[0037] Specifically, the following steps are included: Natural environmental disturbance sample generation and feature representation stability index calculation: The purpose of this step is to simulate the input degradation process under complex environmental conditions such as low light, fog, and nighttime imaging, and to quantitatively calculate multiple indicators of input distribution before and after perturbation, internal model feature representation, and discriminative ability. These indicators can be used to monitor the degree of model representation degradation during subsequent training and can also serve as a quantitative basis for evaluating natural robustness. Specifically, it includes the following steps: S101: Sample generation and distribution offset calculation for natural environmental disturbances: A structured transformation is applied to the original input samples through environmental degradation mapping to generate samples with natural environmental disturbances. Specifically, the original input samples are... Mapping through environmental degradation Transformation to generate natural environmental disturbance samples: , in, Environmental disturbance parameters, including brightness reduction factor, visibility distance, or signal-to-noise ratio, are used to simulate the distribution shift of input data under low-light, foggy, or nighttime imaging conditions.

[0038] The input distribution after complex environmental disturbances can be represented as: , in, Represents the original input distribution. This represents the distribution mapping induced by environmental disturbances. This represents the input distribution under complex environmental conditions.

[0039] The input distribution shift intensity before and after the disturbance is calculated using the following formula to quantify the impact of environmental degradation on the statistical properties of the input data: , in, These are the mean vector and covariance matrix of the original input distribution, respectively. Let the mean vector and covariance matrix of the input distribution after perturbation be given. It is the Frobenius norm. The intensity of the structured distribution offset.

[0040] S102: Feature representation extraction and offset calculation: To examine the impact of perturbations on the model's internal feature representations, original samples and perturbed samples were input into the model separately, and feature representations of a specified layer were extracted. Specifically, the original samples... Samples with natural environmental disturbance Input the neural network model respectively Extract the first Layer feature representation: , in, For the model number Layer representation mapping function.

[0041] The feature representation offset caused by the perturbation is calculated using the following formula, which is used to measure the stability of the feature representation: , in, Indicates the original sample at the 1st Layer feature representation, This represents the offset caused by complex environmental disturbances.

[0042] Furthermore, the intra-class dispersion of samples under natural environmental disturbances is calculated using the following formula to assess the compactness of features within the class: , in, Indicates the first Feature representation of a perturbed sample For the first Feature centers of the category to which each sample belongs For the sample size, Indicates the degree of dispersion within a class.

[0043] S103: Calculation of inter-class separability, prediction entropy, and natural robustness evaluation values: To comprehensively evaluate the model's discriminative ability under natural environmental disturbances, we further calculate the inter-class separability, predicted information entropy, and natural robustness evaluation values. Specifically, the inter-class separability is calculated using the following formula: , in, For the number of categories, The first Class and First The feature center of a class under perturbation.

[0044] The predicted information entropy of the model under natural environmental disturbances is calculated using the following formula to quantify the uncertainty of the prediction: , in, For samples under complex environmental conditions, the prediction is the first The probability of a class This represents the prediction entropy of the model under complex environments.

[0045] Finally, the robustness evaluation value under natural disturbance conditions is calculated by weighting and combining the above three indicators using the following formula: , in, These are preset weighting coefficients. This evaluation value... It can be used directly for quantitative monitoring of the model's natural robustness during training, or as a reference indicator for model selection and hyperparameter tuning.

[0046] Generating samples against perturbations and calculating the feature instability index: The purpose of this step is to generate worst-case adversarial perturbation samples simulating malicious attacks, and to calculate quantitative indicators such as the sensitivity of the model's internal feature representations before and after the perturbation, local geometric changes, and cross-layer propagation instability. These indicators can be used to monitor the model's adversarial robustness during subsequent training and can also serve as a basis for evaluating adversarial defense capabilities. Specifically, it includes the following steps: S201: Generation of worst-case direction adversarial perturbation samples: Based on the sensitivity of the model output loss to the input samples, the worst-case directional perturbation that satisfies the perturbation norm constraint is solved. Specifically, the worst-case directional perturbation is solved using the following formula: , in, The original input sample, For real labels, For model mapping function, For loss function, For disturbance strength constraints, This represents the worst-case perturbation.

[0047] Then, the worst-case perturbation is superimposed onto the original input sample to generate adversarial examples: , in, This represents the input sample after adversarial perturbation.

[0048] As an approximate realization of this perturbation, it can be approximated using the single-step fast gradient sign method as follows: , in, The gradient of the loss function with respect to the input samples. It is a symbolic function.

[0049] S202: Sensitivity of Feature Representation and Calculation of Local Geometric Instability under Adversarial Perturbation: To examine the impact of adversarial perturbations on the model's internal feature representations, original samples and adversarial samples were input into the model separately, and feature representations of each layer were extracted and offsets were calculated. Specifically, the feature representations of the first layer were extracted... Layer adversarial feature representation: , in, The model number is represented by the first... Layer representation mapping function, Indicates adversarial examples, Indicates the first The adversarial characteristics of the layer are represented.

[0050] The feature representation offset caused by the anti-disturbance is calculated using the following formula: , in, For the original sample in the first Layer feature representation.

[0051] Further, calculate the following formula: Local geometric sensitivity of layers to perturbations: , This metric is used to quantify the extent to which a model amplifies resistance to perturbations within a local neighborhood.

[0052] S203: Calculation of cross-layer representation propagation instability and classification margin degradation: To analyze the propagation and cumulative effects of adversarial disturbances in network layers, the cross-layer propagation relationship is calculated using the following formula: , in, For the first The linear transformation matrix of the layer, The first Layer and first The layer represents the offset.

[0053] Calculate the first one according to the following formula The local Lipschitz constant of the layer representation mapping: , This constant is used to characterize the upper limit of how well the layer amplifies input perturbations.

[0054] Finally, the classification margin degradation caused by the adversarial perturbation is calculated using the following formula: , in, For the original sample with respect to the true category The classification interval, This represents the classification margin for adversarial examples. This degradation factor can be directly used to evaluate the boundary stability of the model under adversarial attacks.

[0055] Joint perturbation sample generation and unified representation degradation index calculation: The purpose of this step is to simulate a combined risk scenario of environmental degradation followed by malicious attacks, generate joint perturbation samples, and calculate a unified representation degradation index from three dimensions: statistical stability, geometric stability, and discriminant consistency. These indices can be used to monitor the representation stability of the model under multi-source perturbations during co-training. Specifically, the steps include: S301: Generation of joint perturbation input samples: A natural environment perturbation transformation is applied to the original input sample to obtain a structured degraded sample; a worst-case direction adversarial perturbation is then superimposed on this degraded sample to construct a joint perturbation input. Specifically, the joint perturbation input sample can be represented as: , in, For environmental disturbance changes, Counteracting disturbances in the worst-case scenario.

[0056] Extracting the joint disturbance input at the 1st Layer feature representation: , The characteristic representation offset caused by the joint disturbance is calculated using the following formula: .

[0057] S302: Calculation of statistical stability, geometric stability, and discriminant consistency degradation term: To unify the quantitative impact of joint perturbations on feature representation, degradation terms are calculated from three dimensions: statistical stability, geometric stability, and discriminant consistency.

[0058] Statistical stability degradation term: The mean and covariance shifts of the characteristic distribution before and after the joint perturbation are calculated using the following formula: , in, These are the mean and covariance matrices of the original sample feature distribution, respectively. Let be the mean and covariance matrix of the joint perturbation sample feature distribution.

[0059] Geometric stability degradation term: The Frobenius norm of the Jacobian matrix at the joint perturbation input of the model is calculated as follows: , in, For the model in The Jacobian matrix at that location.

[0060] Discriminant consistency degradation term: Calculate the mean square error between the predicted distributions of the original samples and the jointly perturbed samples using the following formula: , in, The model's predicted distribution of the original samples. This is the predicted distribution for the corresponding joint perturbation samples.

[0061] S303: Unified representation of degradation criteria and calculation of collaborative robustness evaluation value: By weighting and combining the above three degradation terms, we obtain the unified representation degradation criterion under joint perturbation conditions: , in, These are preset weighting coefficients.

[0062] The collaborative robustness characterization value is calculated using the following formula, which is used to comprehensively evaluate the robustness of the model under both types of perturbations: , in, The robustness evaluation value under natural disturbance conditions. To evaluate the robustness under perturbation conditions, For weight fusion.

[0063] Finally, the stability index is calculated using the following formula: , This indicator The larger the value, the more stable the feature representation structure of the model is under the combined effect of natural perturbation and adversarial perturbation, and it can be directly used for early stopping judgment or model selection during the model training process.

[0064] Construction of validation datasets and model evaluation for natural perturbations, adversarial perturbations, and joint perturbations: The purpose of this step is to construct multiple perturbation datasets for collaborative robustness verification and to uniformly evaluate the trained models on two different modalities, visual and language, to quantify the model's robustness under natural perturbation, adversarial perturbation, and a combination of both. Specifically, it includes the following steps: S401: Construction of validation datasets for natural perturbation, adversarial perturbation, and joint perturbation: Based on the original samples, natural environmental disturbance samples of different intensity levels are constructed to form a natural disturbance validation set. Specifically, for each original sample... Applying different disturbance intensity parameters Environmental degradation transformations generate a natural perturbation validation set: , in, Indicates the first One original sample, Indicates the corresponding label, This represents the transformation function for natural environmental disturbances. Indicates the first One disturbance intensity parameter, Indicates the number of samples. This indicates the disturbance intensity level.

[0065] Based on the original samples, worst-case orientation adversarial perturbation samples are constructed to form an adversarial perturbation validation set. Specifically, for each original sample... Solving for the worst-case direction perturbation Generate an adversarial perturbation validation set: , Finally, adversarial perturbations are superimposed on the natural environment perturbation samples to construct a joint perturbation validation set: ， The above three verification sets , , Together, they form a unified validation dataset system for subsequent model evaluation.

[0066] S402: Training and Validation of Collaborative Robust Representation Learning Models To train the model, the original training set is merged with the three types of perturbation sample sets mentioned above to form a joint training dataset: , in, This is the original training dataset.

[0067] The optimization objective for training the model parameters is as follows: This aims to minimize the total loss on the joint training set. , in, For model parameters, This is the total loss function.

[0068] After training, calculate the model's classification accuracy on any validation set using the following formula: , in, This represents the dataset to be evaluated. This indicates the number of samples in the dataset. For the model to sample The prediction results For indicator functions, The model represents the dataset The accuracy rate.

[0069] S403: Unified Assessment and Cross-Modal Analysis of Collaborative Robustness: The robustness evaluation values ​​of the model are calculated on the natural perturbation validation set, the adversarial perturbation validation set, and the joint perturbation validation set, respectively. Specifically: The robustness rating is calculated using the following formula: , The joint perturbation robustness evaluation value is calculated using the following formula: , Furthermore, to evaluate the consistency of this method across different modalities, collaborative robustness evaluation values ​​were calculated for visual and language tasks, respectively, and the cross-modal consistency index was calculated using the following formula: , in, For evaluating collaborative robustness on visual tasks (such as image classification), This is a collaborative robustness evaluation value on language tasks (such as text sentiment classification). The closer the value is to 1, the more consistent the robustness of the method is across different modal tasks, and it can be used to evaluate the cross-modal generality of the method.

[0070] The key point of this application is: (1) Incorporating the two types of perturbations into the same training framework. Instead of treating complex natural environment perturbations and adversarial perturbations separately, this application abstracts natural environment perturbations into structured distribution offsets and adversarial perturbations into worst-case perturbations applied along the sensitive direction of the model during the training process, thereby enhancing the model's resistance to the two types of perturbations under the same optimization system.

[0071] (2) A quantitative index for feature stability under natural environmental disturbances was designed. This application obtains the natural robustness evaluation value by calculating the input distribution shift, feature representation shift, intra-class dispersion, and inter-class separability caused by environmental disturbances such as low illumination, haze, and nighttime imaging. This provides a quantifiable reference for monitoring the natural robustness of the training process.

[0072] (3) A quantitative index for representation instability under adversarial perturbation was designed. This application models adversarial perturbation as worst-case directional perturbation and calculates feature representation offset, local geometric sensitivity, cross-layer propagation relationship, and classification margin degradation to obtain an adversarial robustness evaluation value. This is used to evaluate the model's resistance to malicious attacks.

[0073] (4) A unified representation degradation criterion under joint perturbation is constructed. This application constructs a unified representation degradation criterion from three dimensions: statistical stability (mean and covariance shift), geometric stability (Jacobi matrix norm), and discriminant consistency (difference in predicted distribution). and collaborative robustness characterization value It is used to comprehensively evaluate the representation stability of the model under multi-source perturbations.

[0074] (5) A collaborative training and unified verification mechanism was constructed. This application constructs a natural perturbation verification set. Counter-disturbance verification set and joint perturbation verification set The original samples and three types of perturbed samples are included in the training data stream, and the total loss function is used. To optimize the objectives, we aim to achieve collaborative training and unified validation of the model under various perturbation conditions.

[0075] (6) A cross-visual and language modal validation process has been established. This application not only performs the above training and validation steps on visual tasks (such as image classification), but also performs the same process on natural language understanding tasks (such as sentiment classification), and uses cross-modal consistency metrics. The consistency of the performance of this method across different modal tasks is quantified.

[0076] (7) Highlighting the technological innovation of the learning dimension. The innovation of this application is not simply to improve the empirical performance of a certain task, but to construct a computable stability constraint term from the two dimensions of statistical distribution and local geometric structure of feature representation, and integrate it into a unified training loss function, thereby improving the robustness of the model to natural perturbations and adversarial perturbations in the same optimization process.

[0077] Compared with existing methods that process natural disturbances and counter-disturbances separately, this application has the following verifiable technical effects: This application achieves a synergistic improvement in both natural and adversarial robustness. By incorporating naturally perturbed, adversarially perturbed, and jointly perturbed samples into the training process, and employing a total loss function that includes statistical, geometric, and discriminative constraints, the model simultaneously acquires resistance to both types of perturbations during the same training process. Experiments show that the model trained using this method achieves significantly higher classification accuracy on the jointly perturbed test set than methods that process them separately, and is not inferior to methods designed specifically for their respective scenarios on a single perturbed test set.

[0078] This application provides quantifiable feature representation stability constraints. It constrains the mean and covariance shifts of the feature distribution before and after perturbation using statistical stability loss, constrains the Jacobian matrix norm of the model at the perturbation input using geometric stability loss, and predicts the stability of the distribution using discriminant consistency loss. All three loss functions are computable engineering constraints, and their values ​​can be directly used to monitor changes in representation stability during training.

[0079] The construction method of training data has been optimized. The joint perturbation sample generation method adopted in this application (first applying natural perturbation, and then generating worst-case adversarial perturbation on the degraded sample) simulates the combined risk of "environment degradation followed by malicious attack" in real-world scenarios, thereby improving the scenario realism of the training data. At the same time, training with clean samples and three types of perturbation samples together improves sample utilization efficiency.

[0080] It possesses cross-task and cross-modal versatility. The training method in this application does not depend on a specific network structure or task type. Validation results on computer vision and natural language understanding tasks show that the proposed method achieves consistent collaborative robustness improvement in both modalities, indicating good transferability.

[0081] This application provides a reproducible technical solution for deploying artificial intelligence in high-risk scenarios. Through the specific training steps, loss function construction, and verification process described above, this application provides a quantifiable and reproducible model robustness enhancement scheme for artificial intelligence applications with high reliability requirements, such as autonomous driving and security monitoring.

[0082] The embodiments described above merely illustrate specific implementation methods of this application, and while the descriptions are detailed and specific, they should not be construed as limiting the scope of protection of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the technical solution of this application, and these modifications and improvements all fall within the scope of protection of this application.

Claims

1. A neural network cooperative robust training method based on joint perturbation optimization, characterized in that, Includes the following steps: Obtain the original sample set ,in For the input sample, For the corresponding tags, This represents the total number of samples in the original sample set. For the original sample Applying natural environmental disturbances Generate natural environment disturbance samples This is used to simulate input degradation under low-light, foggy, or nighttime imaging conditions. For environmental disturbance parameters, The transformation function is the natural environment disturbance. For the first The disturbed sample is obtained by perturbing the original sample with natural environmental disturbance; For the original sample Apply worst-case direction counter-perturbation Generate adversarial perturbation samples This is used to simulate the worst-case input under malicious attack. Samples of natural environmental disturbance Further superimposed worst-case direction counter-perturbation Generate joint perturbation samples This is used to simulate a complex risk scenario where the environment degrades and is then attacked maliciously. Original sample Natural environmental disturbance samples Adversarial perturbation samples and joint perturbation samples Common Input Neural Network Model ; Calculate total loss ,in The statistical stability loss is used to constrain the degree of shift in the feature distribution before and after the perturbation. Geometric stability loss is used to constrain the local smoothness of the model at perturbation inputs. To determine the consistency loss, it is used to constrain the stability of the predicted distribution before and after the perturbation. Preset weighting coefficients; Based on total loss The model parameters are updated through backpropagation so that the model maintains the statistical and geometric stability of the feature representation under joint perturbations.

2. The neural network cooperative robust training method based on joint perturbation optimization according to claim 1, characterized in that, The geometric stability loss The calculation includes the following steps: According to the formula Calculation, where For the model in joint perturbation samples Jacobian matrix at the location, Frobenius norm; geometric stability loss This is used to constrain the magnitude of local geometric changes in the model at joint perturbation inputs, in order to suppress non-smooth shifts in feature representations caused by adversarial perturbations.

3. The neural network cooperative robust training method based on joint perturbation optimization according to claim 1, characterized in that, The statistical stability loss The calculation includes the following steps: According to the formula Calculation, where These are the mean vector and covariance matrix of the original samples in the specified layer of the model, respectively. The mean vector and covariance matrix of the feature representations of the jointly perturbed samples at the same layer; statistical stability loss. The statistical offset used to constrain the characteristic distribution before and after the joint perturbation, in order to suppress the degradation of the characteristic distribution caused by natural perturbation.

4. The neural network cooperative robust training method based on joint perturbation optimization according to claim 1, characterized in that, The discriminant consistency loss The calculation includes the following steps: According to the formula Calculation, where This represents the model's predicted probability distribution for the original samples. This represents the predicted probability distribution of the model for the corresponding joint perturbation samples. Given the batch size; determine the consistency loss. This is used to constrain the model to maintain consistent discriminative output before and after perturbation, thereby enhancing the stability of the classification boundary.

5. The neural network cooperative robust training method based on joint perturbation optimization according to claim 1, characterized in that, The method for generating joint perturbation samples is as follows: First, the original sample Applying natural environmental disturbances Obtain intermediate samples Then, using this intermediate sample as a benchmark, the worst-case direction anti-perturbation solution is obtained. Then, this adversarial perturbation is superimposed on the intermediate sample to obtain the joint perturbation sample; where Indicates the true label, Represents the model mapping function. Represents the loss function. This indicates a disturbance strength constraint. For vectors of Norm.

6. The neural network cooperative robust training method based on joint perturbation optimization according to claim 5, characterized in that, The worst-case direction counteracts the disturbance. The single-step fast gradient sign method is used for approximation, and its approximate expression is: ,in For disturbance strength constraints, For symbolic functions, This is the gradient of the loss function with respect to the input samples.

7. A neural network cooperative robust training method based on joint perturbation optimization according to claim 1 or 5, characterized in that, The natural environmental disturbance transformation It is at least one of low-light simulation transformation, haze simulation transformation, or nighttime imaging simulation transformation; environmental disturbance parameters These include brightness reduction factor, visibility distance, or signal-to-noise ratio, used to control the severity of environmental degradation.

8. The neural network cooperative robust training method based on joint perturbation optimization according to claim 1, characterized in that, The weighting coefficient satisfy This is used to balance the relative importance of the three constraints—statistical stability, geometric stability, and discriminant consistency—in the total loss.

9. The neural network cooperative robust training method based on joint perturbation optimization according to claim 1, characterized in that, The method can be applied to computer vision tasks or natural language understanding tasks; when applied to vision tasks, the input samples... For image data; when applied to language tasks, the input samples It is text sequence data.

10. The neural network cooperative robust training method based on joint perturbation optimization according to claim 1, characterized in that, After the method is executed on visual and language tasks respectively, the cross-modal consistency index is calculated using the following formula: ， in, This is a collaborative robustness evaluation value measured on a visual task. This is the collaborative robustness evaluation value measured on language tasks; this index is used to assess the consistency of the method's robustness performance across different modalities.

Images