A dynamic robust adaptation and collaborative defense method for continuous semi-supervised learning

Abstract

The application relates to the field of artificial intelligence and deep learning security technology, and particularly discloses a dynamic robust adaptation and cooperative defense method for continuous semi-supervised learning, which dynamically instantiates a task-specific lightweight expert module at the end of a pre-trained ViT main body, physically isolates new and old task gradients; based on a category Gaussian distribution model, combining classification confidence and self-adaptive annular Mahalanobis distance double filtering unsupervised samples; adopting a mixed norm strategy to generate main noise, guiding the generation of auxiliary noise by historical experts according to feature similarity and fusing; jointly training pure and adversarial samples, introducing a KL divergence knowledge distillation loss to maintain the stability of the historical decision boundary. The application realizes efficient utilization of semi-supervised data and reuse of historical robust knowledge, solves three core problems, significantly improves the model generalization and defense performance, and is suitable for resource-constrained industrial online learning systems.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence and deep learning security technology, specifically to a dynamic robust adaptation and collaborative defense method for continuous semi-supervised learning. Background Technology

[0002] In recent years, deep neural networks (DNNs) have made leaps and bounds in image recognition, natural language processing, and various automated decision-making systems, with their performance even surpassing that of human experts on certain closed-source datasets. However, as artificial intelligence systems move from the laboratory to the real physical world, their deployment environments become highly dynamic, unpredictable, and fraught with malicious adversarial threats. In open-world intelligent applications, artificial intelligence systems often face three intertwined and severe challenges.

[0003] The first challenge is catastrophic forgetting caused by the dynamic changes in data distribution. Real-world data is not static but emerges continuously over time as a task-incremental stream. Models must be able to learn continuously to adapt to new environments. However, within the standard gradient descent optimization framework, when the model updates its weights using the newly arriving data stream to fit a new task, the difference in data distribution between the old and new tasks can cause drastic parameter drift or even complete erasure of key weights responsible for remembering old tasks, leading to a sharp drop in the model's predictive performance on historical tasks. In continuous learning scenarios, data arrives in incremental batches, and once the current task's training is complete, it cannot remain in memory for future reuse. Traditional algorithms used to mitigate catastrophic forgetting, such as Elastic Weight Consolidation (EWC) based on parameter regularization or replay techniques based on massive historical data caching, have limited effectiveness in preventing forgetting under long-sequence incremental tasks and resource-constrained conditions, and incur enormous computational costs.

[0004] The second challenge is the extreme scarcity of labeled data in real-world environments. In most commercial and industrial AI deployments, acquiring data with expert-level clean labels requires extremely high human and time costs. Models can usually easily acquire massive amounts of unsupervised data, but the proportion of supervised data is extremely low. Semi-Supervised Learning (SSL) emerged in this context. Currently, mainstream semi-supervised learning methods mainly rely on pseudo-labels and consistency regularization techniques. For example, by randomly flipping the current batch of data and merging it with the original data, the network is required to make consistent predictions for the same input and output distribution with different angles or pixel perturbations. Some existing patented solutions filter or assist in learning unsupervised data by judging whether the image category is consistent with the image angle flip. However, in continuous learning streams, due to the constantly changing task boundaries, the early prediction boundaries of the model are often extremely immature. If a simple hard threshold is applied to assign pseudo-labels based solely on the classification softmax probability (i.e., classification confidence) of the model output, it is very easy to cause confirmation bias. Numerous noisy or out-of-distribution samples located near the classification boundary are assigned high-confidence false labels and absorbed into the training set. These low-quality or erroneous unsupervised samples, like a virus, continuously pollute the model's feature space during the continuous learning process, leading to a performance collapse. Furthermore, extremely simple samples too close to the cluster center, while possessing very high confidence, offer little information for reshaping the model's decision boundaries. Repeatedly learning from such samples not only fails to improve model performance but also wastes valuable computing power and memory bandwidth.

[0005] The third challenge is the dramatic increase in adversarial vulnerability of deep models under continuous learning settings. Deep models are highly susceptible to being deceived by subtle adversarial perturbations imperceptible to the human eye, leading to absurd classification results. On static closed-source datasets, robustness is typically improved through adversarial training, such as projective gradient descent. However, under continuous semi-supervised learning settings, adversarial defense faces unprecedented difficulties: on the one hand, when the model fits new data, not only does its clean classification accuracy on historical tasks become catastrophically forgotten, but the adversarial defense boundaries built with enormous computational effort for historical tasks also quickly collapse—a phenomenon known as robust catastrophic forgetting. On the other hand, if only a single standard attack algorithm is used to generate adversarial examples for training at each stage of continuous learning, the model is highly susceptible to severe gradient masking or overfitting to specific adversarial perturbation norms after drifting through multiple tasks. Once the attacker switches attack modes, the seemingly robust model defenses collapse instantly.

[0006] In summary, existing deep learning training architectures and defense methods are inadequate for handling complex hybrid scenarios characterized by scarce labels, continuous task increments, and adversarial attacks. Therefore, there is an urgent need to propose an integrated technical architecture and methodology that can prevent catastrophic forgetting at the physical architecture level, accurately mine pure unsupervised features at the mathematical level, and achieve multimodal dynamic collaboration in defense strategies, thereby achieving an optimal balance between plasticity (adapting to new tasks) and stability (robustly memorizing features from old tasks). Summary of the Invention

[0007] To address the problems existing in the prior art, this invention provides a dynamic robust adaptation and collaborative defense method for continuous semi-supervised learning. This invention is the first to deeply integrate the dynamic expansion of the physical architecture, statistical Mahalanobis distribution filtering of the high-dimensional feature space, hybrid norm adversarial perturbation, and knowledge distillation order preservation mechanism. It successfully breaks through the bottleneck of high-quality pseudo-label generation in continuous task flow environment and constructs a collaborative adversarial defense system that runs through the entire life cycle of the model, thus solving the problems mentioned in the background technology.

[0008] To achieve the above objectives, the present invention provides the following technical solution: a dynamic robust adaptation and cooperative defense method for continuous semi-supervised learning, comprising the following steps: S1. Initialize the continuous semi-supervised learning scenario and model architecture: Construct a task incremental flow environment, use a pre-trained Visual Transformer (ViT) as a shared basic backbone network, and when receiving new task data, dynamically instantiate a lightweight expert module at the end of the shared basic backbone network. S2. High-quality unsupervised sample mining based on adaptive circumferential Gaussian boundary: For a new task, the lightweight expert module is pre-trained using a small number of supervised samples from the current task to construct a corresponding Gaussian distribution model for different categories in the feature space; then, the classification confidence of the network output and the circumferential Mahalanobis distance of the sample in the feature space are combined to perform double filtering on the massive unsupervised samples, and samples that conform to a specific distribution interval are selected to construct a high-quality unsupervised training set. S3. Collaborative defense against adversarial noise based on multi-expert feature similarity: In the initial task stage, smooth main noise is generated to strengthen the model base; in subsequent incremental tasks, the main noise is generated alternately through a hybrid norm strategy, and the features of the current clean sample on the shared basic backbone network are extracted. The cosine similarity between the sample and the feature space of all historical expert modules is calculated. The cosine similarity is used as an adaptive weight to guide each historical expert module to generate auxiliary adversarial noise and perform weighted fusion to obtain the final collaborative adversarial perturbation. S4. Perform joint training and knowledge distillation order-preserving optimization: input adversarial samples with the above-mentioned cooperative adversarial perturbation and clean samples into the network for forward propagation. Calculate the difference in logical values ​​between the new network output and the old network output as the knowledge distillation loss. Combine this with classification cross-entropy loss for joint backpropagation optimization. During training, strictly freeze the parameters of all historical expert modules to maintain the topological stability of the historical decision boundary.

[0009] Preferably, in step S1, the initialization of the continuous semi-supervised learning scenario and model architecture specifically includes the following: constructing an online continuous adversarial defense (OCAD) evaluation environment, enabling the model to perform multiple rounds of iterative learning and forward and backward updates and optimizations for any newly arriving incremental data stream containing supervised and unsupervised data; acquiring a pre-trained visual Transformer network model, freezing the parameters of the shallow modules of the visual Transformer throughout the entire lifecycle of continuous semi-supervised learning to retain its global general feature extraction capability obtained in large-scale pre-training, and only opening the last few Transformer Blocks and the classification head as adjustable layers; abandoning the traditional paradigm of fine-tuning parameters for the entire network, whenever a new incremental task is detected, the system dynamically expands and instantiates a new lightweight expert module in the physical architecture, the lightweight expert module containing an independent FC feature dimensionality reduction layer and an independent task classifier. This isolates the parameter gradient interference between new and old tasks at the physical architecture level, fundamentally alleviating the catastrophic forgetting problem.

[0010] Preferably, in step S2, constructing corresponding Gaussian distribution models for different categories specifically includes: performing supervised pre-training on the currently dynamically instantiated lightweight expert module based on a small number of supervised samples from the current task; inputting the supervised samples into the FC feature dimensionality reduction layer of the current lightweight expert module to extract its feature representation vector in the low-dimensional feature space; for each specific category in the current task, calculating the mean vector and covariance matrix of all supervised samples under that category in each dimension of the low-dimensional feature space; and constructing an independent high-dimensional multivariate Gaussian distribution model for each category based on the mean vector and covariance matrix.

[0011] Preferably, in step S2, the process of performing dual filtering on massive unsupervised samples to construct a high-quality unsupervised training set specifically includes: Step S21, Adaptive Threshold Dynamic Calculation: For each specific category, calculate the Mahalanobis distance from all supervised samples in that category to the center of the Gaussian distribution of that category, and sort all Mahalanobis distance values ​​in ascending order, using a preset quantile. Extract the corresponding values ​​and set them as the adaptive distance threshold specific to this category. ; Step S22, Preliminary Filtering of Classification Confidence: Input massive amounts of unsupervised samples one by one into the network for forward computation, obtain the predicted probability distribution of the network output, extract the category corresponding to the maximum probability value as the pseudo-label y, and extract the maximum probability value as the classification confidence; remove samples with a classification confidence value lower than or equal to a preset probability threshold. Unsupervised samples with a probability greater than the preset threshold are retained. The samples will proceed to the next screening stage; Step S23, Ring Mahalanobis Distance Secondary Filtering: For unsupervised samples that have passed the initial confidence filtering, extract their feature vectors output by the FC layer of the lightweight expert module, and calculate the Mahalanobis distance from the feature vector to the center of the Gaussian distribution corresponding to its pseudo-label y. ; Step S24: Determine the Mahalanobis distance Does it strictly fall within a specific "circular region" range? Inside, among which and The preset proportional coefficient and satisfying ; like If the sample is deemed too simple and lacks feature boundary information, it will be removed. like If the sample is determined to be out of the distribution or likely to be misclassified, it will be removed. Satisfy the Mahalanobis distance Unsupervised samples are incorporated into a high-quality unsupervised training set, and an upper limit is set on the number of samples in the empirical memory pool, thereby limiting computational overhead while ensuring sample purity.

[0012] Preferably, in step S3, generating smooth master noise to strengthen the model base in the initial task stage specifically includes: in the first received task (Task 0) stage, since no historical expert module has been established in the system, the system adopts the SmoothGrad mechanism to strengthen the defense base, applying multiple random sampling Gaussian noise perturbations to the given clean input sample; performing forward calculation and backward gradient backpropagation for each noisy sample to extract the input layer gradient; averaging the gradient results obtained under multiple random perturbations to generate smooth master noise with stronger generalization performance, which serves as the initial robust anchor point of the defense system throughout the model's life cycle.

[0013] Preferably, in step S3, the alternating generation of master noise using a hybrid norm strategy specifically includes: in subsequent incremental task processing, to prevent the model from overfitting to specific adversarial attack methods and to avoid gradient masking, the lightweight expert module is no longer limited to using a single adversarial attack method to generate master noise; instead, it adopts a probabilistic routing selection mechanism, randomly and alternately selecting based on a preset probability distribution ratio within a training batch. Projective gradient descent algorithm based on norm (PGD) ,based on Projective gradient descent algorithm based on norm (PGD) And the Fast Gradient Sign Method (FGSM) generates corresponding adversarial main noise for the current clean sample. .

[0014] Preferably, in step S3, calculating the cosine similarity between the sample and the feature space of all historical expert modules, and using the cosine similarity as an adaptive weight to guide each historical expert module to generate auxiliary adversarial noise and perform weighted fusion, specifically includes: for the input clean sample, extracting its deep representation features output on the shared visual Transformer backbone network, and inputting them into the FC layer of the current lightweight expert module and the FC layers of all instantiated historical expert modules respectively; calculating the cosine similarity between the feature vector output by the FC layer of the current lightweight expert module and the feature vector output by the FC layers of each historical expert module; processing all cosine similarity values ​​using the Softmax activation function, normalizing them and converting them into adaptive attention weights, and assigning them to each historical expert module; and having the historical expert module in a frozen state calculate the dual-track targeted adversarial gradient with respect to the input sample using backpropagation within its managed historical decision boundary, which serves as the historical expert auxiliary noise. The adversarial master noise generated by the current lightweight expert module will be... Auxiliary noise generated by all history expert modules Linear weighted fusion is performed according to adaptive attention weights to generate multi-expert collaborative adversarial perturbations.

[0015] Preferably, in step S4, the step of calculating the difference in logistic values ​​between the new network output and the old network output as the knowledge distillation loss, combined with the classification cross-entropy loss for joint backpropagation optimization, specifically includes: constructing adversarial examples that incorporate the cooperative adversarial perturbation; jointly feeding the clean samples and adversarial samples into the model for training calculation; extracting the new logistic values ​​(Logits) output by the network currently containing the newly instantiated lightweight expert modules when processing clean samples, and simultaneously extracting the old logistic values ​​(Logits) output by the old network containing only all frozen historical expert modules when processing the same clean samples; calculating the KL divergence between the new logistic values ​​and the old logistic values, and using it as the knowledge distillation loss (KD). The loss function is calculated by weighting the cross-entropy loss between adversarial samples and clean samples and summing it with the knowledge distillation loss term. The total loss function is then optimized by performing backpropagation. During this process, the parameters of all historical expert modules are strictly kept from being updated. Only the parameters of the open deep modules of the shared backbone network and the current lightweight expert modules are updated using gradients to force the maintenance of the topological shape of the historical decision boundary and achieve a balance between model plasticity and stability.

[0016] Preferably, after training for each incremental task is completed, the clean samples processed by the above steps are immediately used, along with samples generated by various adversarial attack algorithms (including FGSM, PGD-). PGD- The model was subjected to security stress tests using adversarial examples with different perturbation norms generated by (such as CW, AutoAttack, etc.) to quantify and record the average incremental robust accuracy of the model when subjected to different attacks.

[0017] The beneficial effects of this invention are: 1) Achieving extremely high purity unsupervised data utilization under limited computing power: This invention abandons the traditional soft or hard confidence pseudo-label thresholding method and proposes an adaptive cyclic Gaussian boundary to perform topological constraints in deep feature space. This is achieved by forcibly truncating Mahalanobis distances less than... The inner loop of the network eliminates the waste of memory and computing power caused by repeatedly learning from minimal samples; by forcibly cutting off the Mahalanobis distance greater than... The outer ring, from a mathematical expectation perspective, greatly prevents erroneous pseudo-label noise and OOD samples from contaminating the model feature pool, enabling the efficiency and purity of semi-supervised learning to reach unprecedented levels.

[0018] 2) This invention severs the transmission path of catastrophic forgetting at the root of the physical topology: Traditional anti-forgetting algorithms (such as EWC and replay) share all network parameters, resulting in strong coupling between parameters, making them prone to failure under continuous and drastic gradient updates. This invention employs a hybrid architecture combining a frozen basic backbone with dynamically instantiated lightweight experts. Through the physical isolation of the feature reduction layer and the classifier, it ensures that the learning gradient of new tasks will never overwrite the weight matrix of historical experts, allowing the system to maintain its basic memory even after experiencing dozens of incremental tasks.

[0019] 3) A comprehensive, generalized, and gradient-mask-immune defense system was constructed: by introducing PGD- PGD- The hybrid norm alternation generation strategy with FGSM effectively prevents the model from generating rigid, specific defenses or gradient masks for a single perturbation. More importantly, it achieves multi-expert collaborative mutual information-guided defense fusion through cosine similarity calculation. This mechanism means that when defending against novel attacks in the current task, the model can accurately recall and utilize the best-fit defense experience established by historical models based on feature similarity, achieving true dynamic robust adaptation.

[0020] 4) Achieving the best balance between plasticity and stability in intense adversarial tension: Simple adversarial training often accelerates knowledge forgetting due to its high penalty intensity. This invention, through a seamlessly integrated KL divergence knowledge distillation order preservation mechanism, forcibly maintains the topological consistency of the logical output space of the old and new network models while introducing large-scale mixed adversarial gradients, providing a robust comprehensive defense for extremely limited industrial-grade online learning systems. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the steps of the dynamic robust adaptation and collaborative defense method for continuous semi-supervised learning in an embodiment of the present invention; Figure 2 This is a schematic diagram of the physical topology of the frozen visual Transformer base and the lightweight expert module dedicated to the dynamically extended task in an embodiment of the present invention. Figure 3 This is a data flow diagram of the multi-expert collaborative hybrid norm adversarial noise generation, cosine similarity weight allocation, and fusion mechanism in this embodiment of the invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] The experimental verification environment and hardware configuration for this embodiment are as follows: The system runs on a Linux operating system, relying on the Python-based deep learning framework PyTorch for low-level operator implementation and gradient scheduling. The runtime environment is configured with CUDA 12.2 and driver version 535.216.03. On the hardware level, the model training and inference system is equipped with four parallel NVIDIA GeForce RTX 3090 GPUs (each with 24GB VRAM) to provide massive concurrent computation acceleration support for model forward inference and backward adversarial gradient processing.

[0024] This invention provides a technical solution: a dynamic robust adaptation and cooperative defense method for continuous semi-supervised learning, such as... Figure 1 As shown, this illustrates the entire lifecycle closed loop from incremental task input, sample selection, collaborative defense to joint optimization, comprising four core execution stages: (1) Initialize the continuous semi-supervised learning scenario and model architecture like Figure 2 As shown, a pre-trained Visual Transformer (ViT) is used as the foundation for global feature extraction, and its shallow attention module is permanently frozen to maintain general feature representation capabilities. For the continuously arriving task increment stream, catastrophic overlay fine-tuning is abandoned, and a dynamic physical expansion technique is employed to instantiate a dedicated lightweight expert module upon the arrival of each new task. This module embeds an independent fully connected feature dimensionality reduction operator and a task classifier. This step details how the model constructs its initial architecture within a strictly constrained online learning stream.

[0025] Establish an Online Continuous Adversarial Defense (OCAD) evaluation environment: The tasks faced by the system are not statically and completely acquired, but rather acquired incrementally. The tasks are passed to the system in a serialized form. Each contains a very small number of supervised labeled datasets. and a massive amount of unsupervised, clean datasets When processing any new incremental task, this system allows the model to perform multiple rounds of learning and updating optimization on the training data within the current stage to fully fit the current feature distribution; however, once it enters the subsequent incremental task stage, the historical task data is cleared or overwritten, thus simulating the setting of continuous learning in real-world applications.

[0026] Foundation and Shared Feature Extractor: This invention uses a pre-trained Visual Transformer (ViT) model as the backbone network of the system. ViT, through its self-attention mechanism, can capture global relationships between widely spaced pixel blocks in an image, providing a smoother and richer feature manifold than traditional CNNs. During system initialization... At any given time, the system permanently freezes the shallow modules of the ViT model (e.g., for ViT-Base, freezing its Patch Embedding layer and all self-attention weight matrices and MLP weight matrices of the first 9 Transformer Blocks). This freezing operation preserves the "general visual prior knowledge" acquired during model pre-training, significantly reducing the number of samples required for online learning, while preventing shallow features from being biased by new local tasks. The system only opens the last three Transformer Blocks of ViT and the global feature aggregation classification head as a shared bridge for cross-task fine-tuning.

[0027] Lightweight expert instantiation for dynamic tasks: Traditional networks, when encountering new tasks, typically fine-tune the shared classifier fully connected layer to simultaneously output the probability distributions of both the old and new tasks. This coupling, leading to rewriting of weight parameters, is the physical root cause of catastrophic forgetting. In this embodiment, whenever a new task switch is detected in the task increment stream (entering...),... The system's architecture control program performs physical-level architecture topology expansion. Specifically, in the physical logic architecture of the code, a network class structure `vitbackbone` inherited from the basic neural network module is defined. This class contains built-in function interface methods for dynamically constructing experts. Whenever the system detects a new task arriving and needs to allocate the corresponding number of classification categories, this interface is called to dynamically attach a new lightweight expert module to the open-tuning ViT end. This expert module contains independent feature reduction layers and classifiers. When a new expert is instantiated, the system console outputs a build log for process monitoring. This is relevant for the current new task. During joint training, to prevent historical parameters from being overwritten or contaminated by gradients at the code level, the system iterates through the list object `expertlist` containing all historical expert modules using program logic. Then, it uses nested loops to extract the entire weight parameter matrix of each historical expert `old_expert` in the current list element, forcibly setting these extracted historical parameters to a non-updatable state. This hard isolation mechanism based on object traversal completely prevents gradient backpropagation from disrupting the feature boundaries of the old task, both physically and logically.

[0028] (2) High-quality unsupervised sample mining stage based on adaptive cyclic Gaussian boundary To address the pain point of noise-induced confirmation bias in massive unlabeled data, this paper innovatively proposes multivariate Gaussian modeling in a low-dimensional feature space. The Mahalanobis distance distribution of the feature dimensions is calculated based on a small number of clean, supervised samples, and quantiles are used to dynamically define the specific thresholds for each category. Building upon this, a dual filtering strategy based on "circular Mahalanobis distance" is introduced. This strategy not only requires that the classification confidence score meets the standard, but also strictly limits the Mahalanobis distance of the unsupervised samples to fall within a certain range. Within the ring-shaped physical interval (where It is a proportionality coefficient and This allows for the simultaneous filtering out of simple samples lacking information and high-risk out-of-distribution anomalies.

[0029] How to manage the massive amounts of unsupervised data that come with new tasks Accurately extracting samples beneficial to the model is the core technical challenge that this step aims to address. This invention abandons the simplistic approach of relying on a single-dimensional scalar (Softmax confidence) for pseudo-label judgment, and instead delves into the multi-dimensional feature space to measure and constrain geometric distribution.

[0030] Step S21, Expert warm-up and Gaussian distribution modeling in high-dimensional feature space: Before processing unsupervised data, the system first utilizes a very small number of supervised samples from the current task. For the newly generated current lightweight expert module Perform a small number of iterations of pre-training (Pre-training Expert). Then, use this batch of supervised samples again. Through the frozen backbone network and fed into the preheated The FC layer obtains each sample in the reduced-dimensional space (e.g., Compact eigenvector representation in ) For each category c to be classified in the current task, the system calculates the multivariate statistical attributes of all feature vectors under that category, that is, it calculates the mean vector of each category in each dimension. and Covariance matrix of size A simplified Gaussian distribution model is then constructed for each category. .

[0031] Step S22, calculate the adaptive feature distance threshold. In multidimensional space, the scale differences of feature dimensions are extremely large and exhibit significant anisotropy. Using conventional Euclidean distance for filtering results in severe scale bias. This invention employs Mahalanobis distance. The distance is measured by the inverse of the covariance matrix. This effectively eliminates the interference of the correlation between units and dimensions. For a given class c, all its supervised feature vectors are calculated to the mean center of that class. Mahalanobis distance: ; Sort all Mahalanobis distances of the same category c in ascending order and calculate the preset statistical quantiles. (This embodiment is a preferred option and can be adopted.) (i.e., the 85th percentile), which is set as the adaptive distance threshold specific to this category. The threshold is determined through dynamic calculation of quantiles. It can adapt to the inherent looseness of feature manifolds of each category, avoiding the inflexibility caused by hard-coded scalar thresholds.

[0032] Step S23, Ring Mahalanobis Distance Secondary Filtering Strategy (Detailed Explanation of Core Mechanism): Facing Unsupervised Stream Data The system constructs and executes a rigorous dual-filtering strategy: First, it calculates the predicted classification logic of the network's forward propagation output and performs Softmax normalization, obtaining the highest predicted probability as the classification confidence of the sample, and labels its corresponding category as a candidate pseudo-label y. If the confidence is lower than a preset probability threshold... (This embodiment is a preferred option, and may be set as follows) This sample is discarded directly. This step is similar to traditional filtering methods, acting as an initial screening. The more core filtering strategy is annular Mahalanobis distance filtering: for samples that pass the initial screening, their FC layer feature vectors are extracted. Rigorously calculate the Mahalanobis distance from the feature to the center of the Gaussian distribution to which the candidate pseudo-label y belongs. .

[0033] Step S24, determine the Mahalanobis distance. Does it strictly fall within a specific "circular region" range? The system performs spatial truncation based on the following decision logic (setting the inner loop coefficient). With outer ring coefficient And satisfy To facilitate system construction, the preferred value in this embodiment can be set as follows: and ): Rejection Condition 1 (Inner Ring Excluded, Too Close): If The samples within this interval lie at the absolute core of the Gaussian distribution. The model already possesses extremely high determinism and proficiency with such typical features. From an information theory perspective, its information entropy is extremely low. Repeatedly using computational units for forward and backward computation of such samples cannot push any classification boundary outward, only wasting computational power and experience replay pool capacity. The system rejects its inclusion.

[0034] Rejection Condition 2 (Outer Ring Interception, Too Far): If This interval lies in the sparse region at the edge of the feature distribution. In semi-supervised learning without real label correction, these samples are highly likely to be noise with severely distorted features, out-of-distribution anomalies, or incorrect samples forced by the classifier to have high confidence (a phenomenon known as overconfidence misclassification in neural networks). Allowing such samples with incorrect pseudo-labels to participate in gradient updates will severely distort the existing classification surface. Therefore, the system rejects their inclusion.

[0035] Finally, the necessary and sufficient conditions for filtering are mathematically expressed as follows: ; The retained unsupervised samples are considered high-quality samples and included in a high-quality unsupervised training set, possessing the safe attribute of large information increment and no out-of-bounds behavior. They are placed into an experience base with strict memory capacity constraints for use in the next step of adversarial collaborative optimization. This module greatly improves the purity and efficiency of semi-supervised learning under limited computing power.

[0036] (3) Hybrid adversarial noise collaborative defense stage based on multi-expert feature similarity To address overfitting caused by a single attack method, a hybrid norm-alternating attack strategy is introduced to generate master noise during the evolution of continuous tasks. More importantly, a mutual information-guided defense fusion strategy is proposed, such as... Figure 3 As shown: By calculating the cosine similarity between the output representations of the current expert module and all historical expert modules under the current sample input, and transforming it into attention weight allocation using Softmax activation. This weight will "awaken" the corresponding frozen historical experts, collaboratively generating a dual-track targeted adversarial gradient, and finally dynamically weighting and fusing it with the main noise, enabling the model to adaptively reuse historical robust knowledge when facing new threats.

[0037] This step focuses on describing how to construct a long-term, dynamic, collaborative defense system against multimodal adversarial attacks, based on model architecture decoupling and obtaining clean training data. The core idea of ​​adversarial defense is to use adversarial example algorithms to generate perturbation noise targeting the current model. It is then overlaid on the original image to participate in network training in order to prepare for defense and resistance in advance.

[0038] Initial Defense Foundation Strengthening (SmoothGrad Anchor Construction): When processing the first task, Task 0, the system has not yet established any historical experience base. For newly generated clean samples, the network does not simply rely on conventional projected gradients to find the worst extrema. Instead, it introduces a SmoothGrad mechanism to enhance gradient smoothness: the system adds Gaussian random noise with a certain variance to a single input image multiple times, forming K (e.g., K=5) slightly different copies of the input model. The input layer gradients relative to the class are calculated for these K copies and averaged. The adversarial noise generated using this averaged smoothed gradient is called smoothed master noise. This operation provides the foundation network, which is in its early lifecycle, with an extremely robust initial defense anchor that is not prone to overfitting to local extrema.

[0039] Hybrid Norm Defense Strategy for Incremental Tasks: In the subsequent task increment phase (Task t>0), this embodiment of the invention strongly corrects the drawback of relying on a single adversarial generation algorithm in traditional adversarial training. If the model always only uses PGD- that limits the maximum absolute change of each pixel... During training, network weights are specialized and generate gradient masks, which can lead to vulnerabilities if a hacker uses PGD based on total energy variation constraints. Alternatively, a fast gradient sign method based on single large-span linear extrapolation would immediately compromise the model. In this embodiment, for the current clean input sample, the system's built-in collaborative defense engine randomly selects an attack operator according to a probabilistic routing strategy, generating the anti-disturbance master noise for the currently instantiated lightweight expert module. This training method, which alternately exposes the user to the constraints of different norm spaces, makes the current task boundary robust and provides excellent generalization defense capabilities in all weather conditions.

[0040] Mutual information-guided multi-expert similarity-weighted collaborative defense: This is the key innovation addressing the core challenge of "robust degradation" in continuous learning. When learning the current task, models easily forget how to defend against old features. The system implements a similarity-based wake-up mechanism guided by mutual information: after the input sample passes through the shared frozen ViT base, its general feature representation is obtained. The system simultaneously sends this general representation to the currently active new expert module and... The fully connected (FC) layers of each dormant (parameters frozen) historical expert module extract and obtain their respective feature representation vectors. as well as Calculate the cosine similarity between the current expert output features and the features of each historical expert in multidimensional space. : ; The system further utilizes the Softmax function, which incorporates a temperature coefficient T, to measure the scalar similarity. Transform it into an attention weight allocation probability matrix that sums to 1. Weight The magnitude of this weight directly reflects the degree of similarity or overlap between the features of the current input sample and the knowledge space of a specific historical task. Based on this weight, the system instructs the corresponding frozen history expert module to perform inverse gradient calculation, generating targeted auxiliary noise for its own jurisdictional category. Finally, the system intelligently and dynamically fuses the current primary noise with all historical auxiliary noise and applies it to the original input: ; Through the aforementioned collaborative weighted calculations, the previously fragmented old and new defense strategies are now accurately activated and reused at the sample level based on a mutual information mechanism, greatly improving the system-level robustness of the network against unknown complex disturbances.

[0041] (4) Execute joint training and anti-disturbance knowledge distillation and order preservation phase When using the aforementioned fusion-based adversarial noise to penalize the model during training, in addition to the classification cross-entropy, the output Logits divergence (KLDivergence) of the new network structure and the frozen old network structure under the same input is forcibly calculated. This distillation-preserving operation acts as a topological anchor point in the intense spatial tension of adversarial optimization, ensuring that the structural form of the historical task classification boundary is not destroyed.

[0042] To prevent the generated joint adversarial perturbations from violating the logical boundaries of the old model during backpropagation in the network's joint optimization, this system embeds a logistic value tracking mechanism during the forward computation process. The system not only calculates the classification cross-entropy loss between the sample and its true or high-quality pseudo-labels, but also... It also forces the recording of the predicted probability output manifold of the old model framework (excluding the new expert module, with all parameters strictly frozen) for the same undisturbed, pure sample. Simultaneously, it records the predicted probability manifold of the output of the new network architecture (including the latest added and active lightweight expert modules) currently undergoing parameter updates. The Kullback-Leibler divergence (KL Divergence) between the two is calculated as the knowledge distillation penalty loss. : ; The adversarial cross-entropy, ordinary cross-entropy, and KL divergence terms are assembled into a total loss function according to hyperparameter ratios, and then used for unified backpropagation and weight updates. Because the weights of the historical expert are physically locked at the hardware level, the shared trainable layer parameters of the new model are affected during updates. This constraint ensures that, while learning and adapting to new paradigms to resist interference, the topology of its output layer does not deviate significantly from historical experience.

[0043] This invention enables efficient utilization of semi-supervised data and reuse of robust historical knowledge, solving three core problems and significantly improving model generalization and defense performance. It is suitable for resource-constrained industrial online learning systems.

[0044] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0045] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0046] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0047] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0048] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A dynamic robust adaptation and cooperative defense method for continuous semi-supervised learning, characterized in that, Includes the following steps: S1. Initialize the continuous semi-supervised learning scenario and model architecture: Construct a task incremental flow environment, use a pre-trained visual Transformer as a shared basic backbone network, and when new task data is received, dynamically instantiate a lightweight expert module at the end of the shared basic backbone network. S2. High-quality unsupervised sample mining based on adaptive circumferential Gaussian boundary: For a new task, the lightweight expert module is pre-trained using a small number of supervised samples from the current task to construct a corresponding Gaussian distribution model for different categories in the feature space; then, the classification confidence of the network output and the circumferential Mahalanobis distance of the sample in the feature space are combined to perform double filtering on the massive unsupervised samples, and samples that conform to a specific distribution interval are selected to construct a high-quality unsupervised training set. S3. Collaborative defense against adversarial noise based on multi-expert feature similarity: In the initial task stage, smooth main noise is generated to strengthen the model base; in subsequent incremental tasks, the main noise is generated alternately through a hybrid norm strategy, and the features of the current clean sample on the shared basic backbone network are extracted. The cosine similarity between the sample and the feature space of all historical expert modules is calculated. The cosine similarity is used as an adaptive weight to guide each historical expert module to generate auxiliary adversarial noise and perform weighted fusion to obtain the final collaborative adversarial perturbation. S4. Perform joint training and knowledge distillation order-preserving optimization: input adversarial samples with the above-mentioned cooperative adversarial perturbation and clean samples into the network for forward propagation. Calculate the difference in logical values ​​between the new network output and the old network output as the knowledge distillation loss. Combine this with classification cross-entropy loss for joint backpropagation optimization. During training, strictly freeze the parameters of all historical expert modules to maintain the topological stability of the historical decision boundary.

2. The dynamic robust adaptation and cooperative defense method for continuous semi-supervised learning according to claim 1, characterized in that: In step S1, the initialization of the continuous semi-supervised learning scenario and model architecture specifically includes the following: constructing an online continuous adversarial defense (OCAD) evaluation environment, enabling the model to perform multiple rounds of iterative learning and forward and backward updates and optimizations for any new incremental data stream containing supervised and unsupervised data; obtaining a pre-trained visual Transformer network model, freezing the parameters of the shallow modules of the visual Transformer throughout the entire lifecycle of continuous semi-supervised learning to retain its global general feature extraction capability obtained in large-scale pre-training, and only opening the last few TransformerBlocks and the classification head as adjustable layers; abandoning the traditional paradigm of fine-tuning parameters for the entire network, whenever a new incremental task is detected, the system dynamically expands and instantiates a new lightweight expert module in the physical architecture, the lightweight expert module containing an independent FC feature dimensionality reduction layer and an independent task classifier.

3. The dynamic robust adaptation and cooperative defense method for continuous semi-supervised learning according to claim 1, characterized in that: In step S2, constructing corresponding Gaussian distribution models for different categories specifically includes: performing supervised pre-training on the currently dynamically instantiated lightweight expert module based on a small number of supervised samples from the current task; inputting the supervised samples into the FC feature dimensionality reduction layer of the current lightweight expert module to extract its feature representation vector in the low-dimensional feature space; for each specific category in the current task, calculating the mean vector and covariance matrix of all supervised samples under that category in each dimension of the low-dimensional feature space; and constructing an independent high-dimensional multivariate Gaussian distribution model for each category based on the mean vector and covariance matrix.

4. The dynamic robust adaptation and cooperative defense method for continuous semi-supervised learning according to claim 1, characterized in that: In step S2, the process of performing dual filtering on massive unsupervised samples to construct a high-quality unsupervised training set specifically includes: Step S21, Adaptive Threshold Dynamic Calculation: For each specific category, calculate the Mahalanobis distance from all supervised samples in that category to the center of the Gaussian distribution of that category, and sort all Mahalanobis distance values ​​in ascending order, using a preset quantile. Extract the corresponding values ​​and set them as the adaptive distance threshold specific to this category. ; Step S22, Preliminary Filtering of Classification Confidence: Input massive amounts of unsupervised samples one by one into the network for forward computation, obtain the predicted probability distribution of the network output, extract the category corresponding to the maximum probability value as the pseudo-label y, and extract the maximum probability value as the classification confidence; remove samples with a classification confidence value lower than or equal to a preset probability threshold. Unsupervised samples with a probability greater than the preset threshold are retained. The samples will proceed to the next screening stage; Step S23, Ring Mahalanobis Distance Secondary Filtering: For unsupervised samples that have passed the initial confidence filtering, extract their feature vectors output by the FC layer of the lightweight expert module, and calculate the Mahalanobis distance from the feature vector to the center of the Gaussian distribution corresponding to its pseudo-label y. ; Step S24: Determine the Mahalanobis distance Does it strictly fall within a specific "circular region" range? Inside, among which and The preset proportional coefficient and satisfying ; like If the sample is deemed too simple and lacks feature boundary information, it will be removed. like If the sample is determined to be out of the distribution or likely to be misclassified, it will be removed. Satisfy the Mahalanobis distance Unsupervised samples are incorporated into a high-quality unsupervised training set.

5. The dynamic robust adaptation and cooperative defense method for continuous semi-supervised learning according to claim 1, characterized in that: In step S3, generating smooth master noise to strengthen the model base in the initial task stage specifically includes: in the first task stage after receiving the data, the system uses the SmoothGrad mechanism to strengthen the defense base by applying multiple random sampling Gaussian noise perturbations to the given clean input sample; performing forward calculation and backward gradient backpropagation for each noisy sample to extract the input layer gradient; averaging the gradient results obtained under multiple random perturbations to generate smooth master noise with stronger generalization performance, which serves as the initial robust anchor point of the defense system throughout the model's lifecycle.

6. The dynamic robust adaptation and cooperative defense method for continuous semi-supervised learning according to claim 1, characterized in that: In step S3, the alternating generation of main noise using a hybrid norm strategy specifically includes: in subsequent incremental task processing, the lightweight expert module is no longer limited to generating main noise using a single adversarial attack method; instead, it adopts a probabilistic routing selection mechanism to randomly and alternately select based on a preset probability distribution ratio within a training batch. Projective gradient descent algorithm based on norm (PGD) ,based on Projective gradient descent algorithm based on norm (PGD) And the Fast Gradient Signed Method (FGSM) generates corresponding adversarial main noise for the current clean sample. .

7. The dynamic robust adaptation and cooperative defense method for continuous semi-supervised learning according to claim 1, characterized in that: In step S3, calculating the cosine similarity between the sample and the feature space of all historical expert modules, and using the cosine similarity as an adaptive weight to guide each historical expert module to generate auxiliary adversarial noise and perform weighted fusion, specifically includes: for the input clean sample, extracting its deep representation features output on the shared visual Transformer backbone network, and inputting them into the FC layer of the current lightweight expert module and the FC layers of all instantiated historical expert modules respectively; calculating the cosine similarity between the feature vector output by the FC layer of the current lightweight expert module and the feature vector output by the FC layers of each historical expert module; processing all cosine similarity values ​​using the Softmax activation function, normalizing them and converting them into adaptive attention weights, and assigning them to each historical expert module; and having the historical expert module in a frozen state calculate the dual-track targeted adversarial gradient with respect to the input sample using backpropagation within its managed historical decision boundary, which serves as the historical expert auxiliary noise. The adversarial master noise generated by the current lightweight expert module will be... Auxiliary noise generated by all history expert modules Linear weighted fusion is performed according to adaptive attention weights to generate multi-expert collaborative adversarial perturbations.

8. The dynamic robust adaptation and cooperative defense method for continuous semi-supervised learning according to claim 1, characterized in that: In step S4, the step of calculating the difference in logistic values ​​between the new network output and the old network output as the knowledge distillation loss, and performing joint backpropagation optimization in conjunction with the classification cross-entropy loss, specifically includes: constructing adversarial examples that incorporate the cooperative adversarial perturbation; jointly feeding the clean samples and adversarial samples into the model for training calculation; extracting the new logistic values ​​Logits output by the network currently containing the newly instantiated lightweight expert modules when processing clean samples, and simultaneously extracting the old logistic values ​​Logits output by the old network containing only all frozen historical expert modules when processing the same clean samples; calculating the KL divergence between the new logistic values ​​and the old logistic values, and using it as the knowledge distillation loss term; calculating the classification cross-entropy loss of the network on adversarial samples and clean samples, and weighting and summing it with the knowledge distillation loss term to obtain the final total loss function; and executing the backpropagation algorithm to optimize the total loss function. During this process, the parameters of all historical expert modules are strictly kept unupdable, and only the parameters of the open deep modules of the shared backbone network and the current lightweight expert modules are updated using gradients to forcibly maintain the topological shape of the historical decision boundary.

Images