Deep learning model-oriented back door behavior dynamic detection and positioning method

Abstract

The invention discloses a back door behavior dynamic detection and positioning method for a deep learning model, and relates to the field of artificial intelligence security, and the method comprises the steps: S1, monitoring input data and an output prediction result of the deep learning model in a reasoning process in real time; s2, performing statistical feature analysis on the output prediction result based on a preset spatiotemporal behavior pattern library, and identifying abnormal deviation exceeding a normal behavior threshold; s3, in response to the abnormal judgment in the S2, positioning an input data area for triggering a backdoor behavior through reverse gradient tracking and attention weight structure analysis; and S4, deconstructing the deep learning model into an independent analysis unit according to a functional layer, and realizing hierarchical positioning of the backdoor behavior through layer-by-layer activation value comparison. The method has the advantages that real-time dynamic detection is achieved, the rear door triggering area is accurately positioned, the universality of a cross-model architecture is supported, and reasoning delay is reduced.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence security, specifically to a method for dynamic detection and localization of backdoor behavior for deep learning models. Background Technology

[0002] With the widespread application of deep learning technology in fields such as computer vision and natural language processing, model security issues have become increasingly prominent. Among them, backdoor attacks, by implanting specific triggering mechanisms into the model, enable the model to perform well under normal input, but generate malicious output when encountering certain triggering patterns. This type of attack poses a serious threat to the security of intelligent systems.

[0003] Current techniques for backdoor detection in deep learning models suffer from the following limitations: First, existing static detection methods rely on offline analysis of training data or model parameters, making them ill-equipped to handle dynamic adaptive backdoor attacks. These attacks adjust their triggering patterns according to changes in input data, rendering traditional static detection methods ineffective. Second, most detection tools only provide binary results, failing to accurately pinpoint the trigger's shape, location, and propagation path within the model. This significantly limits the effectiveness of defensive measures. Furthermore, existing online monitoring schemes often require calculating the full gradient, leading to a significant increase in inference latency and making it difficult to meet the real-time requirements of practical applications.

[0004] More critically, existing technologies perform poorly when dealing with heterogeneous model architectures. Different types of deep learning models (such as CNN, Transformer, GNN, etc.) have significantly different internal structures and computational characteristics, while current backdoor detection methods are often designed for specific architectures and lack universal solutions across models. Furthermore, as attackers continuously develop new backdoor patterns, traditional fixed-rule detection methods struggle to cope with the challenges of zero-day attacks.

[0005] To address the aforementioned issues, existing technologies urgently need improvement. Summary of the Invention

[0006] In view of this, the present invention provides a method for dynamic detection and localization of backdoor behavior for deep learning models, which has the advantages of real-time dynamic detection, accurate localization of backdoor triggering areas, support for cross-model architecture universality, and reduced inference latency.

[0007] This invention provides a method for dynamic detection and localization of backdoor behavior for deep learning models, comprising: S1. Real-time monitoring of the input data and output prediction results of the deep learning model during the inference process; S2. Based on a preset spatiotemporal behavior pattern library, perform statistical feature analysis on the output prediction results to identify abnormal deviations that exceed the normal behavior threshold. S3. In response to the anomaly determination in S2, the input data region that triggers the backdoor behavior is located through reverse gradient tracing and attention weight structure analysis. S4. Deconstruct the deep learning model into independent analysis units according to functional layers, and realize the hierarchical localization of backdoor behavior by comparing activation values ​​layer by layer.

[0008] In one optional implementation, S1 includes: A lightweight streaming sampling technique is used to extract a subset of the input data as monitoring samples. The time-series volatility index of the output confidence level is calculated in real time. When the volatility index exceeds the dynamic threshold, it is marked as a sudden change point in the predicted distribution.

[0009] In one alternative implementation, S2 includes: Construct a baseline feature set for normal behavior; the baseline feature set includes prediction entropy, class jump frequency, and confidence variance; An adaptive threshold algorithm is used to dynamically update the anomaly detection boundary. The adaptive threshold algorithm adjusts the boundary value in real time according to the behavior drift of the deep learning model.

[0010] In an optional implementation, S2 further includes: When statistical anomalies are detected, controllable micro-perturbations are injected into the input data; If the degree of abrupt change in the output prediction is in a non-linear proportional relationship with the intensity of the disturbance, it is determined to be backdoor behavior.

[0011] In one optional implementation, S3 includes: Generate a gradient sensitivity map of the input space and quantify the degree of influence of each pixel on the output anomaly; Analyze the response consistency of attention weights across network layers to identify regions strongly correlated with backdoor triggering.

[0012] In one optional implementation, S3 achieves positioning through dual-channel fusion: Channel 1 performs spatial localization based on gradient sensitivity maps to identify suspicious spatial regions in the input data; Channel 2 performs semantic localization based on activation propagation paths, tracing unconventional feature propagation paths; The dual-channel output is integrated to generate a backdoor heatmap, marking areas where the trigger probability is higher than a set threshold.

[0013] In one alternative implementation, S4 includes: Establish independent behavioral baselines for convolutional layers, fully connected layers, and attention layers respectively; When the activation value of a specific functional layer deviates from the corresponding behavior baseline by more than a preset threshold, the layer is marked as a suspected backdoor layer.

[0014] In an optional implementation, S4 further includes: Starting from the marked backdoor suspected layer, trace the feature transfer path to the output layer; A multi-level transmission graph of backdoor behavior is generated based on the activation intensity of path nodes.

[0015] In one alternative implementation, it further includes: The new backdoor pattern detected by S2 is dynamically added to the spatiotemporal behavior pattern library; The positioning algorithm parameters of S3 and S4 are optimized based on the new mode to improve the generalization ability to unknown triggering modes.

[0016] As can be seen from the above, the backdoor behavior dynamic detection and localization method for deep learning models provided in this application effectively solves the problems of existing technologies such as inability to dynamically detect adaptive backdoor attacks, insufficient localization accuracy, and excessive computational latency by using real-time monitoring of input and output data streams, dynamic analysis of statistical features, fusion of gradient and attention mechanisms for localization, and hierarchical deconstruction of model functional units. It has the advantages of real-time dynamic detection, accurate localization of backdoor triggering areas, support for cross-model architecture universality, and reduced inference latency. Attached Figure Description

[0017] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating a method for dynamic detection and localization of backdoor behavior based on a deep learning model according to an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] In current technologies, the security of deep learning models has long faced the technical challenge of dynamic backdoor detection and localization. Traditional methods rely on static parameter analysis or offline auditing, which cannot cope with attack scenarios where triggering patterns change dynamically. In real-time video analytics systems, attackers may implant invisible triggers that vary with lighting conditions, causing traditional detection tools to miss detections. Existing solutions generally suffer from problems such as detection lag, ambiguous localization, and excessive resource consumption, making it difficult to meet the security requirements of real-time sensitive scenarios such as autonomous driving.

[0021] To address the aforementioned issues, the inventors discovered that the essence of dynamic backdoor attacks lies in the spatiotemporal anomalies of model behavior patterns. By analyzing the data characteristics of millions of inference processes, they found that backdoor triggering generates unique prediction distribution mutations and cross-layer activation anomalies. Based on this understanding, they proposed a technical approach that combines real-time behavior monitoring with hierarchical anomaly analysis. First, a dynamic behavior baseline is established to capture prediction fluctuations at the micro level; then, through multi-dimensional feature correlation, normal inference is distinguished from backdoor triggering; finally, a hierarchical localization mechanism is designed to achieve end-to-end tracking from the input space to the model's internal structure.

[0022] Therefore, as Figure 1 As shown, this application proposes a method for dynamic detection and localization of backdoor behavior based on deep learning models, including: Step S1: Monitor the input data and output prediction results of the deep learning model in real time during the inference process; Step S2: Based on the preset spatiotemporal behavior pattern library, perform statistical feature analysis on the output prediction results to identify abnormal deviations that exceed the normal behavior threshold. Step S3: In response to the anomaly determination in step S2, the input data region that triggers the backdoor behavior is located through reverse gradient tracing and attention weight structure analysis. Step S4: Deconstruct the deep learning model into independent analysis units according to functional layers, and realize the hierarchical localization of backdoor behavior by comparing activation values ​​layer by layer.

[0023] Real-time monitoring of input data and output prediction results refers to continuously capturing the data interaction process during model inference. This can be implemented using a streaming data processing framework to provide a real-time data source for dynamic detection. Statistical feature analysis involves extracting the distribution characteristics of the prediction results. This can be achieved using a sliding window algorithm to calculate temporal feature vectors for identifying abnormal behavior patterns. Backward gradient tracing calculates the sensitivity of input data to the prediction results. This can be achieved using integral gradient methods to generate heatmaps and reveal potential triggering areas. Hierarchical localization involves decomposing the model according to its network structure. This can be achieved using hierarchical activation monitoring technology to locate the propagation path of abnormal signals.

[0024] Specifically, this method continuously captures input / output data through streaming data processing to establish a dynamic behavioral baseline. When the entropy fluctuation and category jump frequency of the prediction result exceed an adaptive threshold, an anomaly detection mechanism is triggered. Subsequently, gradient backpropagation is used to generate an input space sensitivity map, and attention weight analysis is combined to identify cross-layer correlation regions, thus doubly verifying suspicious triggering regions. At the model structure level, by comparing the activation values ​​of each functional layer with the preset baseline, the network layer that generates the anomalous response is located, and the feature path propagating to the output layer is traced to form a complete chain of evidence for anomalous behavior.

[0025] Compared to existing technologies, traditional methods rely on fixed thresholds and a single detection dimension, failing to address dynamically changing triggering patterns. This invention utilizes a dynamic update mechanism based on a spatiotemporal behavior pattern library to continuously adapt to new attack characteristics. Existing localization techniques are often limited to independent analysis of the input space or model parameters, while this solution integrates dual-channel localization based on gradient sensitivity and attention correlation, effectively identifying the spatial distribution of diffusion-type triggers. Compared to online monitoring schemes requiring full gradient calculation, this method employs a hierarchical activation comparison strategy, significantly reducing computational resource consumption.

[0026] Through the above technical solution, this application can capture dynamic backdoor triggering behavior in real time during model operation, accurately identify triggering areas in the input data, and locate abnormal components inside the model. This solution effectively solves the shortcomings of traditional methods in terms of detection timeliness, location accuracy, and computational efficiency, providing dynamic security protection capabilities for deep learning models. Through multi-dimensional feature fusion and hierarchical analysis mechanisms, it can adapt to the security protection needs of different architecture models, achieving precise defense against complex backdoor attacks.

[0027] In one optional implementation, step S1 includes: A lightweight streaming sampling technique is used to extract a subset of the input data as monitoring samples. The time-series volatility index of the output confidence level is calculated in real time. When the volatility index exceeds the dynamic threshold, it is marked as a sudden change point in the predicted distribution.

[0028] Lightweight streaming sampling technology refers to dynamically extracting a subset of input data for monitoring within a time window. Specifically, it can be implemented using a sliding window random sampling algorithm, where each time window retains only a randomly selected subset of samples, with data from the previous window being overwritten in real time. This technology reduces computational resource consumption by decreasing the amount of data processed, while maintaining continuous monitoring capabilities over time.

[0029] The time-series volatility index is a quantitative indicator reflecting the change in model output confidence over time. Specifically, it can be calculated as the ratio of the standard deviation of the confidence level to the mean within a sliding window. By capturing the dynamic fluctuation characteristics of the confidence level distribution, this index can effectively identify prediction mutations caused by backdoor triggering.

[0030] Specifically, the input data stream is divided into continuous time segments using a sliding window, with a fixed proportion of samples randomly selected within each window for monitoring. Output confidence levels are recorded in real-time and input to the time-series analysis module. The volatility index is calculated by dividing the standard deviation of the confidence sequence within the current window by its mean. A dynamic threshold is periodically adjusted based on the moving average and standard deviation of historical volatility indices. When the real-time volatility index exceeds the current threshold, the system triggers an anomaly marking mechanism. This scheme, through a combination of streaming sampling and dynamic computation, maintains sensitivity in mutation detection while keeping the computational load at a constant level.

[0031] Through the above technical solution, this application can capture abrupt changes in the output distribution in real time during model inference. By dynamically adjusting the judgment threshold, it adapts to the normal drift of the model's prediction behavior, avoiding false alarms caused by fixed thresholds. The streaming sampling mechanism effectively balances detection accuracy and computational efficiency, enabling the solution to be deployed on edge devices with limited computing resources, thus overcoming the technical obstacle of traditional methods that cannot achieve real-time monitoring due to excessive computational load.

[0032] In one optional implementation, step S2 includes: During the detection process, a baseline feature set for normal behavior is constructed; the baseline feature set includes prediction entropy, class jump frequency, and confidence variance. An adaptive threshold algorithm is used to dynamically update the anomaly detection boundary. The adaptive threshold algorithm adjusts the boundary value in real time according to the behavior drift of the deep learning model.

[0033] Among these, prediction entropy refers to the entropy value of the model's output probability distribution. Specifically, it can be calculated using the information entropy formula to determine the degree of disorder in each prediction result, used to capture abnormal fluctuations in model decision uncertainty. Class jump frequency refers to the number of times the predicted class changes between adjacent time steps. Specifically, it can be calculated using the sliding window statistical method to determine the number of class switches per unit time, used to identify non-steady-state abrupt changes in the output results. Confidence variance refers to the dispersion of the model's output confidence score. Specifically, it can be measured using time series variance calculation methods to measure the deviation from prediction stability. The adaptive threshold algorithm is a calculation method for dynamically adjusting the anomaly judgment boundary. Specifically, it can use the exponentially weighted moving average method to track the distribution offset trend of feature data and dynamically reconstruct the judgment boundary based on the historical data mean and standard deviation.

[0034] Specifically, during model inference, data from three dimensions—prediction entropy, class jump frequency, and confidence variance—are collected in real time, and a time-series feature vector is constructed using a sliding window mechanism. Prediction entropy quantifies the model's level of certainty in deciding on the current input by calculating the Shannon entropy value of the output probability distribution; this value will exhibit abnormal fluctuations when a backdoor trigger is activated. Class jump frequency counts the number of times the prediction result changes class per unit time, used to capture output mutations caused by backdoor attacks. Confidence variance identifies abnormal stability changes in the output distribution by calculating the dispersion of continuous prediction confidence. The combination of these three orthogonal features forms a multi-dimensional monitoring system capable of covering different abnormal behavior patterns of the model.

[0035] The adaptive thresholding algorithm is based on the exponentially weighted moving average method, establishing a dynamic baseline for each feature dimension. The algorithm adjusts the weighting of historical data on the current threshold by setting a decay factor. When a feature value continuously deviates from the baseline, it automatically expands the judgment boundary; conversely, it shrinks the boundary range. For example, when the model's prediction entropy baseline value drifts due to parameter updates, the algorithm automatically adjusts the entropy anomaly judgment threshold to avoid false positives or false negatives caused by a fixed threshold.

[0036] Compared to existing technologies, traditional static threshold detection methods rely on manually setting fixed judgment boundaries, which cannot adapt to behavioral changes caused by data distribution drift or parameter updates during model operation, easily leading to false alarms. Our solution, however, utilizes multi-dimensional feature joint analysis and a dynamic threshold adjustment mechanism to automatically track the evolution trend of model behavior and correct anomaly judgment criteria in real time. For example, when the input data domain shifts after model deployment, static threshold methods generate numerous false alarms, while our adaptive algorithm can recalibrate the judgment boundaries based on the actual feature distribution, maintaining detection accuracy.

[0037] Through the above technical solutions, this application solves the problems of misjudgment and missed detection caused by the dynamic behavior changes of the model in traditional static threshold methods. The multi-dimensional feature monitoring system can capture abnormal signals of backdoor behavior from different dimensions, avoiding the limitations of single index analysis. The dynamic threshold adjustment mechanism keeps the anomaly detection standard synchronized with the current model state by continuously tracking the drift trend of model behavior, effectively improving the adaptability of the detection system to changes in the operating environment.

[0038] In an optional implementation, step S2 further includes: When statistical anomalies are detected, controllable micro-perturbations are injected into the input data; If the degree of abrupt change in the output prediction is in a non-linear proportional relationship with the intensity of the disturbance, it is determined to be backdoor behavior.

[0039] Among them, controllable micro-perturbation refers to a sequence of small perturbations that are dynamically adjusted according to the characteristics of the input data. Specifically, it can be achieved by superimposing Gaussian noise or using adversarial perturbation generation techniques. The perturbation intensity is controlled within the normal robustness range of the model to avoid interfering with the normal inference process.

[0040] Among them, the nonlinear proportional relationship refers to the exponential or threshold jump correlation between the output prediction mutation degree and the disturbance intensity. Specifically, it can be achieved by calculating the Pearson correlation coefficient and comparing it with the preset linearity threshold. The presence of superlinear sensitivity features can be determined by fitting the disturbance-response curve through a statistical model.

[0041] Specifically, upon detecting that the prediction entropy or confidence variance exceeds the normal range, a micro-perturbation sequence optimized along the gradient direction is injected into the input data. By monitoring the frequency of class jumps in the model output under different perturbation intensities, a correlation curve between perturbation intensity and the degree of prediction mutation is constructed. When this curve exhibits non-linear characteristics, such as a sharp increase in prediction mutations after a specific perturbation threshold, it is determined to be backdoor-triggered behavior. This process effectively distinguishes between the smooth response of a normal model to noise and the sensitive triggering characteristics of backdoor behavior by actively constructing a controlled experimental environment.

[0042] Compared to existing technologies, traditional methods rely on fixed thresholds to identify abnormal behavior, failing to distinguish between fluctuations caused by normal data perturbations and abrupt responses triggered by backdoors. Existing technologies typically employ full data perturbation or fixed-pattern injection, resulting in excessive computational overhead and susceptibility to normal robustness interference. This proposed solution significantly improves discrimination accuracy while maintaining low computational load through directional injection of controllable micro-perturbations and nonlinear response analysis.

[0043] Through the above technical solution, this application effectively solves the technical problem of distinguishing between normal data disturbances and backdoor triggering behavior. By dynamically constructing the disturbance-response relationship curve, it identifies abnormal sensitive areas in real time during model inference. This solution can accurately identify the activation threshold characteristics of backdoor triggers, avoid misjudgments caused by normal noise interference, and maintain computational resource consumption that is 30% lower than that of traditional methods.

[0044] In one optional implementation, step S3 includes: Generate a gradient sensitivity map of the input space and quantify the degree of influence of each pixel on the output anomaly; Analyze the response consistency of attention weights across network layers to identify regions strongly correlated with backdoor triggering.

[0045] Generating a gradient sensitivity map of the input space involves calculating the gradient contribution of each pixel in the input data to the model's anomaly prediction, forming a spatial sensitivity distribution map. This can be achieved by using the backpropagation algorithm to calculate the partial derivatives of the input data with respect to the anomaly output. This feature quantifies the influence of different regions of the input data on backdoor behavior. Analyzing the response consistency of attention weights across network layers compares the degree to which attention mechanisms in different network layers focus on specific regions of the input data. This can be achieved by calculating the spatial correlation coefficients of the attention weight matrices of each layer. This feature reveals the propagation path of the backdoor trigger signal within the model.

[0046] Specifically, in the gradient sensitivity map generation process of the input space, the absolute value of the gradient of each pixel in the input data with respect to the abnormal output is obtained through the backpropagation algorithm. After normalizing the gradient values, a heatmap is formed, where highly sensitive areas correspond to possible backdoor trigger locations. In the cross-layer attention analysis process, for the multi-head attention layers in the Transformer architecture, the Pearson correlation coefficient of attention weights at different levels is calculated in the spatial dimension. When a specific region maintains a correlation coefficient higher than 0.8 in three or more consecutive network layers, it is determined to be a cross-layer associated region. The gradient sensitivity map provides a spatial localization benchmark, and attention consistency analysis verifies semantic correlation. The synergistic effect of the two can effectively identify the spatial distribution of diffusion-type triggers and their propagation paths within the model.

[0047] Compared to existing technologies, traditional methods such as Neural Cleanse rely solely on gradient sensitivity maps for single-layer analysis, failing to identify dynamically changing triggering patterns. Furthermore, single-layer detection based on attention weights easily misses cross-layer propagation features. This proposed solution, by fusing gradient sensitivity and cross-layer attention correlation into a dual verification mechanism, improves trigger region localization accuracy from the image patch level of traditional methods to the pixel level while maintaining black-box detection conditions. It also achieves, for the first time, the visual tracking of the trigger signal propagation path.

[0048] Through the above technical solution, this application can accurately locate the spatial region in the input data that triggers backdoor behavior without accessing the model's internal parameters. Particularly for diffusion-type dynamic triggers such as color gradients and texture blending, it can accurately identify their physical boundaries and propagation trajectories across different levels of the model. This solution effectively solves the technical shortcomings of traditional localization methods, such as poor adaptability to dynamic triggering patterns and lack of cross-layer correlation analysis, providing a dual localization basis of spatial location and semantic propagation for backdoor behavior analysis.

[0049] In one optional implementation, step S3 achieves positioning through dual-channel fusion: Channel 1 performs spatial localization based on gradient sensitivity maps to identify suspicious spatial regions in the input data; Channel 2 performs semantic localization based on activation propagation paths, tracing unconventional feature propagation paths; The dual-channel output is integrated to generate a backdoor heatmap, marking areas where the trigger probability is higher than a set threshold.

[0050] The gradient sensitivity map is a visual heatmap generated by calculating the gradient change of each pixel in the input data with respect to the output prediction. Specifically, it can be implemented by using the backpropagation algorithm to calculate the partial derivatives of the abnormal nodes in the output layer with respect to the input data, which is used to quantify the degree of influence at the pixel level.

[0051] The activation propagation path refers to the node connection path where the activation value deviates significantly from the normal pattern during the feature transmission process within the neural network. Specifically, it can be implemented using layer-by-layer activation value tracking and path correlation analysis algorithms to capture abnormal propagation chains at the semantic level.

[0052] Among them, the backdoor heatmap refers to a visualized probability distribution map that integrates spatial positioning results and semantic path analysis. Specifically, it can be implemented using a weighted fusion algorithm of spatial probability matrix and semantic path weight, and is used to cross-validate the credibility of the triggering region.

[0053] Specifically, Channel 1 calculates the gradient contribution of each region of the input data to the output anomaly through backpropagation, generates a spatial sensitivity map, and filters out regions with gradient values ​​higher than a dynamic threshold as physically suspicious regions. Channel 2 analyzes the propagation paths of activation values ​​in each layer of the neural network, identifying anomalous paths that deviate significantly from the normal inference pattern as semantically suspicious paths. The fusion process of the dual-channel outputs uses a weighted superposition algorithm of spatial probability matrix and semantic path weights to generate a comprehensive heatmap. When the trigger probability of a certain region exceeds a set threshold in both the spatial and semantic dimensions, it is marked as a high-confidence backdoor trigger region.

[0054] Compared to existing technologies, traditional localization methods rely solely on single-dimensional gradient analysis or activation tracking, failing to analyze the spatial distribution and semantic propagation relationship of dynamic triggers. For example, the existing BEAT technology only outputs binary judgment results, while this solution, through a dual-channel cross-validation mechanism, can both locate the physical position of the trigger and reveal its propagation path within the model, achieving accurate analysis of diffusion-type dynamic triggers.

[0055] Through the above technical solution, this application can simultaneously capture the spatial distribution features and semantic propagation patterns of backdoor triggers without accessing the model's internal parameters, effectively solving the problem of missed detection of composite dynamic triggers by traditional methods. By employing a probabilistic fusion mechanism with dual-channel output, the false positive rate of single-dimensional localization is reduced, improving the detection accuracy for semantically perturbative backdoors and diffusion-type triggers. This is applicable to pixel-level anomaly localization in computer vision models and semantic path tracking in natural language processing models.

[0056] In one alternative implementation, step S4 includes: Establish independent behavioral baselines for convolutional layers, fully connected layers, and attention layers respectively; When the activation value of a specific functional layer deviates from the corresponding behavior baseline by more than a preset threshold, the layer is marked as a suspected backdoor layer.

[0057] Functional layer deconstruction refers to breaking down a deep learning model into network layer units with independent functions. This can be achieved using modular decomposition methods, such as breaking down a convolutional neural network into a sequence of convolutional layers and a sequence of fully connected layers, or breaking down a Transformer model into a sequence of attention layers. This feature allows the detection process to perform independent analysis on different functional layers.

[0058] The independent behavior baseline refers to the standard range of normal activation values ​​established for different functional layer types. Specifically, it can be implemented using statistical distribution modeling of historical operational data. For example, for convolutional layers, a baseline of the mean and variance of feature map activation values ​​can be used, and for attention layers, an entropy baseline of multi-head attention weights can be used. This feature provides differentiated detection benchmarks for different functional layers.

[0059] Activation value deviation refers to the degree of difference between the network layer output tensor and the corresponding behavioral baseline. This can be calculated using cosine similarity or KL divergence, for example, by using a sliding window to statistically analyze the difference between the current activation value distribution and the baseline distribution. This feature can quantify the magnitude of deviation from anomalous behavior.

[0060] Specifically, during model inference, the network architecture is first deconstructed and divided according to functional layer type, forming independent analysis units. A baseline for feature map activation intensity is established for convolutional layers, a baseline for weight update frequency is established for fully connected layers, and a baseline for cross-head correlation is established for attention layers. During real-time monitoring, the activation output of each functional layer is dynamically sampled, and its deviation from the corresponding baseline is calculated. When the feature map activation intensity of a convolutional layer exceeds its baseline threshold, the convolutional layer is determined to have abnormal behavior; when the weight update frequency of a fully connected layer deviates from its baseline range, the fully connected layer is determined to have abnormal behavior. By setting categorized baselines, the normal behavior patterns of different functional layers are effectively distinguished, avoiding misjudgments caused by a uniform detection standard.

[0061] Through the above technical solution, this application can accurately locate the specific network layer where backdoor behavior occurs. For example, it can identify abnormal feature activation in the third convolutional layer in an image classification model, or locate weight shift in the second attention layer in a text model. This hierarchical localization provides clear detection targets for subsequent analysis, enabling maintenance personnel to quickly identify suspicious network layers with backdoor implantation and avoid the resource consumption of performing full-scale detection on the entire model.

[0062] In an optional implementation, step S4 further includes: Starting from the marked backdoor suspected layer, trace the feature transfer path to the output layer; A multi-level transmission graph of backdoor behavior is generated based on the activation intensity of path nodes.

[0063] Among them, the backdoor suspected layer refers to the functional layer that is identified as deviating from the normal behavior baseline by comparing the activation values ​​layer by layer. Specifically, it can be implemented by calculating the deviation of the behavior baseline independently using convolutional layers, fully connected layers, or attention layers, and is used to determine the starting position of abnormal features inside the model.

[0064] The feature propagation path refers to the computational topology link from the suspect layer to the output layer within the model. Specifically, it can be implemented by using the backpropagation algorithm or the activation propagation algorithm to trace the connection relationship between each network layer, in order to reveal the direction of the diffusion of abnormal features in the model.

[0065] Activation intensity refers to the degree of response of a neural network node to a specific input. Specifically, it can be achieved by calculating the norm of the feature map output of each layer or the maximum value of the channel during the forward propagation process, which is used to quantify the influence of anomalous features during the propagation process.

[0066] Among them, the multi-level transmission map refers to the visual representation of the integrated network hierarchical structure and activation intensity. Specifically, it can be generated by overlaying a heat map with a topology map. The heat values ​​map the contribution of each layer node to the backdoor behavior and are used to show the spatial distribution and intensity changes of the abnormal propagation.

[0067] Specifically, when an activation value of a convolutional layer deviates from a preset behavior baseline, all connection paths from that layer back along the model computation graph to the output layer are traced. During path tracing, activation intensity data for each intermediate layer is recorded, including the weight product results of fully connected layers and the correlation matrix values ​​of attention layers. By aggregating the activation intensity data of each layer, a propagation graph containing both network depth and spatial dimensions is generated, where highlighted areas represent key path nodes for the propagation of anomalous features. This achieves the reconstruction of the complete propagation chain from the initial anomalous layer to the final output prediction, revealing the diffusion mechanism of backdoor triggers within the model.

[0068] Through the above technical solution, this application solves the problem of blind repair caused by the invisibility of the backdoor behavior propagation path. It can clearly indicate the diffusion path and key influencing nodes of abnormal features within the model, providing a hierarchical positioning basis for targeted repair. At the same time, it supports the analysis of the differences in backdoor behavior patterns of different network architectures through transmission graph analysis.

[0069] In an optional implementation, the method further includes: The new backdoor pattern detected in step S2 is dynamically added to the spatiotemporal behavior pattern library; The positioning algorithm parameters of steps S3 and S4 are optimized based on the newly added modes to improve the generalization ability to unknown triggering modes.

[0070] Among them, the spatiotemporal behavior pattern library refers to a dynamic knowledge base that stores multi-dimensional temporal features and spatial correlation features. Specifically, it can be implemented using an incremental graph database. By using timestamp indexes and feature similarity clustering to build a correlation network between patterns, this library accumulates defense knowledge by continuously integrating new attack features.

[0071] Among them, the optimization of localization algorithm parameters refers to adjusting the gradient sensitivity threshold and attention association weight based on the newly added pattern features. Specifically, it can be implemented using a reinforcement learning framework. By designing a reward function, the parameters are guided to evolve in the direction of improving the recognition rate of unknown patterns. This optimization process enables the localization module to adapt to dynamic attacks.

[0072] Specifically, when a backdoor behavior not recorded in the pattern library is detected, its spatiotemporal feature vector is extracted and added to the incremental storage area of ​​the pattern library. The pattern library update trigger parameter optimization module is activated. This module analyzes the differences between newly added patterns and historical patterns, adjusts the sensitivity threshold of the gradient tracking algorithm to capture more subtle anomaly signals, and corrects the correlation coefficient in the attention weight analysis to enhance the spatial correlation recognition of new triggers. This closed-loop feedback mechanism enables the localization system to automatically adjust its detection strategy based on the latest threat characteristics. For example, when a time-dependent dynamic trigger is detected, the system automatically increases the detection weight of temporal fluctuation features.

[0073] In some specific implementations, the pattern library can be updated using a sliding window mechanism to maintain storage capacity, for example, retaining new patterns detected within the last three months. The parameter optimization process can incorporate online learning algorithms, such as initiating a batch parameter adjustment after every ten new triggering patterns are detected.

[0074] Compared to existing technologies, traditional solutions rely on static feature libraries and fixed-parameter algorithms, making them unable to adapt to changes in triggering patterns. This solution accumulates multi-dimensional attack features through a dynamically updated pattern library, combined with an adaptive parameter adjustment mechanism, enabling the system to continuously learn the characteristic patterns of new attack modes, thus overcoming the dependence on predefined attack features.

[0075] Through the above technical solution, this application enables the backdoor defense system to continuously accumulate new attack characteristics during operation, automatically optimize detection strategies to adapt to the ever-evolving threat forms, effectively improve the accuracy of identifying unknown backdoor triggering modes, and enhance the robustness of the defense system in the face of zero-day attacks.

[0076] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for dynamic detection and localization of backdoor behavior in deep learning models, characterized in that, include: S1. Real-time monitoring of the input data and output prediction results of the deep learning model during the inference process; S2. Based on a preset spatiotemporal behavior pattern library, perform statistical feature analysis on the output prediction results to identify abnormal deviations that exceed the normal behavior threshold. S3. In response to the anomaly determination in S2, the input data region that triggers the backdoor behavior is located through reverse gradient tracing and attention weight structure analysis. S4. Deconstruct the deep learning model into independent analysis units according to functional layers, and realize the hierarchical localization of backdoor behavior by comparing activation values ​​layer by layer.

2. The method according to claim 1, characterized in that, S1 includes: A lightweight streaming sampling technique is used to extract a subset of the input data as monitoring samples. The time-series volatility index of the output confidence level is calculated in real time. When the volatility index exceeds the dynamic threshold, it is marked as a sudden change point in the predicted distribution.

3. The method according to claim 1, characterized in that, S2 includes: Construct a baseline feature set for normal behavior; the baseline feature set includes prediction entropy, class jump frequency, and confidence variance; An adaptive threshold algorithm is used to dynamically update the anomaly detection boundary. The adaptive threshold algorithm adjusts the boundary value in real time according to the behavior drift of the deep learning model.

4. The method according to claim 3, characterized in that, S2 further includes: When statistical anomalies are detected, controllable micro-perturbations are injected into the input data; If the degree of abrupt change in the output prediction is in a non-linear proportional relationship with the intensity of the disturbance, it is determined to be backdoor behavior.

5. The method according to claim 1, characterized in that, S3 includes: Generate a gradient sensitivity map of the input space and quantify the impact of each pixel on the output anomaly. Analyze the response consistency of attention weights across network layers to identify regions strongly correlated with backdoor triggering.

6. The method according to claim 5, characterized in that, The S3 achieves positioning through dual-channel fusion: Channel 1 performs spatial localization based on gradient sensitivity maps to identify suspicious spatial regions in the input data; Channel 2 performs semantic localization based on activation propagation paths, tracing unconventional feature propagation paths; The dual-channel output is integrated to generate a backdoor heatmap, marking areas where the trigger probability is higher than a set threshold.

7. The method according to claim 1, characterized in that, S4 includes: Establish independent behavioral baselines for convolutional layers, fully connected layers, and attention layers respectively; When the activation value of a specific functional layer deviates from the corresponding behavior baseline by more than a preset threshold, the layer is marked as a suspected backdoor layer.

8. The method according to claim 7, characterized in that, S4 further includes: Starting from the marked backdoor suspect layer, trace the feature propagation path to the output layer; A multi-level transmission graph of backdoor behavior is generated based on the activation intensity of path nodes.

9. The method according to claim 1, characterized in that, Also includes: The new backdoor pattern detected by S2 is dynamically added to the spatiotemporal behavior pattern library; The positioning algorithm parameters of S3 and S4 are optimized based on the new mode to improve the generalization ability to unknown triggering modes.

Images