An artificial intelligence-based backdoor attack detection method

Abstract

The application relates to the field of network security, and specifically discloses a backdoor attack detection method based on artificial intelligence, which applies a series of standardized disturbances to clean samples systematically, quantifies the response change of model output probability relative to baseline prediction, and thus constructs a comprehensive response matrix. In addition, by performing cross-class response spectrum consistency aggregation on the response matrix, the purpose is to amplify the universal and trigger-dominated deterministic signal and suppress the random noise related to specific samples. Finally, through suspiciousness scoring, abnormal response patterns caused by backdoor attacks can be accurately identified, so that the detection task can be efficiently completed and the backdoor information can be located without relying on internal information of the model, and the problem of insufficient detection capability of the prior art in a limited access scene is solved.

Description

Technical Field

[0001] This application relates to the field of cybersecurity, and more specifically, to an artificial intelligence-based method for detecting backdoor attacks. Background Technology

[0002] With the rapid development and widespread application of artificial intelligence (AI) technology, the deployment of pre-trained models across various industries has become commonplace. Enterprises and developers are increasingly acquiring powerful models through Model-as-a-Service (MAS) platforms or open-source communities to accelerate product development. However, this open and collaborative model has also introduced new security vulnerabilities, among which backdoor attacks pose a particularly serious and insidious threat. Attackers can implant malicious samples with specific triggers during the model training phase, planting backdoors in the model. The model behaves normally when processing normal data, but once the input contains a pre-set trigger, its behavior is hijacked, forcing the output of malicious results specified by the attacker. The stealth and potential harm of this attack make the effective detection of the security and reliability of AI models an urgent need to ensure the healthy development of AI technology.

[0003] Current backdoor attack detection methods primarily rely on white-box analysis, such as techniques like neuron pruning or activation clustering. These methods require complete access to the model's internal weights, gradients, and network structure. While these methods allow for in-depth analysis of the model's internal mechanisms, their application is severely limited. In real-world business environments, due to intellectual property protection or business model considerations, model providers typically only offer an inference interface, preventing users from accessing detailed information about the model's internals. This renders white-box detection methods completely ineffective in gray-box or black-box scenarios like Model-as-a-Service (MAS). Another approach attempts to reverse engineer potential triggers through input optimization, but these methods often incur huge computational costs and are less effective for complex or semantically sensitive triggers distributed throughout the input space. Therefore, developing an efficient and accurate backdoor detection method under conditions of restricted model access is a key technical challenge facing the field of AI security.

[0004] Therefore, an optimized AI-based backdoor attack detection scheme is desired. Summary of the Invention

[0005] To address the aforementioned technical problems, this application is proposed. Embodiments of this application provide an artificial intelligence-based backdoor attack detection method.

[0006] According to one aspect of this application, an artificial intelligence-based backdoor attack detection method is provided, comprising:

[0007] Obtain a clean test set;

[0008] Baseline predictions are performed based on a clean test set to obtain baseline prediction results;

[0009] Perturbations are added to the baseline prediction results to obtain a perturbation input corpus;

[0010] The model query and response difference calculation are performed on the perturbation input corpus and baseline prediction results to obtain the response matrix;

[0011] Cross-class response spectrum consistency aggregation is performed on the response matrix to obtain the suspicion score vector;

[0012] Backdoor detection and trigger identification are performed based on the suspiciousness score vector to obtain detection results and backdoor information.

[0013] Compared to existing technologies, this application provides an AI-based backdoor attack detection method that detects backdoor attacks in black-box or gray-box scenarios where internal model parameters are inaccessible. Specifically, it constructs a comprehensive response matrix by systematically applying a series of standardized perturbations to clean samples and quantifying the change in the model's output probability relative to the baseline prediction. Furthermore, considering that the malicious behavior triggered by a genuine backdoor trigger possesses a unique fingerprint with cross-class consistency—meaning its effect is independent of the original sample's category and consistently points to a single malicious target—this solution amplifies this universal, trigger-driven deterministic signal by performing cross-class response spectrum consistency aggregation on the response matrix, while suppressing random noise related to specific samples. Finally, through suspiciousness scoring, it can accurately identify abnormal response patterns caused by backdoor attacks, thus efficiently completing the detection task and locating backdoor information without relying on internal model information, solving the problem of insufficient detection capabilities in restricted access scenarios in existing technologies. Attached Figure Description

[0014] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0015] Figure 1 This is a flowchart of an AI-based backdoor attack detection method according to an embodiment of this application;

[0016] Figure 2 This is a schematic diagram of the data flow of an AI-based backdoor attack detection method according to an embodiment of this application;

[0017] Figure 3 This is a flowchart illustrating the process of obtaining a response matrix by performing model query and response difference calculation on a perturbation input corpus and baseline prediction results according to the AI-based backdoor attack detection method of this application.

[0018] Figure 4 This is a flowchart illustrating the process of performing cross-class response spectrum consistency aggregation on the response matrix to obtain a suspiciousness score vector according to the AI-based backdoor attack detection method of this application. Detailed Implementation

[0019] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0020] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not specifically singular and may include plural forms. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0021] To address the technical challenge of traditional backdoor detection methods failing in black-box scenarios such as Model-as-a-Service (MAS) due to the inability to access internal model parameters, this application proposes an AI-based backdoor attack detection method. The implementation begins by establishing a reliable performance benchmark. First, a clean and class-balanced test set is used to query the target model to obtain its baseline predicted probability distribution under normal input. Then, a standardized perturbation space (e.g., containing pixel patches of different positions and colors) is defined, and each perturbation primitive is systematically applied to all clean samples, generating a massive perturbation input corpus. Further, these massive perturbation samples are input back into the model, and their output probabilities are compared one by one with the corresponding baseline probabilities to calculate the response differences, thereby constructing a response matrix that quantifies the model's sensitivity to various perturbations. Subsequently, cross-class response spectrum consistency aggregation analysis is performed on this matrix: it does not view individual responses in isolation but seeks a universal key—that is, a specific perturbation, regardless of the class of the original sample, can stably and significantly guide the model's output towards the same target class. This highly consistent response pattern, transcending category boundaries, is a typical fingerprint of backdoor attacks. By quantifying this consistency through a scoring mechanism, a suspiciousness score vector can be generated. An abnormally high score can not only accurately reveal the existence of a backdoor, but also simultaneously pinpoint the target category and trigger pattern of the attack. Thus, accurate detection of backdoor attacks is achieved without relying on internal model information.

[0022] Figure 1 This is a flowchart of an AI-based backdoor attack detection method according to an embodiment of this application. Figure 2 The data flow diagram of the AI-based backdoor attack detection method according to the embodiments of this application is as follows: Figure 1 and Figure 2 As shown, the AI-based backdoor attack detection method according to an embodiment of this application includes: S100, obtaining a clean test set; S200, performing baseline prediction based on the clean test set to obtain a baseline prediction result; S300, adding perturbations to the baseline prediction result to obtain a perturbation input corpus; S400, performing model query and response difference calculation on the perturbation input corpus and the baseline prediction result to obtain a response matrix; S500, performing cross-class response spectrum consistency aggregation on the response matrix to obtain a suspiciousness score vector; S600, performing backdoor determination and trigger identification based on the suspiciousness score vector to obtain detection results and backdoor information.

[0023] Specifically, in steps S100 and S200, a clean test set is obtained, and baseline prediction is performed based on the clean test set to obtain the baseline prediction result. It should be understood that since backdoor detection relies on the accurate quantification of the model's response changes before and after perturbation, the uneven distribution of categories in the original test set can introduce statistical bias, interfering with the judgment of the true response changes. Therefore, in the technical solution of this application, a clean test set is obtained, and baseline prediction is performed based on the clean test set to obtain the baseline prediction result, thereby establishing an unbiased and standardized model response benchmark for each clean sample. This provides a stable and reliable reference for subsequent response difference calculations, ensuring the accuracy and reliability of subsequent analyses.

[0024] More specifically, in this embodiment, baseline prediction based on a clean test set to obtain a baseline prediction result includes: extracting a balanced test subset from the clean test set, wherein the number of samples in each category of the balanced test subset is equal; inputting each sample tuple from the balanced test subset into the inference interface of the model to be detected to obtain a baseline probability vector corresponding to each sample tuple; and combining each sample tuple and the baseline probability vector corresponding to each sample tuple to obtain the baseline prediction result. That is, in a specific example of this application, firstly, from an original clean test set containing a large number of labeled handwritten digit (0 to 9) images, a fixed number (e.g., 50) of image samples are randomly selected for each digit category (0, 1, 2...9) to construct a balanced test subset containing a total of 500 samples, wherein the number of samples in each category of the subset is strictly equal. Then, each image sample in the balanced test subset is traversed and transmitted as input data to the inference interface of the handwritten digit recognition model to be detected. After receiving the image data, the model returns a 10-dimensional baseline probability vector. Each component of this vector corresponds to the confidence level of the model predicting the image as a digit from 0 to 9. Finally, the original image sample data and its corresponding 10-dimensional baseline probability vector are combined and stored as a data unit. After processing all 500 samples, a structured baseline prediction result set is obtained, laying the data foundation for subsequent perturbation analysis.

[0025] Specifically, in step S300, perturbations are added to the baseline prediction results to obtain a perturbed input corpus. It should be understood that, due to the highly covert nature of backdoor attack triggering mechanisms, simply analyzing the model's response to clean samples cannot expose its potential vulnerabilities. It is necessary to actively and systematically investigate the model's behavioral stability in the face of various minute input changes. Therefore, in the technical solution of this application, perturbations are further added to the baseline prediction results to obtain a perturbed input corpus, thereby generating a large-scale perturbed input corpus covering multiple potential triggering patterns. This provides sufficient and diverse input data for subsequent model query and response difference analysis, thus ensuring the comprehensiveness and effectiveness of the detection process.

[0026] More specifically, in the embodiments of this application, perturbation is added to the baseline prediction results to obtain a perturbation input corpus, including: obtaining a perturbation parameter set; performing a Cartesian product on each dimension of the perturbation parameter set to obtain a perturbation space; extracting a clean sample set from the baseline prediction results; and applying perturbation to each clean sample in the clean sample set based on the perturbation space to obtain a perturbation input corpus.

[0027] In a specific example of this application, the process first involves procedurally constructing a standardized perturbation space, and then applying each perturbation primitive within that space to all clean samples. First, a perturbation parameter set is defined, where the position dimension contains the coordinates of the four corner points of the image, and the feature dimension contains pure white and pure black pixel values. By performing a Cartesian product on these two dimensions, a perturbation space containing eight independent perturbation primitives is generated; for example, one perturbation primitive is defined as applying a 3x3 white pixel block to the upper left corner of the image. Next, all 500 clean handwritten digit images are extracted from the baseline prediction results generated in the preceding steps. These 500 images are iterated through, and all eight perturbation primitives in the perturbation space are applied sequentially to each image. Each application operation generates a new perturbated image, which is then combined with the original image and the applied perturbation primitives into an information tuple. Finally, after completing all combinations, a perturbation input corpus containing 4000 such information tuples is formed. This corpus records the context information of each perturbation operation in detail. Each tuple in the perturbation input corpus includes a clean sample, the applied perturbation primitive, and the perturbated sample.

[0028] Specifically, in step S400, a model query and response difference calculation are performed on the perturbation input corpus and baseline prediction results to obtain a response matrix. It should be understood that the original output probabilities of the perturbation-triggered samples do not directly reveal the abnormal behavior of the model; key information is contained in their deviation from the unperturbed state. Therefore, in the technical solution of this application, a model query and response difference calculation are further performed on the perturbation input corpus and baseline prediction results to obtain a response matrix, thereby accurately separating and quantifying the specific impact of each standardized perturbation on the model prediction confidence. This transforms the dynamic response behavior of the model into a structured response matrix that facilitates subsequent statistical analysis, laying a data foundation for identifying systematic abnormal patterns.

[0029] Figure 3 This is a flowchart illustrating the process of obtaining a response matrix by performing model query and response difference calculation on a perturbed input corpus and baseline prediction results according to an embodiment of this application's AI-based backdoor attack detection method. Figure 3 As shown, step S400 includes: S410, inputting each perturbed sample in the perturbed input corpus into the inference interface of the model to be detected to obtain the output probability vector; S420, extracting the baseline probability vector corresponding to the perturbed sample from the baseline prediction result; S430, calculating the response difference vector between the output probability vector and the baseline probability vector, wherein the response difference vector is the row vector in the response matrix.

[0030] Accordingly, in step S410, each perturbed sample from the perturbation input corpus is input into the inference interface of the model to be detected to obtain the output probability vector. It should be understood that subsequent response difference calculations must be based on the model's actual predictions of the perturbed samples, and these results can only be obtained by directly querying the model. Therefore, in the technical solution of this application, each perturbed sample from the perturbation input corpus is further input into the inference interface of the model to be detected to obtain the output probability vector, thereby systematically collecting the complete output behavior of the model under the influence of each standardized perturbation. This provides the necessary raw data, corresponding one-to-one with the baseline prediction results, for subsequent difference calculations, ensuring the data completeness of the response change quantification process.

[0031] Specifically, in one example of this application, a perturbed input corpus containing 4000 information tuples generated in the preceding steps is programmatically traversed. In each iteration, the perturbed image data is extracted from the current tuple. This image data is sent to the inference interface of the handwritten digit recognition model to be detected. Upon receiving the input, the model performs forward propagation computation and returns a 10-dimensional floating-point vector, where each element represents the probability confidence that the model determines the image belongs to the corresponding digit category (0 to 9). This output probability vector is stored and a clear correspondence is established between it and the source perturbed sample. This process is repeated until all 4000 perturbed samples in the corpus have been processed, ultimately generating a set containing 4000 output probability vectors, providing complete and necessary experimental data for the next step of response difference calculation.

[0032] Accordingly, in step S420, a baseline probability vector corresponding to the perturbed sample is extracted from the baseline prediction result. It should be understood that since the calculation of the response difference must be based on a comparison of the same original sample before and after the perturbed state, any mismatched comparison will lead to invalid and erroneous difference results. Therefore, in the technical solution of this application, a baseline probability vector corresponding to the perturbed sample is further extracted from the baseline prediction result to accurately match a unique and correct reference benchmark for each independent perturbed event. This ensures that the two operands of the subsequent difference operation have a strict correspondence, thereby guaranteeing that the calculated response difference vector can truly reflect the impact of the perturbed.

[0033] Specifically, in one example of this application, when the system processes a specific perturbation sample, for example, a sample generated from the original handwritten digit image No. 128 (let's say 5) and the perturbation primitive No. 6 (e.g., the black pixel block in the lower right corner), the system first identifies the source of the sample as the original image No. 128. Then, it accesses the baseline prediction result set containing 500 data units and directly locates the data unit associated with the original image No. 128. From this data unit, a pre-stored 10-dimensional baseline probability vector is extracted. This vector is the unique baseline reference corresponding to this specific perturbation sample. This retrieval process is performed alongside the processing of each perturbation sample, ensuring that the correct baseline data is used in all 4000 response difference calculations.

[0034] Accordingly, in step S430, the response difference vector between the output probability vector and the baseline probability vector is calculated, where the response difference vector is a row vector in the response matrix. It should be understood that since the original output probability vector and the baseline probability vector of the model only represent two independent states, they cannot directly reveal the dynamic changes caused by the perturbation. Therefore, in the technical solution of this application, the response difference vector between the output probability vector and the baseline probability vector is further calculated to explicitly and quantitatively represent the implicit behavioral changes of the model. This generates a structured data unit that directly reflects the direction and magnitude of the perturbation's influence, namely the response difference vector. This vector, as a basic component of the response matrix, provides a direct analytical object for subsequent pattern recognition.

[0035] Specifically, in this embodiment, calculating the response difference vector between the output probability vector and the baseline probability vector includes: calculating the positional difference between the output probability vector and the baseline probability vector to obtain the response difference vector. That is, in a specific example of this application, the system processes a sample generated by perturbing the top-left white pixel block of an original handwritten digit 7 image. Assume the baseline probability vector of the original 7 image is [0.01, 0.02, ..., 0.94, ...], where the 8th element (corresponding to the digit 7) has the highest confidence. After applying the perturbation, the model output probability vector becomes [0.01, 0.85, ..., 0.10, ...], where the confidence of the 2nd element (corresponding to the digit 1) increases significantly. At this point, positional difference is performed, i.e., subtracting the former vector from the latter vector, to obtain the response difference vector [-0.00, +0.83, ..., -0.84, ...]. This vector clearly shows that the perturbation increased the model's confidence in predicting the number 1 by 0.83, while decreasing the confidence in predicting the original correct class 7 by 0.84. This calculated 10-dimensional response difference vector was then stored as a row vector in the response matrix being constructed.

[0036] Specifically, in step S500, the response matrix undergoes cross-class response spectrum consistency aggregation to obtain a suspiciousness score vector. It should be understood that the original response matrix contains a large amount of discrete response data, which may contain both systemic signals triggered by backdoor triggers and random fluctuations caused by model characteristics or data noise. Existing backdoor suspiciousness scoring mechanisms primarily rely on simply summing the probability offsets generated by different samples and taking the maximum value as the final judgment criterion. This mechanism has a fundamental flaw: it cannot distinguish between a cumulative effect stably and consistently contributed by a large number of samples and a cumulative effect dominated by an extreme anomalous response from a very small number of samples. A well-designed backdoor attack should have a universal trigger effect, meaning it should stably induce a decision shift towards the malicious target on various non-target samples. The original mechanism simply pursues the sum of response intensities, ignoring the crucial characteristic of response distribution consistency. This makes it extremely sensitive to accidental decision instability or noise on certain samples, easily leading to misjudgments and misidentifying accidental, non-generalizable model behavior as intentional backdoor implantation. To overcome these shortcomings, this preferred embodiment proposes a novel consistency-weighted scoring mechanism based on response statistical characteristics. This mechanism no longer relies on simple numerical accumulation but treats the response of each perturbation on different samples as a statistical distribution. By quantifying the central tendency and dispersion of this distribution, it more accurately assesses the likelihood of a backdoor trigger. Therefore, in the technical solution of this application, the response matrix is ​​further subjected to cross-class response spectrum consistency aggregation to obtain a suspicion score vector, thereby systematically examining and accumulating the response intensities triggered by each perturbation on all non-target categories, all pointing towards the same potential target category. In this way, the consistency signal generated by the real backdoor trigger can be effectively amplified, while suppressing and canceling random, category-related noise. Ultimately, the complex response data is refined into a suspiciousness score vector that can clearly quantify the degree of suspicion of each category as a backdoor target, providing a direct and robust basis for the final judgment.

[0037] Figure 4 This is a flowchart illustrating the process of performing cross-class response spectrum consistency aggregation on the response matrix to obtain a suspiciousness score vector according to the AI-based backdoor attack detection method of this application. Figure 4 As shown, step S500 includes: S510, extracting class-conditional response vectors from the response matrix based on the sample label vectors to obtain a set of response vectors; S520, parameterizing the response vector set with statistical features to obtain a set of statistical features; S530, performing a consistency-weighted suspicion score on the set of statistical features to obtain the suspicion score vector.

[0038] Accordingly, in step S510, class-conditional response vectors are extracted from the response matrix based on the sample label vectors to obtain a set of response vectors. It should be understood that because the complex response matrix contains all responses from all samples to all perturbations, the signal of a backdoor attack—that is, the consistent impact of a specific perturbation on a specific target category—is submerged in the massive amount of data and difficult to analyze directly. Therefore, it is necessary to construct independent response data vectors from the complex response matrix for each potential attack target category and each combination of candidate perturbations. In the technical solution of this application, class-conditional response vectors are extracted from the response matrix based on the sample label vectors to obtain a set of response vectors, thereby refining the granularity of the analysis to a single perturbation-target pair, thus isolating and evaluating the independent effectiveness of each potential attack path. A well-designed backdoor attack should have a universal triggering effect, that is, it should stably induce a decision shift towards the malicious target on various non-target samples. This step is precisely to effectively aggregate the data that verifies this universality. In this way, the original three-dimensional data structure can be transformed into a series of one-dimensional response sample sets that can be directly used for subsequent statistical analysis, laying the foundation for accurately quantifying the consistency of the response.

[0039] Specifically, in the embodiments of this application, for any candidate target category With any standardized perturbation Traverse the response matrix and extract all source categories that are not... The samples in application Then, the model output is The probability changes are aggregated into a response vector. In this way, the original three-dimensional data structure is transformed into a series of one-dimensional response sample sets that can be directly statistically analyzed. Specifically, in a concrete example of this application, to construct the set of response vectors, each candidate target category will be targeted. Data extraction is performed using combinations of (0 to 9) with each of the eight standardized perturbation primitives. For example, when processing a combination of the candidate target category 1 and the third perturbation primitive (e.g., the white pixel in the lower right corner), the system iterates through the 500×8×10 response matrix and filters based on the sample label vector, retaining only the response data corresponding to samples whose original label is not 1 (450 samples in total). Then, from the matrix slices corresponding to these filtered samples, the change in predicted probability towards 1 after being subjected to the third perturbation is precisely extracted. These 450 scalar values ​​are aggregated to form a 450-dimensional response vector. This process is repeated for all combinations of all 10 candidate target categories and all 8 perturbation primitives, ultimately generating a set of response vectors containing 80 independent response vectors, each of which provides a clean and centralized data source for subsequent statistical feature parameterization.

[0040] Accordingly, in step S520, the response vector set is statistically parameterized to obtain a statistical feature set. It should be understood that although the original response vectors gather all the response data for a specific perturbation-target pair, as high-dimensional data vectors, they contain too much redundant information and cannot intuitively reveal the central trend and dispersion of the response; that is, they cannot directly quantify the two key attributes of a potential trigger: signal strength and signal consistency. Therefore, in the technical solution of this application, the response vector set is further statistically parameterized to obtain a statistical feature set, thereby refining and reducing the dimensionality of each high-dimensional response vector into a pair of key statistical parameters that can quantify its average influence strength and the universality of its effect. In this way, the original, difficult-to-compare response data can be transformed into a statistical feature set with extremely high information density, easily quantifiable and comparable, providing a precise numerical basis for subsequent scoring and judgment.

[0041] Specifically, in the embodiments of this application, for each response vector generated in the previous step... Calculate its sample mean with sample standard deviation The formula for calculating the sample mean is as follows:

[0042]

[0043] in, Is the source category non The total number of samples; It is a response vector The first in One response value, Represents disturbance For the target The average impact strength quantifies the average performance when the disturbance acts as a trigger.

[0044] The formula for calculating the sample standard deviation is:

[0045]

[0046] in, Represents disturbance The degree of dispersion or instability that affects the effect; other symbols are defined as above. The effectiveness of this perturbation as a trigger under different background samples was quantified; the smaller the value, the stronger the universality. This allows high-dimensional response data to be reduced to a form with extremely high information density. Feature pairs transform raw data into a set of statistical features that are easier to analyze and compare.

[0047] Specifically, in a particular example of this application, response vectors are retrieved from the set of response vectors. This vector contains 450 response values, each representing the change in probability of the model outputting the number 1 when the third perturbation primitive is applied to a non-class 1 original sample. First, the sample mean of this vector is calculated. The average value is obtained by summing all 450 response values ​​and then dividing by 450. In the technical scenario, when the perturbation is quantified as a trigger, the average performance or signal strength directed towards target 1 will be predicted. Next, the sample standard deviation of this vector is calculated. This value quantifies the dispersion of these 450 response values ​​around their mean. In a technical scenario, a smaller... The value indicates that the effect of the perturbation is very stable across samples with different backgrounds, meaning it has strong universality; conversely, it indicates that its effect is unstable. These two calculated values ​​( , This constitutes a statistical feature pair. This parameterization process is performed on all 80 response vectors one by one, ultimately generating a statistical feature set containing 80 such statistical feature pairs.

[0048] Accordingly, in step S530, a consistency-weighted suspicion score is applied to the statistical feature set to obtain the suspicion score vector. It should be understood that, due to the independent statistical features... While signal strength and consistency are quantified separately, a single criterion that integrates both is not provided. A response with only high strength but poor consistency (high...) ,high The noise is likely dominated by extreme anomalies in a few samples, while a true backdoor trigger must possess both high average influence strength and high consistency in universality (high...). ,Low Therefore, in the technical solution of this application, a consistency-weighted suspicion score is further applied to the statistical feature set to obtain the suspicion score vector. This constructs a metric that can simultaneously reward high signal strength (high mean) and high signal consistency (low standard deviation), thereby fundamentally solving the misjudgment problem that may arise in the original mechanism due to neglecting the consistency of response distribution. This scoring mechanism draws on the concept of signal-to-noise ratio, where the final effectiveness evaluation of a perturbation is proportional to its average effect and inversely proportional to the instability of its effect. In this way, the effectiveness of each potential attack path can be condensed into a robust quantitative indicator, ultimately forming a suspicion score vector whose score more realistically reflects the intrinsic characteristics of backdoor attacks.

[0049] Specifically, in the embodiments of this application, for each Feature pairs are used to compute a consistent weighted performance score. It can be expressed by the following formula:

[0050]

[0051] in, That is, the weighted performance score; This is a minimal normal number introduced to maintain numerical stability. This means that the final effectiveness evaluation of a perturbation is proportional to its average effect and inversely proportional to the instability of its effect. Subsequently, the final suspicion score for a target category... Defined as the highest efficiency score that can be generated among all candidate perturbations, it is expressed by the following formula:

[0052]

[0053] in, Is the target category The final suspicion score; It is the total number of disturbances. This is the function for finding the maximum value.

[0054] Specifically, in a concrete example of this application, the statistical feature set containing 80 statistical feature pairs generated in the preceding steps is traversed. For one of the feature pairs, for example, the pair corresponding to target category 1 and the third perturbation primitive (… , Substitute this value into the consistency-weighted performance score calculation formula to obtain the performance score. After calculating all 80 weighted efficacy scores, the final score for each target category is determined. Taking target category 1 as an example, all 8 efficacy scores related to that target category are aggregated (…). , ,..., Subsequently, these eight effectiveness scores were compared, and the maximum value was taken as the final suspicion score for target category 1. This process will be performed sequentially for all 10 candidate target categories (0 to 9), ultimately generating a 10-dimensional suspiciousness score vector, where each component of the vector is the final suspiciousness score for the corresponding category.

[0055] Through the aforementioned preferred implementation method, a statistical measure of response consistency is introduced, achieving a technological leap from simple intensity accumulation to a comprehensive evaluation of signal quality. This enhances the robustness of the detection algorithm, enabling it to effectively filter out noise and pseudo-signals caused by individual samples, and accurately identify genuine backdoor attack patterns that simultaneously possess high intensity and high universality. Ultimately, this significantly reduces the false positive rate of backdoor attack detection, improves the ability to detect advanced and covert backdoor attacks, and provides a more reliable and theoretically supported technical guarantee for the security evaluation of artificial intelligence models.

[0056] Specifically, in step S600, backdoor determination and trigger identification are performed based on the suspiciousness score vector to obtain detection results and backdoor information. It should be understood that since the suspiciousness score vector itself is only a set of quantitative indicators, it does not directly provide a clear, binary judgment conclusion, nor does it explicitly indicate the specific attack path. Therefore, in the technical solution of this application, backdoor determination and trigger identification are further performed based on the suspiciousness score vector to obtain detection results and backdoor information, thereby establishing a final decision-making mechanism that transforms the quantified suspiciousness score into a clear conclusion about the existence of a backdoor and backtracks to locate the most effective attack mode. In this way, the entire detection process can be closed, not only confirming the existence of the threat but also providing the key intelligence necessary for analyzing and defending against backdoor attacks.

[0057] More specifically, this process involves threshold comparison and maximum value location of the suspiciousness score vector, and backtracking to find the corresponding attack element based on the maximum value index. In a specific example of this application, the 10-dimensional suspiciousness score vector generated in the preceding steps is first analyzed. Assume this vector is [0.8, 15.7, 0.5, ..., 1.1], and a backdoor detection threshold of 5.0 is preset. The maximum value of 15.7 in the vector is compared with this threshold. Since 15.7 is much greater than 5.0, a backdoor attack is determined to exist in the model to be detected, and a detection result "BackdoorDetected" is generated. Subsequently, the index position of the maximum value 15.7 in the vector is located as 1, which corresponds to the numeric category 1; therefore, the target category of the backdoor attack is identified as 1. To identify the trigger, backtracking is performed to this score. =15.7 is derived from the calculation of the 8 consistency-weighted efficacy scores of target category 1 ( to The maximum value taken when ( ). A search revealed that this maximum value originates from the performance score corresponding to the 3rd perturbation primitive (e.g., the white pixel in the lower right corner). Based on this, the system identifies the third perturbation primitive as a trigger for a backdoor attack. Finally, the system outputs complete backdoor information, including the attack target being 1 and the trigger mode being the white pixel block in the lower right corner.

[0058] In summary, the AI-based backdoor attack detection method according to the embodiments of this application is explained. In black-box or gray-box scenarios where the model's internal parameters are inaccessible, it detects backdoor attacks by analyzing the differences in the model's input-output behavior. Specifically, a comprehensive response matrix is ​​constructed by systematically applying a series of standardized perturbations to clean samples and quantifying the response change of the model's output probability relative to the baseline prediction. Furthermore, considering that the malicious behavior triggered by a genuine backdoor trigger has a unique fingerprint with cross-class consistency—that is, its effect is independent of the original sample's category and stably points to a single malicious target—this solution aims to amplify this universal, trigger-driven deterministic signal by performing cross-class response spectrum consistency aggregation on the response matrix, while suppressing random noise related to specific samples. Finally, through suspiciousness scoring, abnormal response patterns caused by backdoor attacks can be accurately identified, thus efficiently completing the detection task and locating backdoor information without relying on the model's internal information, solving the problem of insufficient detection capability in restricted access scenarios in existing technologies.

[0059] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A backdoor attack detection method based on artificial intelligence, characterized in that, include: Obtain a clean test set; Baseline predictions are performed based on a clean test set to obtain baseline prediction results; Perturbations are added to the baseline prediction results to obtain a perturbation input corpus; The model query and response difference calculation are performed on the perturbation input corpus and baseline prediction results to obtain the response matrix; Cross-class response spectrum consistency aggregation is performed on the response matrix to obtain a suspiciousness score vector, thereby systematically examining and accumulating the response intensity induced by each perturbation in all non-target categories toward the same potential target category; Backdoor detection and trigger identification are performed based on the suspiciousness score vector to obtain detection results and backdoor information; Cross-class response spectrum consistency aggregation is performed on the response matrix to obtain a suspiciousness score vector, including: Based on the sample label vectors, class-conditional response vectors are extracted from the response matrix to obtain a set of response vectors. This refines the granularity of the analysis to individual perturbation-target pairs, thereby isolating and evaluating the independent effectiveness of each potential attack path for any candidate target category. With any standardized perturbation Traverse the response matrix and extract all source categories that are not... The samples in application Then, the model output is The probability changes are aggregated into a response vector. ; Statistical feature parameterization is performed on the response vector set to obtain a statistical feature set. This allows each high-dimensional response vector to be refined and dimensionality reduced into a pair of key statistical parameters that can quantify its average impact strength and the universality of its effect. For each response vector generated in the previous step... Calculate its sample mean with sample standard deviation These two calculated numbers constitute a statistical feature pair. This process is performed on each of the response vectors one by one to generate a set containing multiple statistical feature pairs. The set of statistical characteristics; A consistency-weighted suspiciousness score is applied to the statistical feature set to obtain the suspiciousness score vector. This is used to construct a metric that can simultaneously reward high signal strength and high signal consistency, thereby fundamentally solving the misjudgment problem that may arise in the original mechanism due to neglecting the consistency of response distribution. Feature pairs are used to compute a consistent weighted performance score. It can be expressed by the following formula: in, That is, the weighted performance score; It is a very small positive number introduced to maintain numerical stability; After calculating all weighted effectiveness scores, a final score is determined for each target category, which is the final suspicion score for a target category. Defined as the highest efficiency score that can be generated among all candidate perturbations, it is expressed by the following formula: in, Is the target category The final suspicion score; It is the total number of disturbances. This is a function to find the maximum value. The process will be executed sequentially for all candidate target categories, ultimately generating a suspiciousness score vector. Each component of this vector represents the final suspiciousness score for the corresponding category.

2. The backdoor attack detection method based on artificial intelligence according to claim 1, characterized in that, Baseline predictions are performed based on a clean test set to obtain baseline prediction results, including: Extract a balanced test subset from the clean test set, wherein the number of samples in each class in the balanced test subset is equal; Input each sample tuple from the balanced test subset into the inference interface of the model to be tested to obtain the baseline probability vector corresponding to each sample tuple. The baseline prediction result is obtained by combining each sample tuple with the baseline probability vector corresponding to each sample tuple.

3. The backdoor attack detection method based on artificial intelligence according to claim 1, characterized in that, Perturbations are added to the baseline prediction results to obtain a perturbed input corpus, including: Obtain the disturbance parameter set; The perturbation space is obtained by performing a Cartesian product on each dimension of the perturbation parameter set. Extract a clean sample set from the baseline prediction results; The perturbation space is used to apply perturbation operations to each clean sample in the clean sample set to obtain a perturbation input corpus.

4. The backdoor attack detection method based on artificial intelligence according to claim 3, characterized in that, Each tuple in the perturbation input corpus includes a clean sample, the applied perturbation primitives, and the perturbation-post-sample.

5. The backdoor attack detection method based on artificial intelligence according to claim 1, characterized in that, The model query and response difference calculation is performed on the perturbed input corpus and baseline prediction results to obtain the response matrix, including: The perturbated samples in the perturbation input corpus are input into the inference interface of the model to be detected to obtain the output probability vector. Extract the baseline probability vector corresponding to the perturbated sample from the baseline prediction results; Calculate the response difference vector between the output probability vector and the baseline probability vector, where the response difference vector is a row vector in the response matrix.

6. The backdoor attack detection method based on artificial intelligence according to claim 5, characterized in that, Calculating the response difference vector between the output probability vector and the baseline probability vector includes: calculating the positional difference between the output probability vector and the baseline probability vector to obtain the response difference vector.

Images