Artificial intelligence model confrontation sample defense method

By synchronously acquiring internal state data of the model to calculate consistency indicators and generate repair parameters, the problem of online detection and repair of adversarial examples in artificial intelligence models is solved, realizing real-time defense and parameter adjustment, and is applicable to a variety of deep learning models.

CN121859331AInactive Publication Date: 2026-04-14WUXI UNIV

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-02-02
Publication Date
2026-04-14
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing technologies lack deep integration of online detection and real-time repair capabilities in artificial intelligence models, and independent detection models may have generalization limitations and cannot adjust model parameters in real time during inference.

Method used

By synchronously acquiring intermediate state data of the model's internal computational layer, calculating internal consistency indices, detecting adversarial disturbances in real time, generating repair control parameters, adjusting intermediate states using built-in correction units, and updating and optimizing the model in conjunction with parameter updates.

Benefits of technology

It achieves online detection and repair, can identify and repair adversarial perturbations during model inference, has parameter adjustment capabilities, meets real-time and applicability requirements, and is suitable for a variety of deep learning models.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121859331A_ABST
    Figure CN121859331A_ABST
Patent Text Reader

Abstract

The invention discloses an artificial intelligence model confrontation sample defense method, and relates to the technical field of artificial intelligence security. The method is synchronously executed in a model reasoning process, and an internal consistency index is calculated by synchronously acquiring an intermediate state of an internal association calculation layer and is compared with a preset reference range so as to identify antagonistic disturbance. When disturbance is recognized, a repair control parameter is generated based on the index deviation characteristic, a built-in state correction unit is driven to directionally adjust the intermediate state, and the corrected state is backfilled to the model to complete reasoning. And finally, according to the processing information, carrying out self-adaptive updating on relevant parameters of the model. According to the method, a mechanism for online detection, real-time repair and model parameter adjustment of the adversarial sample is provided, and the robustness of the model in actual deployment can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence security technology, and more specifically, to a method for defending against adversarial examples in artificial intelligence models. Background Technology

[0002] With the widespread application of deep learning technology in critical tasks such as image recognition and natural language processing, the security of artificial intelligence models has become a concern. These models are vulnerable to adversarial attacks, where attackers can induce incorrect predictions by applying subtle perturbations to the input data that are imperceptible to the human eye. This poses a threat to the reliability of AI technology in security-sensitive scenarios.

[0003] Existing defense techniques mainly fall into two categories: adversarial training and runtime detection. Adversarial training introduces adversarial examples during model training to improve robustness, but this method is computationally expensive and its effectiveness may decrease when defending against attack types not seen during training. Runtime detection identifies and filters adversarial examples during inference, but existing methods mostly rely on preset static thresholds or independent detection models, which limits their defense effectiveness and generalization ability, and may affect normal service due to misjudgments.

[0004] For example, Chinese patent document CN119397538A discloses a method for AI model security protection and adversarial verification. This method uploads the model and test samples, and generates adversarial examples based on the test samples for security detection. This approach provides a way to conduct offline security assessments of trained models, focusing on batch evaluation before model deployment, but does not address continuous, online proactive defense during the model's real-time inference process.

[0005] For example, Chinese patent document CN119669439A discloses a method for defending against adversarial example attacks. This method filters input by training a dedicated adversarial example detection model. This approach employs a detection framework based on an independent discriminator. Its defensive effectiveness depends on the training data and generalization ability of the detection model, and the detection model is separate from the target model to be protected. This approach does not describe a mechanism for real-time remediation of adversarial examples that have passed detection.

[0006] In summary, there is a need for a defense solution that can be deeply integrated with the model inference process and has the capabilities of online detection, real-time repair, and parameter adjustment. Therefore, an adversarial example defense method based on artificial intelligence models is proposed to address the above problems. Summary of the Invention

[0007] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide an artificial intelligence model adversarial example defense method, which aims to improve some of the shortcomings of existing adversarial example defense technologies, such as the difficulty of offline detection in dealing with unknown attacks, the potential generalization limitations of independent detection models and their separation from the main model, and the fact that existing methods generally do not have a mechanism to adjust the model itself in real time during inference.

[0008] To achieve the above objectives, the present invention provides the following technical solution: a method for defending against adversarial examples in artificial intelligence models, wherein while the artificial intelligence model performs forward inference calculations on the input data, the following steps are executed sequentially: S1. Synchronously acquire the first intermediate state data of the first computing layer and the second intermediate state data of the second computing layer inside the model. In order to monitor the internal processing logic of the model in real time, when the model performs forward calculation on the input data, synchronously capture the intermediate state of its internal related layers. The second computing layer receives and processes the output of the first computing layer on the forward propagation link of the model. S2. The first intermediate state data and the second intermediate state data are vectorized to obtain the first state vector and the second state vector. An internal consistency index value is calculated based on the first state vector and the second state vector. In order to quantify the logical self-consistency of the current internal processing of the model, an internal consistency index is calculated based on the intermediate state. S3. Compare the internal consistency index value with a predetermined benchmark range, which is defined by a lower threshold and an upper threshold. The benchmark range is obtained based on normal sample statistics and is used to determine whether the input causes an internal logic anomaly in the model. When the internal consistency index value is less than the lower threshold or greater than the upper threshold, it is determined that the input data contains adversarial perturbation. S4. Based on the determination result of step S3, when there is an adversarial disturbance, the correction direction is determined according to whether the internal consistency index value is lower than the lower threshold or higher than the upper threshold, and the correction intensity is determined according to the absolute difference between the internal consistency index value and the exceeded threshold, so as to quantify the severity of the inconsistency; a repair control parameter is generated according to the correction direction and the correction intensity. The repair control parameter includes at least the target layer identifier and the intensity value. This parameter is used to guide subsequent state correction. S5. Based on the intensity value in the repair control parameters, perform mathematical operations to modify the original intermediate state data of the calculation layer indicated by the target layer identifier, generate corrected intermediate state data, and use the repair control parameters to control the state correction unit built into the model to make directional adjustments to the intermediate state in order to eliminate logical discrepancies. S6. During the current forward inference process of the model, the corrected intermediate state data is backfilled into the output position of the computation layer indicated by the target layer identifier, replacing its original intermediate state data, and the computation of subsequent layers continues to be executed to generate the final output result. The corrected intermediate state replaces the corresponding original intermediate state to complete the inference of the input data and output the result. S7. Based on the internal consistency index value, the repair control parameters, and the corrected intermediate state data generated during this inference process, construct an optimization objective function and calculate the gradient of the optimization objective function with respect to a set of preset trainable parameters in the model to perform a parameter update operation and adaptively update the relevant parameters of the model according to the information processed in this process.

[0009] Furthermore, in step S2, the calculation of the internal consistency index value specifically involves: calculating the cosine similarity value between the first state vector and the second state vector, and using the calculation result of the cosine similarity as the internal consistency index to evaluate the logical coherence between the two states.

[0010] Furthermore, in step S3, the lower and upper threshold values ​​of the reference range are predetermined through the following offline calibration process, and the reference index range is established through the following steps: Collect a calibration dataset consisting of multiple verified normal samples to obtain a benchmark of the model's behavior without perturbation; Input each sample in the calibration dataset into the model and execute steps S1 and S2 to obtain the internal consistency index value corresponding to each sample; Calculate the arithmetic mean and standard deviation of the intra-sample consistency index values; The lower threshold is set to the arithmetic mean minus N times the standard deviation, and the upper threshold is set to the arithmetic mean plus N times the standard deviation; where N is a preset constant greater than 0, used to control the tolerance of the normal fluctuation range.

[0011] Furthermore, in step S4, the determination of the target layer identifier follows a preset mapping rule: when the internal consistency index value is lower than the lower threshold, the identifier of the first calculation layer is used as the target layer identifier; when the internal consistency index value is higher than the upper threshold, the identifier of the second calculation layer is used as the target layer identifier. This mapping rule associates the deviation direction with the responsibility calculation layer.

[0012] Furthermore, in step S5, the specific process of modifying the intermediate state data through mathematical operations is as follows: Generate a correction vector with the same dimension as the original intermediate state data of the target layer; The intensity value in the repair control parameters is used as a scaling factor and multiplied by the correction vector to obtain the scaled correction vector; The scaling correction vector is added element-wise to the original intermediate state data of the target layer to obtain the corrected intermediate state data, and the original state is corrected by additive fusion.

[0013] Furthermore, the correction vector is generated by a trainable correction subnetwork built into the model. The parameters of the trainable correction subnetwork are included in the preset set of trainable parameters updated in step S7. The mapping rule for the state correction unit to generate the adjustment vector can be updated according to the changes in the model after multiple historical parameter updates.

[0014] Furthermore, the optimization objective function constructed in step S7 is a linear combination of a first sub-loss term and a second sub-loss term, which constitutes a self-supervised optimization objective. The first sub-loss term represents the deviation between the recalculated internal consistency index value after using the corrected intermediate state data and a desired target value. The desired target value is usually the center value of the benchmark range. This loss term aims to make the internal logic of the repaired model regress to the normal benchmark. The second sub-loss term represents the difference between the output result obtained using the original intermediate state data and the output result obtained using the corrected intermediate state data. This loss term is used to constrain the degree of influence of the repair operation on the final output of the model.

[0015] Furthermore, the optimization objective function is: L_total=α*L1+β*L2, where L1 represents the first sub-loss term, L2 represents the second sub-loss term, α and β are fixed weight coefficients greater than 0 set before model deployment, and the weight coefficients used in the weighted sum are preset positive values.

[0016] Furthermore, steps S1 to S7 constitute a complete defense processing cycle, which is executed synchronously during the model's forward inference of the current input data. The parameter update operation in step S7 is completed at the end of the cycle, and the update result affects the model's processing of subsequent input data in real time. The processing cycle is completed within the time sequence of a single inference of the model, and the parameter update takes effect immediately at the end of the cycle.

[0017] Furthermore, the additional computational load introduced by executing steps S1 to S7, in terms of floating-point operations, does not exceed 20% of the number of floating-point operations required for the model to perform one equivalent forward inference without the defense method. To ensure the real-time performance of the defense method, the additional computational load introduced is limited to a low proportion.

[0018] The technical effects and advantages of this invention are as follows: First, it provides an online detection and repair mechanism. This invention synchronously monitors the intermediate states of the interconnected computational layers within the model and calculates their internal consistency indices, enabling it to identify adversarial disturbances based on the model's own state information. When an anomaly is detected, the method generates repair control parameters based on the direction and magnitude of the index deviation, driving the built-in correction unit to adjust the affected state and then backfilling the corrected state into the computation flow to complete inference. This process is completed within a single inference iteration of the model, achieving continuous model service maintenance.

[0019] Second, a mechanism for online adjustment of model parameters is provided. This invention includes a step for adjusting model parameters. Based on information recorded during a single detection and repair process, an objective function that combines considerations of "consistency recovery" and "output stability" is constructed to adjust relevant parameters in the model with small update increments. This online, incremental adjustment method allows the model to accumulate experience from interactions with adversarial examples.

[0020] Third, this method considers the efficiency and applicability of practical deployment. The entire process of perception, repair, and parameter adjustment is designed to work collaboratively during model inference, and the additional computational overhead introduced is controlled to help meet the requirements of most real-time inference scenarios. Furthermore, this method is not dependent on a specific model architecture, and its core logic is applicable to various deep learning models based on hierarchical structures. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating the overall execution process of the defense method of the present invention.

[0022] Figure 2 A logic structure diagram is generated for the repair control parameters of this invention.

[0023] Figure 3 The flowchart for state correction and adaptive parameter update of the present invention is as follows. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] Example 1 As attached Figures 1 to 3 As shown, this embodiment uses a deep convolutional neural network for image classification as the target model and elaborates on the specific implementation process of the defense method.

[0026] I. Overall Process Overview and Step-by-Step Connection Logic The startup of the closed loop is completely synchronized with the model's forward computation of a single input sample, and its core logic flow is as follows: Step S1 captures the output status of the two preset related computation layers inside the model in real time, providing raw data for analysis.

[0027] Step S2 immediately processes these states and calculates a quantified internal consistency index. This is used to evaluate the logical consistency of the model's current processing.

[0028] Step S3 will The comparison is performed with the pre-calculated normal range [L, U]. This comparison is the core decision point: if... If the result falls within the interval [L, U], the input is considered normal, the loop terminates, and a standard inference result is output; otherwise... Beyond this range (i.e.) or If an adversarial disturbance is detected, the subsequent repair and adaptation process will be immediately triggered.

[0029] Once a disturbance is identified, step S4 immediately begins analysis. The deviation pattern specifically includes its deviation direction (below) or higher ) and deviation range Based on this, repair control parameters with clear guiding significance are generated, the core of which is the target layer identifier. and adjust intensity .

[0030] Step S5 receives these parameters and, through a built-in differentiable correction unit, adjusts the target layer. Apply a force to the original intermediate state by Precisely controlled adjustment vector This results in a logically more consistent corrected state.

[0031] Step S6 is responsible for seamlessly backfilling the corrected state into the computation graph of the model, replacing the original state, so that the model can continue to complete this inference based on the corrected information flow and output the final result.

[0032] S7 performs online learning, utilizing the data generated throughout the entire processing (including raw and corrected status and indicators). (Control parameters, etc.) to construct a loss function that integrates the dual objectives of "consistency recovery" and "output stability". This is used to perform a small gradient update on the few parameters in the model that are most relevant to this anomaly handling.

[0033] The entire S1 to S7 closed loop is completed within a single forward propagation time sequence. Its output is the inference result processed by real-time defense, and the model achieves immediate absorption of current adversarial experience through S7.

[0034] II. Detailed technical descriptions and multiple implementation methods for each step Step S1: Synchronously acquire internal intermediate states During model initialization, two computational layers that have a direct dependency on the data flow are pre-defined. Taking a typical CNN as an example: the first computational layer (denoted as L1) is selected as the last convolutional layer, and its output is a 3D feature map. The dimensions are [C, H, W] (C, H, W represent the number of channels, height, and width, respectively). The second computational layer (denoted as L2) is selected as the first fully connected layer immediately following it, and its output is a one-dimensional feature vector. The dimension is [D] ( (Number of neurons). In implementation, forward hooks are registered for L1 and L2 in the model's forward propagation function. When forward computation reaches these two layers, the hook callback function is automatically triggered, taking the output tensor... and The data is copied and stored in a dedicated buffer as an intermediate state snapshot for this inference.

[0035] As a multi-option implementation, for a deployed inference engine, the output data of a specified layer can be read asynchronously during the model calculation process through the intermediate layer output debugging interface or online performance analysis tools.

[0036] Step S2: Calculate the internal consistency index This step addresses the intermediate state. and Processing is performed to calculate consistency metrics. .

[0037] 1. Vectorization processing: processing the feature map Perform spatial dimension compression to transform it into a vector. The standard implementation is to perform global average pooling: for Each channel Calculate all of them The arithmetic mean of the activation values ​​at each spatial location, i.e. Finally, a [C]-dimensional vector is obtained. For feature vectors directly take it as use.

[0038] 2. Indicator Calculation: Calculate the vector and Cosine similarity as a consistency index The calculation formula is: .in, It represents the dot product of vectors (the sum of the products of corresponding elements). The L2 norm of a vector is represented by the square root of the sum of the squares of its elements. The range of values ​​is The closer the value is The more consistent the semantic direction of the two states, the better.

[0039] As one multi-selection implementation, vectorization can employ global max pooling, where the maximum value across all spatial locations of each channel is taken as the representative value for that channel. In another multi-selection implementation, vectors can be calculated... and Pearson correlation coefficient as an internal consistency indicator This calculation method is mathematically equivalent to centering the two vectors (subtracting their mean) and then calculating the cosine similarity.

[0040] Step S3: Determining the effectiveness of counter-disturbances based on the reference range [L, U] This step will use real-time indicators. The system compares the data with a predefined normal fluctuation range [L, U] to make a binary determination of whether there is an anti-disturbance.

[0041] 1. Baseline Range Establishment (Offline Calibration Phase): Before model deployment, a baseline range is established using a large number of (e.g., ...) benchmarks. The calibration dataset consists of 100 confirmed normal samples. For each sample, steps S1 and S2 are performed to obtain the corresponding set of internal consistency index values. Calculate the arithmetic mean of this set. and standard deviation Set a range coefficient. ( ,For example ), and calculate the lower threshold accordingly. and upper limit threshold Interval center value .

[0042] As a multi-option implementation method, the benchmark range can be based directly on the set of index values ​​without relying on the normal distribution assumption. The empirical quantiles are determined. For example, the lower bound threshold is used. Set as this set quantiles (e.g.) quantiles), upper bound threshold Set as quantiles (e.g.) quantiles).

[0043] 2. Online decision-making logic: During the reasoning process, the current sample's... Then, execute the judgment: if or If the input contains adversarial perturbation, the defense process proceeds to step S4; otherwise, it is determined to be a normal sample, the model directly outputs the result, and the current defense loop terminates.

[0044] Step S4: Generate repair control parameters (target layer) With adjustment intensity ) Once a disturbance is identified, this step analyzes the abnormal pattern and generates specific parameters to guide the repair process.

[0045] 1. Deviation Feature Analysis: Determine the direction of deviation :like If it is a negative deviation; If so, it is a positive deviation.

[0046] Calculate the deviation range : ,in It is the boundary value that has been surpassed. or ). The severity of the anomaly was quantified.

[0047] 2. Determine the target adjustment layer According to the preset mapping rules, the deviation direction will be... Mapped to a specific responsibility calculation layer. As a basic implementation method, the following rules are adopted: When When the deviation is negative, it is considered that the state of the lower feature extraction layer (L1) is abnormal, therefore let ;when When the deviation is positive, it is considered an anomaly in the state of the higher-level semantic integration layer (L2), therefore, it is set to... .

[0048] 3. Calculate the adjustment intensity Adjusting the intensity It is a nonnegative scalar, obtained through a mapping function. From the deviation The calculation shows that, A linear mapping function is as follows: ,in It is a preset positive scaling constant (e.g., 0.05).

[0049] As a multi-option implementation method, the mapping function It can be in a non-linear form, for example ,in The preset maximum strength value, Scaling factor This is the bias parameter. This function can... The value is smoothly limited to Within the interval, prevent due to An excessively large value leads to overcorrection.

[0050] Step S5: The state correction unit performs adjustments. The state correction unit is a lightweight, trainable subnetwork integrated within the model, whose function is to adjust the state based on the parameters. For the target layer The original intermediate state Perform targeted corrections.

[0051] 1. Generate adjustment vector The process is differentiable and includes the following sub-steps: (1) Dimensionality reduction: The latent vectors are obtained by mapping to a low-dimensional latent space through a small trainable projection network (such as a single fully connected layer). .

[0052] (2) Fusion and transformation: merging and transforming latent vectors With adjustment intensity The data is fused (e.g., stitched together), then fed into a multilayer perceptron, and outputs a low-dimensional adjusted basis. .

[0053] (3) Dimensional Upgrading and Shaping: Through another trainable projection network, Upgrade and reshape to obtain the original state. Adjusting the directional base with completely consistent dimensions .

[0054] (4) Normalization and scaling: for the base Perform L2 norm normalization to obtain a unit direction vector, then multiply by the adjusted intensity. This yields the final adjustment vector: This ensures The direction is determined by network learning, and its size is determined by... Precise control.

[0055] 2. Perform correction calculations: Correct the original state by adding elements one by one. .

[0056] As a versatile implementation, when applied to models processing sequential data (such as Transformers), the dimensionality reduction and expansion operations in the state correction unit need to be adapted to the three-dimensional tensor input. In this case, the dimensionality reduction operation can employ one-dimensional convolution or fully connected layers targeting the feature dimension to ensure the generated adjustment vector... It maintains the same three-dimensional structure as the original sequence state.

[0057] Step S6: Inject the correction state and complete the inference. Corrected state Backfill into the model's computational graph for precise replacement The original output of the layer From this point onward, the model's forward computation will begin, and all subsequent layers will be based on this point. The calculations are performed to ultimately produce the inference output after the defense is completed. Meanwhile, to calculate the loss later, a parallel computing branch is needed to preserve and use the original state. Calculated output .

[0058] Step S7: Adaptive Update of Model Parameters This step utilizes all the information generated in this defense loop to perform a one-time, small-scale online optimization of the model.

[0059] 1. Construct the loss function : First, use the corrected state. Re-execute step S2 to calculate the new internal consistency index. .

[0060] Consistency recovery loss :calculate .

[0061] Output stability loss :calculate .

[0062] Total loss: The sum of the two losses mentioned above is calculated by weighting them according to a preset fixed weight. .in, and All are greater than Preset constants (e.g.) , ).

[0063] 2. Perform gradient update: Calculate the total loss Gradients for two sets of key parameters: the first set is for the target layer. The first group consists of the weight parameters most directly related to the current abnormal activation mode; the second group consists of all trainable parameters within the state correction unit (i.e., generated parameters). (Network parameters involved).

[0064] Using the stochastic gradient descent algorithm with a very small learning rate (For example An update operation is performed on these two sets of parameters. This update is incremental and allows the model and correction unit to accumulate effective experience from the processing of the current adversarial examples.

[0065] As a multi-option implementation method, the weight coefficients in the loss function and Based on the current severity of the disturbance Make dynamic adjustments. For example, when When it is large, it can be appropriately increased. The relative value of is more focused on consistency recovery; when When it is small, it can be appropriately increased. The relative value of is more focused on output stability.

[0066] Through the closed loop formed by steps S1 to S7, this invention can monitor the internal health of the model in real time during inference, perform online repairs when adversarial perturbations are detected, and continuously fine-tune the model based on the repair results. All additional operations are lightweight and reasonably implemented, and the introduced computational overhead can be controlled to the same level as the original model's single forward inference computation. This ensures the practicality of the method.

[0067] Those skilled in the art can adapt and optimize the specific implementation of the steps based on the core logic of this invention according to different model architectures and application scenarios, and all such changes fall within the protection scope of this invention.

[0068] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for defending against adversarial examples in artificial intelligence models, characterized in that, While the artificial intelligence model performs forward inference calculations on the input data, the following steps are executed sequentially: S1. Synchronously acquire the first intermediate state data of the first computing layer and the second intermediate state data of the second computing layer inside the model, wherein the second computing layer receives and processes the output of the first computing layer on the forward propagation link of the model; S2. Vectorize the first intermediate state data and the second intermediate state data to obtain the first state vector and the second state vector, and calculate an internal consistency index value based on the first state vector and the second state vector. S3. Compare the internal consistency index value with a predetermined benchmark range, the benchmark range being defined by a lower threshold and an upper threshold; when the internal consistency index value is less than the lower threshold or greater than the upper threshold, it is determined that the input data contains adversarial perturbation. S4. Based on the determination result of step S3, when there is an adversarial disturbance, the correction direction is determined according to whether the internal consistency index value is lower than the lower threshold or higher than the upper threshold, and the correction intensity is determined according to the absolute difference between the internal consistency index value and the threshold exceeded; repair control parameters are generated according to the correction direction and correction intensity, and the repair control parameters include at least the target layer identifier and the intensity value. S5. Based on the intensity value in the repair control parameters, perform mathematical operations to modify the original intermediate state data of the calculation layer indicated by the target layer identifier, and generate corrected intermediate state data. S6. During the current forward inference process of the model, the corrected intermediate state data is backfilled into the output position of the computation layer indicated by the target layer identifier, replacing its original intermediate state data, and the computation of subsequent layers continues to be executed to generate the final output result. S7. Based on the internal consistency index value, the repair control parameters, and the corrected intermediate state data generated during this inference process, construct an optimization objective function and calculate the gradient of the optimization objective function with respect to a set of preset trainable parameters in the model to perform a parameter update operation.

2. The artificial intelligence model adversarial example defense method according to claim 1, characterized in that, In step S2, the calculation of the internal consistency index value specifically involves calculating the cosine similarity value between the first state vector and the second state vector.

3. The artificial intelligence model adversarial example defense method according to claim 1, characterized in that, In step S3, the lower and upper threshold values ​​of the reference range are predetermined through the following offline calibration process: Collect a calibration dataset consisting of multiple verified normal samples; Input each sample in the calibration dataset into the model and execute steps S1 and S2 to obtain the internal consistency index value corresponding to each sample; Calculate the arithmetic mean and standard deviation of the intra-sample consistency index values; The lower threshold is set to the arithmetic mean minus N times the standard deviation, and the upper threshold is set to the arithmetic mean plus N times the standard deviation; where N is a preset constant greater than 0.

4. The artificial intelligence model adversarial example defense method according to claim 1, characterized in that, In step S4, the determination of the target layer identifier follows a preset mapping rule: when the internal consistency index value is lower than the lower threshold, the identifier of the first computing layer is used as the target layer identifier; when the internal consistency index value is higher than the upper threshold, the identifier of the second computing layer is used as the target layer identifier.

5. The method for defending against adversarial examples using artificial intelligence models according to claim 1 or 4, characterized in that, In step S5, the specific process of modifying the intermediate state data through mathematical operations is as follows: Generate a correction vector with the same dimension as the original intermediate state data of the target layer; The intensity value in the repair control parameters is used as a scaling factor and multiplied by the correction vector to obtain the scaled correction vector; The scaling correction vector is added element-wise to the original intermediate state data of the target layer to obtain the corrected intermediate state data.

6. The artificial intelligence model adversarial example defense method according to claim 5, characterized in that, The correction vector is generated by a trainable correction subnetwork built into the model, the parameters of which are included in the preset set of trainable parameters updated in step S7.

7. The method for defending against adversarial examples using artificial intelligence models according to claim 1, characterized in that, The optimization objective function constructed in step S7 is composed of a linear combination of a first sub-loss term and a second sub-loss term; the first sub-loss term represents the deviation between the recalculated internal consistency index value and a desired target value after using the corrected intermediate state data; the second sub-loss term represents the difference between the output result obtained using the original intermediate state data and the output result obtained using the corrected intermediate state data.

8. The method for defending against adversarial examples using artificial intelligence models according to claim 7, characterized in that, The optimization objective function is: L_total=α*L1+β*L2, where L1 represents the first sub-loss term, L2 represents the second sub-loss term, and α and β are fixed weight coefficients greater than 0 set before model deployment.

9. The method for defending against adversarial examples using artificial intelligence models according to claim 1, characterized in that, Steps S1 to S7 constitute a complete defense processing cycle. This cycle is executed synchronously during the model's forward inference of the current input data. The parameter update operation in step S7 is completed at the end of the cycle, and the update result affects the model's processing of subsequent input data in real time.

10. The method for defending against adversarial examples using artificial intelligence models according to claim 1, characterized in that, The additional computational cost introduced by performing steps S1 to S7, in terms of floating-point operations, does not exceed 20% of the number of floating-point operations required for the model to perform one equivalent forward inference without the defense method.

Citation Information

Patent Citations

  • AI model safety protection and confrontation verification method and system

    CN119397538A

  • Method and system for defending against sample attack, and storage medium

    CN119669439A

Cited By

  • AIGC model robustness enhancement method and system based on adversarial sample defense

    CN122065906A