Anti-sample attack detection method based on lightweight artificial intelligence model
Through the end-to-end detection framework based on lightweight artificial intelligence models, using teacher model knowledge to distillate and train student models, generate verification codes and calculate feature similarity, the fragility problem of deep learning models in combating sample attacks is solved, and the problem of fragility of deep learning models is achieved. Effective detection and robustness of data samples.
Patent Information
- Application Number
- CN202410981583.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-22
- Publication Date
- 2024-11-12
AI Technical Summary
Deep learning models are vulnerable when facing adversarial sample attacks, and are difficult to effectively detect and defend. Especially in security-critical applications, it is difficult for the prior art to effectively detect attacks on data samples during transmission.
Adopting an end-to-end detection framework based on lightweight artificial intelligence models, students' models are trained through teacher model knowledge distillation, verification codes are generated and feature similarity is calculated to detect adversarial samples, and using ResNet50 and MobileNetV2 model structures to reduce computational costs and improve robustness sex.
Effective attack detection of data samples such as images, point clouds, and sounds is realized. The detection effect is not affected by the enhancement of the attack of the adversarial sample, but instead becomes stronger as the attack migration is enhanced, improving the robustness of the model.
Smart Images

Figure CN118940797A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an adversarial sample attack detection method, and specifically to an adversarial sample attack detection method based on a lightweight artificial intelligence model. Background Art
[0002] Adversarial examples are input data used to deceive the model in machine learning, especially in deep learning models. These samples do not appear to be significantly different from normal samples to human observers, but can cause the model to make incorrect predictions or classifications. The existence of adversarial examples reveals the vulnerability of deep learning models, especially in safety-critical applications. Adversarial examples were first proposed in a paper by Szegedy et al. in 2013. They found that by making small, almost imperceptible changes to the input image, the neural network could be made to make incorrect judgments. This discovery attracted academic attention to the robustness of deep learning models. In the following years, researchers began to explore this phenomenon in more depth. A variety of methods have emerged to generate adversarial examples, such as Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD). At the same time, some studies have begun to focus on how to defend against adversarial attacks and improve the robustness of models. With the widespread application of deep learning in various industries, the research on adversarial examples has also begun to pay more attention to practical application scenarios, such as autonomous driving, medical diagnosis, and financial security. Security has become an important issue because adversarial examples may be used for malicious purposes. Summary of the invention
[0003] In order to solve the needs and problems in the background technology, the present invention provides an adversarial sample attack detection method based on a lightweight artificial intelligence model to realize attack detection of data samples during transmission.
[0004] The technical solution of the present invention is as follows:
[0005] 1. A method for detecting adversarial sample attacks based on a lightweight artificial intelligence model
[0006] After the data verifier uses the target model to train the lightweight artificial intelligence model, it obtains the verification code generation model and sends it to the data sender;
[0007] The data sender pre-processes the original data sample and then inputs it into the verification code generation model. The model outputs the first verification code. The data sender transmits the original data sample and the first verification code together to the data verifier;
[0008] The data verifier pre-processes the received original data sample and then inputs it into the verification code generation model. The model outputs a second verification code, and calculates the attack detection result of the original data sample based on the first verification code and the second verification code.
[0009] The data verification party uses the target model to train a lightweight artificial intelligence model, specifically:
[0010] The target model is used as the teacher model, and the lightweight artificial intelligence model is used as the student model. The verification code generation model is obtained by training the student model through knowledge distillation of the teacher model.
[0011] The type of the original data sample is point cloud, single frame image, sound or video.
[0012] The data sender preprocesses the original data samples in the same manner as the data verifier preprocesses the received original data samples.
[0013] The first verification code and the second verification code are sample recognition results output by the verification code generation model, including multiple categories of labels and corresponding confidence levels.
[0014] The step of calculating the attack detection result of the original data sample according to the first verification code and the second verification code includes:
[0015] The feature similarity between the first verification code and the second verification code is calculated. If the feature similarity is less than or equal to a similarity threshold, the original data sample has not been attacked during the transmission process; otherwise, the original data sample is an adversarial data sample.
[0016] When the type of the original data sample is a single-frame image, the first verification code and the second verification code both contain multiple labels and corresponding scores. Several labels with the highest scores are taken, the Euclidean distance between the scores corresponding to the same label in the first verification code and the second verification code is calculated, and the Euclidean distances of different labels are summed and squared to obtain the feature similarity of the first verification code and the second verification code.
[0017] 2. A computer device
[0018] The device comprises a memory and a processor, wherein the memory stores a computer program, and the steps of the method are implemented when the processor executes the computer program.
[0019] 3. A computer-readable storage medium
[0020] The medium stores a computer program, which implements the steps of the method when executed by a processor.
[0021] IV. A COMPUTER PROGRAM PRODUCT
[0022] The product comprises a computer program / instructions which, when executed by a processor, implement the steps of the method.
[0023] The beneficial effects of the present invention are as follows:
[0024] 1. Compared with the existing adversarial sample defense methods, the present invention proposes an end-to-end detection framework for detecting adversarial samples by extracting AI model sensitive features in data samples as verification codes. The framework extracts AI sensitive feature information in images as verification codes, which can detect whether the image is attacked by adversarial samples. The detection effect will not be affected by the enhancement of the adversarial sample attack effect, but will become stronger with the enhancement of attack mobility. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 The figure is a flowchart of a method using an image as an example in an embodiment of the present invention.
[0026] Figure 2 A flowchart for comparing AI model features.
[0027] Figure 3 Schematic diagram of the knowledge distillation process of the teacher-student model. DETAILED DESCRIPTION
[0028] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.
[0029] like Figure 1 As shown, the present invention proposes a method for detecting adversarial sample attacks based on a lightweight artificial intelligence model, including:
[0030] After the data verifier uses the target model to be protected to train a lightweight artificial intelligence model, it obtains the verification code generation model and sends it to the data sender;
[0031] Among them, the data verification party uses the target model to train the lightweight artificial intelligence model, specifically:
[0032] The training of the verification code generation model does not require adversarial samples, such as Figure 3 As shown in the figure, the target model is used as the teacher model. In order to reduce the computational cost, the lightweight artificial intelligence model is used as the student model. The verification code generation model is obtained by training the student model through the knowledge distillation of the teacher model. The specific model architecture and parameters can be replaced, and the verification code generation model and the target model maintain a similar decision boundary. The specific label content selection does not have to be the model output, but can also be the intermediate layer feature output, etc.
[0033] The data sender pre-processes the original data sample and then inputs it into the verification code generation model. The model outputs the first verification code. The data sender transmits the original data sample and the first verification code together to the data verifier;
[0034] The data verifier pre-processes the received original data sample and then inputs it into the verification code generation model. The model outputs a second verification code. The attack detection result of the original data sample is calculated based on the first verification code and the second verification code, thereby completing the detection of whether the original data sample has been attacked during the transmission process.
[0035] The first verification code and the second verification code are sample recognition results output by the verification code generation model, including multiple class labels and corresponding confidence levels.
[0036] The attack detection result of the original data sample is calculated according to the first verification code and the second verification code, including:
[0037] The feature similarity between the first verification code and the second verification code is calculated. If the feature similarity is less than or equal to the similarity threshold, the original data sample has not been attacked during the transmission process and is determined to have passed the verification. Otherwise, the original data sample is an adversarial data sample and is abandoned.
[0038] The data sender preprocesses the original data samples in the same way as the data verifier preprocesses the received original data samples.
[0039] The type of raw data samples is point cloud, single frame image, sound or video, not text and numbers.
[0040] When the type of the original data sample is a single-frame image, the preprocessing is to scale the image to a size of 512×512 by bilinear interpolation, and then perform histogram equalization on the image to improve the accuracy of the subsequent feature extraction algorithm. The final image can be recorded as a matrix I^(512×512×3).
[0041] The target model uses ResNet50, and the lightweight AI model uses MobileNetV2. In terms of parameter quantity, the number of parameters of MobileNetV2 is about 3.4 million (3.4M), and the number of parameters of ResNet50 is about 25.6 million (25.6M). By choosing a lightweight model structure, the computational burden can be greatly reduced.
[0042] like Figure 2As shown in the figure, Score1[i] and Score2[i] represent the scores corresponding to the i-th label in the first verification code and the second verification code respectively; j represents the label index that has not been traversed in the second verification code. Both the first verification code and the second verification code contain multiple labels and corresponding scores, satisfying Output={label(1),score(1),label(2),score(2),...,label(N),score(N)}, where Output represents the verification code, label() is the label, score() is the score, and N represents the total number of labels. The 10 labels with the highest scores are taken, and the Euclidean distances between the scores corresponding to the same labels in the first verification code and the second verification code are calculated, and the Euclidean distances of different labels are summed and squared to obtain the feature similarity of the first verification code and the second verification code. If there is no identical label in the first verification code or the second verification code, the score corresponding to the label is directly squared and added to the total distance. The method of calculating the distance comparison threshold instead of comparing whether the labels with the highest scores are the same is chosen in the present invention because a successful adversarial sample attack will definitely change the output of the model, but it will not necessarily cause the ranking of the labels to change. If the above second method is used to identify adversarial samples, some adversarial samples with weaker strength may be ignored. At the same time, because the output of the model has different uses, although the classifier only needs the label with the highest score, some models may require multiple labels with the top scores at the same time, so these adversarial samples that do not change the ranking of the highest-ranking labels also need to be detected. If the attacker knows our defense framework and wants to eliminate the transferability of adversarial samples, that is, to make the adversarial samples only effective for the target model but not for the verification code generation model, then more label content can be introduced when calculating feature similarity, thereby increasing the difficulty and cost of the attacker's attack.
[0043] When the type of the original data sample is point cloud and the target model is a classifier, preprocessing can be done by using filters to remove noise points, isolated points and abnormal points, such as using statistical filtering and radius filtering, and then aligning the point cloud to a unified coordinate system to ensure the consistency of the point cloud coordinates. The subsequent features are the same as the above image classification, using the output of the target model as the feature, and the final feature similarity calculation is also the same.
[0044] When the type of the original data sample is sound and the target model is an audio-to-text model, the preprocessing first unifies the sampling rate of the sound (such as 16khz), then applies the Mel-frequency cepstral coefficient transform to divide the audio per second into 50 frames, and maps each frame to the frequency domain. Since the audio-to-text model generates a corresponding character probability distribution for each frame after inference, the probability distribution set can be used as the feature of the target model, and the final feature similarity calculation can be calculated using the same method as above.
[0045] An embodiment of the present invention also provides a data attack detection device based on a lightweight artificial intelligence model, comprising one or more processors, for implementing an adversarial sample attack detection method based on a lightweight artificial intelligence model in the above embodiment.
[0046] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.
[0047] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0048] An embodiment of the present invention also provides a readable storage medium on which a program is stored. When the program is executed by a processor, an adversarial sample attack detection method based on a lightweight artificial intelligence model in the above embodiment is implemented.
[0049] The readable storage medium may be an internal storage unit of any device with data processing capability described in any of the aforementioned embodiments, such as a hard disk or a memory. The readable storage medium may also be an external storage device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), an SD card, a flash card (Flash Card), etc. equipped on the device. Furthermore, the readable storage medium may also include both an internal storage unit of any device with data processing capability and an external storage device. The readable storage medium is used to store the computer program and other programs and data required by any device with data processing capability, and may also be used to temporarily store data that has been output or is to be output.
[0050] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A method for detecting adversarial sample attacks based on a lightweight artificial intelligence model, characterized in that: include: After the data verifier uses the target model to train the lightweight artificial intelligence model, it obtains the verification code generation model and sends it to the data sender; The data sender pre-processes the original data sample and then inputs it into the verification code generation model. The model outputs the first verification code. The data sender transmits the original data sample and the first verification code together to the data verifier; The data verifier pre-processes the received original data sample and then inputs it into the verification code generation model. The model outputs a second verification code, and calculates the attack detection result of the original data sample based on the first verification code and the second verification code.
2. According to claim 1, a method for detecting adversarial sample attacks based on a lightweight artificial intelligence model is characterized in that: The data verification party uses the target model to train a lightweight artificial intelligence model, specifically: The target model is used as the teacher model, and the lightweight artificial intelligence model is used as the student model. The verification code generation model is obtained by training the student model through knowledge distillation of the teacher model.
3. According to claim 1, a method for detecting adversarial sample attacks based on a lightweight artificial intelligence model is characterized in that: The type of the original data sample is point cloud, single frame image, sound or video.
4. The method for detecting adversarial sample attacks based on a lightweight artificial intelligence model according to claim 1, characterized in that: The data sender preprocesses the original data samples in the same manner as the data verifier preprocesses the received original data samples.
5. The method for detecting adversarial sample attacks based on a lightweight artificial intelligence model according to claim 1, characterized in that: The first verification code and the second verification code are sample recognition results output by the verification code generation model, including multiple categories of labels and corresponding confidence levels.
6. The method for detecting adversarial sample attacks based on a lightweight artificial intelligence model according to claim 1, characterized in that: The step of calculating the attack detection result of the original data sample according to the first verification code and the second verification code includes: The feature similarity between the first verification code and the second verification code is calculated. If the feature similarity is less than or equal to a similarity threshold, the original data sample has not been attacked during the transmission process; otherwise, the original data sample is an adversarial data sample.
7. The method for detecting adversarial sample attacks based on a lightweight artificial intelligence model according to claim 3, characterized in that: When the type of the original data sample is a single-frame image, the first verification code and the second verification code both contain multiple labels and corresponding scores. Several labels with the highest scores are taken, the Euclidean distance between the scores corresponding to the same label in the first verification code and the second verification code is calculated, and the Euclidean distances of different labels are summed and squared to obtain the feature similarity of the first verification code and the second verification code.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.
10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.
Citation Information
Patent Citations
Large character set verification code attack defense method based on adversarial sample
CN116665214A
Model verification sample generation method and device, equipment and storage medium
CN116756553A
Artificial intelligence image recognition anti-attack method and system based on gradient average
CN117079053A
Adversarial sample detection method and device and computer equipment
CN118015308A
Reinforcement learning simulator device for calculating weapon combinations, including oc-gaws(optimal configuration of ground attack weapon system), for each optimal attack strategy of a weapon system and its control method
KR1020250135661A
