Adversarial sample generation method and model evaluation method

Through the adversarial sample generation method of multi-task learning, the optimization of different recognition models is used to generate adversarial perturbations, which solves the problems of poor migration and low efficiency of traditional adversarial attack methods, and improves the success rate and efficiency of adversarial attacks.

CN114662581BActive Publication Date: 2024-12-06TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210248900.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-03-14
Publication Date
2024-12-06
Estimated Expiration
2042-03-14

AI Technical Summary

Technical Problem

Traditional adversarial attack methods have poor migration and low efficiency in deep neural network models, poor migration and black box attacks require massive queries, resulting in low success rate and efficiency of adversarial attacks.

Method used

By using the multi-task learning model, different recognition tasks of the first recognition model and the second recognition model are used to optimize the generation of adversarial perturbations, and the transferability and generation efficiency of adversarial samples are improved.

Benefits of technology

It improves the migration and success rate of the counterattack attacks, avoids massive queries, and improves the efficiency of counterattack attacks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114662581B_ABST
    Figure CN114662581B_ABST
Patent Text Reader

Abstract

The present application relates to an adversarial sample generation method and a model evaluation method, which can be applied to various scenarios such as cloud technology, artificial intelligence, smart transportation, and assisted driving. The adversarial sample generation method includes: identifying the attack sample and the attacked sample respectively through a first recognition model to obtain a first recognition result that matches the attack sample and the attacked sample respectively; identifying the attack sample and the attacked sample respectively through a second recognition model to obtain a second recognition result that matches the attack sample and the attacked sample respectively, and the recognition tasks of the second recognition model and the first recognition model are different; based on the first recognition result and the second recognition result, the first recognition task that matches the first recognition model and the second recognition task that matches the second recognition model are optimized, and the adversarial sample that matches the attack sample is obtained according to the adversarial perturbation generated during the optimization. The use of this method can improve the success rate and efficiency of adversarial attacks.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an adversarial sample generation method and a model evaluation method. Background Art

[0002] With the development of artificial intelligence technology, deep neural network models have emerged. Deep neural network models naturally have vulnerabilities and loopholes. When faced with noise samples generated by some special means, they will output wrong prediction results, resulting in misidentification. In order to defend against such vulnerabilities, we often use adversarial attack methods to actively detect deep neural network models to evaluate the model's vulnerabilities. The success rate returned by the adversarial attack algorithm can be used as an indicator to evaluate the vulnerability of deep neural network models.

[0003] In traditional methods, adversarial attack methods on deep neural network models are mainly divided into the following two types: one is the white-box attack, which attacks by obtaining the internal gradient of the deep neural network model; the other is the black-box attack based on query iteration, which updates the adversarial samples through the query optimization iteration strategy.

[0004] However, with traditional methods, white-box attacks have poor transferability, and the success rate of adversarial attacks on other deep neural network models using trained adversarial samples is very low. Black-box attacks based on query iterations require a large number of queries to obtain adversarial samples, and the efficiency of adversarial attacks is low. Summary of the invention

[0005] Based on this, it is necessary to provide an adversarial sample generation method, device, computer equipment, computer-readable storage medium and computer program product that can improve the success rate and efficiency of adversarial attacks, as well as a model evaluation method, device, computer equipment, computer-readable storage medium and computer program product that can improve the success rate and efficiency of adversarial attacks in response to the above-mentioned technical problems.

[0006] In a first aspect, the present application provides a method for generating adversarial samples. The method for generating adversarial samples comprises:

[0007] The attack sample and the attacked sample are respectively identified by the first identification model to obtain first identification results that match the attack sample and the attacked sample respectively;

[0008] The attack sample and the attacked sample are respectively identified by the second identification model to obtain second identification results that match the attack sample and the attacked sample respectively, and the identification tasks of the second identification model and the first identification model are different;

[0009] Based on the first recognition result and the second recognition result, the first recognition task matching the first recognition model and the second recognition task matching the second recognition model are optimized, and an adversarial sample matching the attack sample is obtained according to the adversarial perturbation generated during the optimization.

[0010] In a second aspect, the present application further provides an adversarial sample generation device. The adversarial sample generation device comprises:

[0011] A first recognition module, used to recognize the attack sample and the attacked sample respectively through a first recognition model, and obtain a first recognition result that matches the attack sample and the attacked sample respectively;

[0012] A second recognition module is used to recognize the attack sample and the attacked sample respectively through a second recognition model to obtain a second recognition result that matches the attack sample and the attacked sample respectively, and the recognition task of the second recognition model is different from that of the first recognition model;

[0013] An adversarial sample generation module is used to optimize a first recognition task that matches a first recognition model and a second recognition task that matches a second recognition model based on a first recognition result and a second recognition result, and obtain an adversarial sample that matches an attack sample according to the adversarial perturbation generated during the optimization.

[0014] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0015] The attack sample and the attacked sample are respectively identified by the first identification model to obtain first identification results that match the attack sample and the attacked sample respectively;

[0016] The attack sample and the attacked sample are respectively identified by the second identification model to obtain second identification results that match the attack sample and the attacked sample respectively, and the identification tasks of the second identification model and the first identification model are different;

[0017] Based on the first recognition result and the second recognition result, the first recognition task matching the first recognition model and the second recognition task matching the second recognition model are optimized, and an adversarial sample matching the attack sample is obtained according to the adversarial perturbation generated during the optimization.

[0018] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0019] The attack sample and the attacked sample are respectively identified by the first identification model to obtain first identification results that match the attack sample and the attacked sample respectively;

[0020] The attack sample and the attacked sample are respectively identified by the second identification model to obtain second identification results that match the attack sample and the attacked sample respectively, and the identification tasks of the second identification model and the first identification model are different;

[0021] Based on the first recognition result and the second recognition result, the first recognition task matching the first recognition model and the second recognition task matching the second recognition model are optimized, and an adversarial sample matching the attack sample is obtained according to the adversarial perturbation generated during the optimization.

[0022] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0023] The attack sample and the attacked sample are respectively identified by the first identification model to obtain first identification results that match the attack sample and the attacked sample respectively;

[0024] The attack sample and the attacked sample are respectively identified by the second identification model to obtain second identification results that match the attack sample and the attacked sample respectively, and the identification tasks of the second identification model and the first identification model are different;

[0025] Based on the first recognition result and the second recognition result, the first recognition task matching the first recognition model and the second recognition task matching the second recognition model are optimized, and an adversarial sample matching the attack sample is obtained according to the adversarial perturbation generated during the optimization.

[0026] In a sixth aspect, the present application provides a model evaluation method. The model evaluation method comprises:

[0027] Using the to-be-evaluated recognition model to identify the evaluation sample pairs in the acquired evaluation sample set, and obtain the matching recognition results of the evaluation sample pairs, wherein the evaluation sample pairs include adversarial samples and adversarial samples, and the adversarial samples are generated by the above-mentioned adversarial sample generation method;

[0028] Based on the recognition result of the adversarial sample and the recognition result of the adversarial sample in the recognition result, determine the similarity of the recognition result matching the evaluation sample pair;

[0029] When the similarity of the recognition result is greater than the similarity threshold, the attack result matching the evaluation sample pair is determined as a successful attack;

[0030] Based on the attack results that match the evaluation sample pairs in the evaluation sample set, the attack success rate is calculated;

[0031] According to the attack success rate, the model evaluation results are obtained.

[0032] In a seventh aspect, the present application further provides a model evaluation device. The model evaluation device comprises:

[0033] A model recognition module is used to recognize evaluation sample pairs in the acquired evaluation sample set through the recognition model to be evaluated, and obtain recognition results matching the evaluation sample pairs, wherein the evaluation sample pairs include adversarial samples and adversarial samples, and the adversarial samples are generated by the above-mentioned adversarial sample generation method;

[0034] A similarity determination module is used to determine the similarity of the recognition result matching the evaluation sample pair based on the adversarial sample recognition result and the adversarial sample recognition result in the recognition result;

[0035] An attack analysis module, used for determining an attack result matching the evaluation sample pair as a successful attack when the similarity of the recognition result is greater than a similarity threshold;

[0036] A statistical module, used to calculate the attack success rate based on the attack results that match the evaluation sample pairs in the evaluation sample set;

[0037] The processing module is used to obtain the model evaluation result according to the attack success rate.

[0038] In an eighth aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0039] Using the recognition model to be evaluated, the evaluation sample pairs in the acquired evaluation sample set are identified to obtain identification results matching the evaluation sample pairs, where the evaluation sample pairs include adversarial samples and adversarial samples, and the adversarial samples are generated by the adversarial sample generation method described above;

[0040] Based on the recognition result of the adversarial sample and the recognition result of the adversarial sample in the recognition result, determine the similarity of the recognition result matching the evaluation sample pair;

[0041] When the similarity of the recognition result is greater than the similarity threshold, the attack result matching the evaluation sample pair is determined as a successful attack;

[0042] Based on the attack results that match the evaluation sample pairs in the evaluation sample set, the attack success rate is calculated;

[0043] According to the attack success rate, the model evaluation results are obtained.

[0044] In a ninth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0045] Using the to-be-evaluated recognition model to identify the evaluation sample pairs in the acquired evaluation sample set, and obtain the matching recognition results of the evaluation sample pairs, wherein the evaluation sample pairs include adversarial samples and adversarial samples, and the adversarial samples are generated by the above-mentioned adversarial sample generation method;

[0046] Based on the recognition result of the adversarial sample and the recognition result of the adversarial sample in the recognition result, determine the similarity of the recognition result matching the evaluation sample pair;

[0047] When the similarity of the recognition result is greater than the similarity threshold, the attack result matching the evaluation sample pair is determined as a successful attack;

[0048] Based on the attack results that match the evaluation sample pairs in the evaluation sample set, the attack success rate is calculated;

[0049] According to the attack success rate, the model evaluation results are obtained.

[0050] In a tenth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0051] Using the to-be-evaluated recognition model to identify the evaluation sample pairs in the acquired evaluation sample set, and obtain the matching recognition results of the evaluation sample pairs, wherein the evaluation sample pairs include adversarial samples and adversarial samples, and the adversarial samples are generated by the above-mentioned adversarial sample generation method;

[0052] Based on the recognition result of the adversarial sample and the recognition result of the adversarial sample in the recognition result, determine the similarity of the recognition result matching the evaluation sample pair;

[0053] When the similarity of the recognition result is greater than the similarity threshold, the attack result matching the evaluation sample pair is determined as a successful attack;

[0054] Based on the attack results that match the evaluation sample pairs in the evaluation sample set, the attack success rate is calculated;

[0055] According to the attack success rate, the model evaluation results are obtained.

[0056] The above-mentioned adversarial sample generation method, device, computer equipment, storage medium and computer program product respectively identify the attack sample and the attacked sample through the first recognition model to obtain a first recognition result that matches the attack sample and the attacked sample respectively, and respectively identify the attack sample and the attacked sample through the second recognition model with a recognition task different from that of the first recognition model to obtain a second recognition result that matches the attack sample and the attacked sample respectively. Based on the first recognition result and the second recognition result, the first recognition task matching the first recognition model and the second recognition task matching the second recognition model are optimized, and the adversarial sample matching the attack sample is obtained according to the adversarial perturbation generated during the optimization. The multi-task learning mode can be adopted in the adversarial attack to improve the transferability of the adversarial attack and obtain adversarial samples with strong transferability, and then the adversarial samples with strong transferability can be used to improve the success rate of the adversarial attack. At the same time, in the process of obtaining the adversarial sample, only optimization needs to be performed without massive query, which can improve the efficiency of generating adversarial samples and thus improve the efficiency of adversarial attacks.

[0057] The above-mentioned model evaluation method, device, computer equipment, storage medium and computer program product use the recognition model to be evaluated to identify the evaluation sample pairs in the obtained evaluation sample set, and obtain the recognition results that match the evaluation sample pairs. The evaluation sample pairs include adversarial samples and adversarial samples. Based on the adversarial sample recognition results and the adversarial sample recognition results in the recognition results, the similarity of the recognition results matching the evaluation sample pairs is determined. When the similarity of the recognition results is greater than the similarity threshold, the attack results matching the evaluation sample pairs are determined to be successful attacks. Based on the attack results matching the evaluation sample pairs in the evaluation sample set, the attack success rate is calculated, and the model evaluation results are obtained according to the attack success rate. Since adversarial samples with high mobility and high generation efficiency are used in the model evaluation process, the success rate and efficiency of adversarial attacks can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 A diagram of an application environment of an adversarial sample generation method in one embodiment;

[0059] Figure 2 Schematic diagram of a process of generating adversarial samples in one embodiment;

[0060] Figure 3 A schematic diagram of a flow chart of a model evaluation method in one embodiment;

[0061] Figure 4 is a flowchart of a method for generating adversarial samples in another embodiment;

[0062] Figure 5 Schematic diagram of a process of generating adversarial samples in another embodiment;

[0063] Figure 6 is a structural block diagram of an adversarial sample generating device in one embodiment;

[0064] Figure 7 is a structural block diagram of a model evaluation device in one embodiment;

[0065] Figure 8 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0066] The present application relates to the field of artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that the machines have the functions of perception, reasoning and decision-making.

[0067] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0068] Among them, the main focus of this application is on machine learning. Machine Learning (ML) is a multi-disciplinary interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithmic complexity theory and other disciplines. It specializes in how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include technologies such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning by teaching.

[0069] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0070] The adversarial sample generation method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown in Figure 1 In the application environment shown, the adversarial sample generation method is illustrated by the interactive implementation of the terminal and the server. It can be understood that the method can also be applied to the server and the terminal. Among them, the terminal 102 recognizes the attack sample and the attacked sample respectively through the first recognition model, and obtains the first recognition result matching the attack sample and the attacked sample respectively. The attack sample and the attacked sample are recognized by the second recognition model respectively, and the second recognition result matching the attack sample and the attacked sample is obtained. The recognition task of the second recognition model is different from that of the first recognition model. The first recognition result and the second recognition result are sent to the server 104. The server 104 optimizes the first recognition task matching the first recognition model and the second recognition task matching the second recognition model based on the first recognition result and the second recognition result, and obtains the adversarial sample matching the attack sample according to the adversarial perturbation generated during the optimization. Among them, the terminal 102 can be, but is not limited to, various desktop computers, laptops, smart phones, tablet computers, Internet of Things devices, portable wearable devices and aircraft, etc. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. Server 104 can be implemented as an independent server or a server cluster consisting of multiple servers, or can be a node on the blockchain.

[0071] In one embodiment, Figure 2 As shown, a method for generating adversarial samples is provided, and the adversarial sample generation method is applied to Figure 1 The terminal in is used as an example to illustrate, including the following steps:

[0072] Step 202: Identify the attack sample and the attacked sample respectively using a first identification model to obtain first identification results that match the attack sample and the attacked sample respectively.

[0073] Among them, the attack sample refers to the unoptimized initial adversarial sample. The adversarial sample refers to the input sample formed by deliberately adding subtle interference to the original sample, which will cause the model to give an erroneous output with high confidence. The attacked sample is a concept opposite to the attack sample, that is, the original sample without subtle perturbation data added. The attack sample and the attacked sample can be the same type of data. For example, in image recognition, the attacked sample can be the original image without subtle perturbation, and the attack sample can be the image with subtle perturbation added to the attacked sample.

[0074] Among them, the first recognition model refers to a model used to identify the attack sample and the attacked sample. In this embodiment, the first recognition model is not specifically limited, as long as it can achieve sample recognition. For example, the first recognition model can be a face recognition model that converts input data into a vector representation. The first recognition result refers to the result output by the first recognition model after identifying the attack sample and the attacked sample respectively. For example, the first recognition result can be a feature vector that characterizes the attack sample and the attacked sample.

[0075] Specifically, when generating adversarial samples, the terminal obtains attack samples and attacked samples, and respectively identifies the attack samples and attacked samples through the first recognition model to obtain a first recognition result matching the attack sample and a first recognition result matching the attacked sample. The first recognition model may be obtained in real time during recognition or may be obtained in advance.

[0076] Step 204: The attack sample and the attacked sample are respectively identified by a second identification model to obtain second identification results that match the attack sample and the attacked sample respectively. The second identification model and the first identification model have different identification tasks.

[0077] Among them, the second recognition model refers to a model used to identify attack samples and attacked samples and has a different recognition task from the first recognition model. In this embodiment, the second recognition model is not specifically limited, as long as it can realize sample recognition and has a different recognition task from the first recognition model. For example, when the first recognition model is a face recognition model, the second recognition model can specifically be a face attribute recognition model that has a different recognition task from the face recognition model. The face attributes here can specifically be gender, age, expression, etc.

[0078] Specifically, while the attack sample and the attacked sample are respectively identified by the first identification model, the terminal will respectively identify the attack sample and the attacked sample by the second identification model to obtain a second identification result matching the attack sample and a second identification result matching the attacked sample. The second identification model may be obtained in real time during identification or may be obtained in advance.

[0079] Furthermore, when the attack sample and the attacked sample are input into the first recognition model and the second recognition model respectively, the terminal first extracts features of the attack sample and the attacked sample, and then inputs the extracted features matching the attack sample and the features matching the attacked sample into the first recognition model and the second recognition model respectively. When extracting features from the attack sample and the attacked sample, the feature extraction network used is the same.

[0080] Step 206, based on the first recognition result and the second recognition result, optimize the first recognition task matching the first recognition model and the second recognition task matching the second recognition model, and obtain an adversarial sample matching the attack sample according to the adversarial perturbation generated during the optimization.

[0081] Among them, adversarial perturbations refer to some tiny perturbations added to the sample that can change the prediction results of the model. In this embodiment, adversarial perturbations mainly refer to some tiny perturbations added to the attack sample that can change the prediction results of the model, and can optimize the attack sample. The adversarial sample that matches the attack sample refers to an adversarial sample with strong transferability obtained by optimizing the attack sample through the adversarial perturbation generated during optimization.

[0082] Specifically, the terminal obtains a first loss of the first recognition model based on the first recognition result, and obtains a second loss of the second recognition model based on the second recognition result, and then optimizes and iterates the first recognition task matching the first recognition model and the second recognition task matching the second recognition model based on the first loss and the second loss, and obtains an adversarial sample matching the attack sample according to the adversarial perturbation generated in each iteration during the optimization iterative process.

[0083] Further, when optimizing and iterating the first recognition task and the second recognition task based on the first loss and the second loss, the terminal will first optimize the first recognition task and the second recognition task based on the first loss and the second loss in turn to obtain the first iterative adversarial perturbation, update the attack sample according to the first iterative adversarial perturbation, obtain the first iterative attack sample, and then continue to optimize and iterate the first recognition task and the second recognition task based on the first iterative attack sample and the attacked sample until the number of iterations of the optimization iteration reaches the iteration number threshold, and update the latest generated iterative attack sample according to the latest generated adversarial perturbation to obtain an adversarial sample matching the attack sample. The iteration number threshold can be set as needed.

[0084] Furthermore, when optimizing and iterating the first recognition task and the second recognition task, the optimization order of the recognition tasks in two adjacent iterations is different. For example, when the optimization order of the recognition tasks in the initial iteration is to optimize the first recognition task first and then optimize the second recognition task, the optimization order of the recognition tasks in the second iteration adjacent to the initial iteration may be to optimize the second recognition task first and then optimize the first recognition task.

[0085] The above-mentioned adversarial sample generation method uses a first recognition model to respectively identify the attack sample and the attacked sample to obtain a first recognition result that matches the attack sample and the attacked sample respectively, and uses a second recognition model with a recognition task different from that of the first recognition model to respectively identify the attack sample and the attacked sample to obtain a second recognition result that matches the attack sample and the attacked sample respectively. Based on the first recognition result and the second recognition result, the first recognition task that matches the first recognition model and the second recognition task that matches the second recognition model are optimized, and the adversarial sample that matches the attack sample is obtained according to the adversarial perturbation generated during the optimization. The multi-task learning mode can be adopted in the adversarial attack to improve the transferability of the adversarial attack and obtain adversarial samples with strong transferability, and then the adversarial samples with strong transferability can be used to improve the success rate of the adversarial attack. At the same time, in the process of obtaining the adversarial sample, only optimization needs to be performed without massive query, which can improve the efficiency of generating adversarial samples and thus improve the efficiency of adversarial attacks.

[0086] In one embodiment, based on the first recognition result and the second recognition result, the first recognition task matching the first recognition model and the second recognition task matching the second recognition model are optimized, and the adversarial sample matching the attack sample is obtained according to the adversarial perturbation generated by the optimization, including:

[0087] Obtaining a first loss of a first recognition model based on the first recognition result, and obtaining a second loss of a second recognition model based on the second recognition result;

[0088] Based on the first loss and the second loss, the first recognition task matching the first recognition model and the second recognition task matching the second recognition model are optimized and iterated, and an adversarial sample matching the attack sample is obtained according to the adversarial perturbation generated during the optimization iteration.

[0089] The first loss refers to the inconsistency between the attack sample and the attacked sample that match the first recognition model. The second loss refers to the inconsistency between the attack sample and the attacked sample that match the second recognition model. Optimizing and iterating the first recognition task and the second recognition task refers to generating adversarial perturbations for the attack sample based on the first loss and the second loss, optimizing the attack sample using the adversarial perturbations, and obtaining adversarial samples that match the attack sample.

[0090] Specifically, the terminal obtains the first loss of the first recognition model based on the intermediate layer recognition features and the first recognition result generated by the first recognition model when recognizing the attack sample and the attacked sample, and obtains the second loss of the second recognition model based on the intermediate layer recognition features and the second recognition result generated by the second recognition model when recognizing the attack sample and the attacked sample. The first recognition model and the second recognition model both include several layers of networks, and the intermediate layer recognition features refer to the features output by the intermediate layer networks of the first recognition model and the second recognition model, which are different from the output of the last layer network, and the output of the last layer network is the first recognition result and the second recognition result.

[0091] Specifically, after obtaining the first loss and the second loss, the terminal will first optimize the first recognition task and the second recognition task in turn based on the first loss and the second loss to obtain the first iterative adversarial perturbation, update the attack sample according to the first iterative adversarial perturbation, obtain the first iterative attack sample, and then continue to optimize and iterate the first recognition task and the second recognition task based on the first iterative attack sample and the attacked sample until the number of iterations of the optimization iteration reaches the iteration number threshold, and update the latest generated iterative attack sample according to the latest generated adversarial perturbation to obtain an adversarial sample that matches the attack sample. The iteration number threshold can be set as needed.

[0092] Among them, based on the first loss and the second loss, the first recognition task and the second recognition task are optimized in turn to obtain the initial iterative adversarial perturbation, which means that the first recognition task is first optimized based on the first loss to generate an intermediate adversarial perturbation, and then the second recognition task is optimized based on the intermediate adversarial perturbation and the second loss to obtain the initial iterative adversarial perturbation.

[0093] In this embodiment, by obtaining the first loss based on the first recognition result and obtaining the second loss based on the second recognition result, the first recognition task and the second recognition task can be optimized and iterated based on the first loss and the second loss, so that an adversarial sample matching the attack sample can be obtained according to the adversarial perturbation generated during the optimization iteration.

[0094] In one embodiment, obtaining a first loss of a first recognition model based on the first recognition result includes:

[0095] Based on the intermediate layer recognition features generated by the first recognition model when recognizing the attacking sample and the attacked sample, a feature loss of the first recognition model is obtained;

[0096] Based on the first recognition result, calculating the recognition result loss of the first recognition model;

[0097] According to the feature loss and the recognition result loss, the first loss of the first recognition model is obtained.

[0098] Among them, feature loss refers to the inconsistency between the attack sample and the attacked sample determined based on the intermediate layer recognition feature. For example, feature loss can specifically refer to the loss calculated using the Euclidean distance. Recognition result loss refers to the inconsistency between the attack sample and the attacked sample determined based on the first recognition result. For example, recognition result loss can specifically refer to the loss calculated using the cosine distance.

[0099] Specifically, when the first recognition model identifies the attack sample and the attacked sample, it will generate an intermediate layer recognition feature that matches the attack sample and the attacked sample respectively. When calculating the first loss, the terminal will calculate the feature difference between the intermediate layer recognition features that match the attack sample and the attacked sample respectively, and obtain the feature loss of the first recognition model based on the feature difference. Then, the recognition result similarity between the first recognition results that match the attack sample and the attacked sample respectively is calculated, and the recognition result loss of the first recognition model is calculated according to the recognition result similarity. Finally, the feature loss and the recognition result loss are weighted to obtain the first loss of the first recognition model.

[0100] Among them, the intermediate layer identification features that match the attack sample and the attacked sample can be specifically the feature matrix that characterizes the attack sample and the attacked sample. Then, the feature difference between the intermediate layer identification features can be calculated by matrix subtraction. After the feature difference is obtained, the second norm of the feature difference (commonly used to calculate the vector length) can be used as the feature loss. For example, the calculation formula of the feature loss can be: in, To identify the middle layer features that match the attack samples, Identify features of the middle layer that match the attacked sample.

[0101] Among them, the first recognition results that match the attack sample and the attacked sample can be specifically feature vectors that characterize the attack sample and the attacked sample, and the recognition result similarity between the first recognition results can be calculated by cosine distance. After obtaining the recognition result similarity, the similarity difference between the recognition result similarity distance and the similarity threshold can be used as the recognition result loss, wherein the similarity threshold can be set as needed. For example, the calculation formula for the recognition result loss can be: in, is the first recognition result that matches the attack sample, is the first recognition result that matches the attacked sample, and 1 is the similarity threshold.

[0102] Among them, the weighting coefficients in weighted feature loss and recognition result loss can be set as needed. For example, the calculation formula of the first loss can be: Among them, β1 and β2 are weighting coefficients, To identify the resulting loss, is the feature loss.

[0103] It should be noted that the method of obtaining the second loss of the second recognition model based on the second recognition result is similar to the method of obtaining the first loss of the first recognition model based on the first recognition result, and this application will not describe it again in the embodiments.

[0104] In this embodiment, by obtaining feature loss based on the intermediate layer recognition features and calculating recognition result loss based on the first recognition result, the model loss can be accurately calculated by combining the feature loss and the recognition result loss to obtain the first loss of the first recognition model.

[0105] In one embodiment, based on the first loss and the second loss, optimizing and iterating the first recognition task matching the first recognition model and the second recognition task matching the second recognition model, and obtaining the adversarial sample matching the attack sample according to the adversarial perturbation generated during the optimization iteration includes:

[0106] Based on the first loss and the second loss, optimizing and iterating a first recognition task matched with the first recognition model and a second recognition task matched with the second recognition model;

[0107] When the number of optimization iterations reaches the iteration threshold, an adversarial sample matching the attack sample is obtained based on the latest generated adversarial perturbation;

[0108] Among them, the optimization order of the recognition task in two adjacent iterations in the optimization iteration is different.

[0109] Specifically, the terminal will first optimize the first recognition task and the second recognition task in turn based on the first loss and the second loss to obtain the first iterative adversarial perturbation, update the attack sample according to the first iterative adversarial perturbation, obtain the first iterative attack sample, and then continue to optimize the first recognition task and the second recognition task based on the first iterative attack sample and the attacked sample until the number of iterations of the optimization iteration reaches the iteration number threshold, and update the latest generated iterative attack sample according to the latest generated adversarial perturbation to obtain an adversarial sample that matches the attack sample. The iteration number threshold can be set as needed.

[0110] Furthermore, when optimizing and iterating the first recognition task and the second recognition task, the optimization order of the recognition tasks in two adjacent iterations is different. By setting different optimization orders for alternating optimization during the optimization iteration process, the stability of the optimization can be improved.

[0111] In one embodiment, based on the first loss and the second loss, iteratively optimizing the first recognition task matched with the first recognition model and the second recognition task matched with the second recognition model includes:

[0112] Based on the first loss and the second loss, the first recognition task and the second recognition task are optimized in turn to obtain the first iterative adversarial perturbation;

[0113] Update the attack sample according to the first iterative adversarial perturbation to obtain the first iterative attack sample;

[0114] Based on the initial iterative attack samples and attacked samples, the first recognition task and the second recognition task are optimized and iterated.

[0115] Specifically, the terminal will optimize the first recognition task and the second recognition task in turn based on the first loss and the second loss to obtain the first iterative adversarial disturbance, and then superimpose the first iterative adversarial disturbance on the attack sample to optimize the attack sample and obtain the first iterative attack sample. Finally, based on the first iterative attack sample and the attacked sample, the terminal will continue to optimize and iterate the first recognition task and the second recognition task until the number of optimization iterations reaches the iteration number threshold, and then stop the optimization iteration.

[0116] Among them, sequentially optimizing the first recognition task and the second recognition task means first optimizing one of the recognition tasks based on the matching loss to generate an intermediate adversarial perturbation, and then optimizing the other recognition task based on the intermediate adversarial perturbation and the matching loss. For example, the first recognition task can be optimized based on the first loss to generate an intermediate adversarial perturbation, and then the second recognition task can be optimized based on the intermediate adversarial perturbation and the second loss.

[0117] Among them, continuing to optimize and iterate the first recognition task and the second recognition task based on the initial iterative attack sample and the attacked sample means that the initial iterative attack sample and the attacked sample are respectively identified by the first recognition model matched with the first recognition task to obtain the first recognition result matching the initial iterative attack sample and the attacked sample respectively, and the initial iterative attack sample and the attacked sample are respectively identified by the second recognition model matched with the second recognition task to obtain the second recognition result matching the initial iterative sample and the attacked sample respectively, the first recognition task and the second recognition task are optimized based on the first recognition result and the second recognition result to generate the secondary iterative adversarial perturbation, the initial iterative attack sample is updated according to the secondary iterative adversarial perturbation to obtain the secondary iterative attack sample, and then based on the secondary iterative attack sample and the attacked sample, the first recognition task and the second recognition task are optimized and iterated until the number of iterations of the optimization iteration reaches the iteration number threshold, and the optimization iteration is stopped.

[0118] It should be noted that in the optimization iteration process, each iteration optimizes the first recognition task and the second recognition task based on the latest generated iterative attack samples and attacked samples to generate the adversarial perturbation of this iteration, and uses the adversarial perturbation of this iteration to update the latest generated iterative attack samples to generate new iterative attack samples for continued optimization iteration.

[0119] In this embodiment, the first recognition task and the second recognition task are optimized in turn based on the first loss and the second loss to obtain the initial iterative adversarial disturbance, and then the attack sample is updated according to the initial iterative adversarial disturbance to obtain the initial iterative attack sample, and finally the first recognition task and the second recognition task are optimized and iterated based on the initial iterative attack sample and the attacked sample. The optimization stability can be improved through optimization iteration and a good optimization effect can be obtained.

[0120] In one embodiment, based on the first loss and the second loss, the first recognition task and the second recognition task are optimized in sequence to obtain the first iterative adversarial perturbation, including:

[0121] The first recognition task is optimized based on the first loss to obtain an intermediate adversarial perturbation;

[0122] Based on the intermediate adversarial perturbation and the second loss, the second recognition task is optimized to obtain the first iterative adversarial perturbation.

[0123] Specifically, the terminal will perform a gradient solution on the attack sample based on the first loss to optimize the first recognition task and obtain the intermediate adversarial perturbation, and then perform a gradient solution on the intermediate adversarial perturbation based on the second loss to optimize the second recognition task and obtain the first iterative adversarial perturbation.

[0124] For example, the specific calculation method is shown in the following formula:

[0125]

[0126]

[0127] Among them, γ1 and γ2 are hyperparameters and can be set as needed. represents the gradient solution, ∈ a Represents an attack sample, ∈` a represents the intermediate adversarial perturbation, ∈`` a represents the first iterative adversarial perturbation, represents the first loss, Indicates the second loss.

[0128] In this embodiment, the first recognition task is first optimized based on the first loss to obtain the intermediate adversarial perturbation, and then the second recognition task is optimized based on the intermediate adversarial perturbation and the second loss to obtain the initial iterative adversarial perturbation. The idea of ​​meta-learning can be used to use the optimization result of one recognition task as the optimization basis for another recognition task, thereby improving the stability of the optimization and achieving good optimization effects.

[0129] In one embodiment, Figure 3 As shown, a model evaluation method is provided, and the model evaluation method is applied to Figure 1 The terminal in is used as an example to illustrate, including the following steps:

[0130] Step 302, using the to-be-evaluated recognition model to identify the evaluation sample pairs in the acquired evaluation sample set, and obtain matching recognition results of the evaluation sample pairs, wherein the evaluation sample pairs include adversarial samples and adversarial samples, and the adversarial samples are generated by the above-mentioned adversarial sample generation method.

[0131] Among them, the recognition model to be evaluated refers to a model with the same recognition task as the first recognition model and a different model structure. For example, when the first recognition model is a face recognition model, the recognition model to be evaluated may specifically refer to a face recognition model with a different model structure from the first recognition model. The evaluation sample set refers to a set of evaluation sample pairs, and the evaluation sample pairs include adversarial samples and adversarial samples. The recognition results that match the evaluation sample pairs include adversarial sample recognition results that match the adversarial samples and adversarial sample recognition results that match the adversarial samples, and the recognition results here refer to feature representations that characterize the adversarial samples and the adversarial samples. For example, the recognition results may specifically be feature vectors that characterize the adversarial samples and the adversarial samples.

[0132] Specifically, during model evaluation, for each evaluation sample pair in the acquired evaluation sample set, the terminal will identify the adversarial sample and the adversarial sample in the evaluation sample pair through the model to be evaluated to obtain the adversarial sample identification result and the adversarial sample identification result.

[0133] Step 304: Determine the similarity of the recognition result that matches the evaluation sample pair based on the recognition result of the adversarial sample and the recognition result of the adversarial sample in the recognition result.

[0134] The recognition result similarity refers to the degree of similarity between the recognition result of the adversarial sample and the recognition result of the adversarial sample.

[0135] Specifically, the terminal calculates the similarity between the adversarial sample recognition result and the adversarial sample recognition result in the recognition result, and uses the similarity as the recognition result similarity that matches the evaluation sample pair. Furthermore, the similarity between the adversarial sample recognition result and the adversarial sample recognition result can be specifically cosine similarity, and the cosine similarity is calculated as follows: Where T represents the recognition model to be evaluated, Represents the recognition result of the adversarial sample, Represents the adversarial sample recognition result.

[0136] Step 306: When the similarity of the recognition result is greater than the similarity threshold, the attack result matching the evaluation sample pair is determined as a successful attack.

[0137] The similarity threshold can be preset as needed.

[0138] Specifically, when the similarity of the recognition result is greater than the similarity threshold, it means that the recognition model to be evaluated cannot accurately distinguish the adversarial sample and the adversarial sample in the evaluation sample pair, then the adversarial sample successfully attacks the adversarial sample, and the terminal determines that the attack result that matches the evaluation sample pair is a successful attack.

[0139] Step 308: Count the attack success rate based on the attack results that match the evaluation sample pairs in the evaluation sample set.

[0140] Specifically, the terminal will count the attack successes based on the attack results that match the evaluation sample pairs in the evaluation sample set to obtain the attack success rate. The attack success rate is the ratio of the number of evaluation sample pairs that match the attack success to the total number of evaluation sample pairs in the evaluation sample set, that is, the attack success rate is calculated as: Among them, N1 represents the number of evaluation sample pairs that match the attack success, and N represents the total number of evaluation sample pairs in the evaluation sample set.

[0141] Step 310, obtaining a model evaluation result according to the attack success rate.

[0142] Specifically, the higher the attack success rate, the worse the security of the identification model to be evaluated, and the lower the attack success rate, the better the security of the identification model to be evaluated. The terminal can directly obtain the model evaluation result based on the attack success rate, that is, whether the security of the identification model to be evaluated is good or bad. Further, the terminal can determine the security level of the identification model to be evaluated based on the preset corresponding relationship between the attack success rate and the security level, and use the security level as the model evaluation result. Among them, the corresponding relationship between the attack success rate and the security level can be set as needed. For example, the corresponding relationship between the attack success rate and the security level can be: the attack success rate is less than 50% for a high security level, the attack success rate is greater than 50% and less than 75% for a medium security level, and the attack success rate is greater than 75% for a low security level.

[0143] The above-mentioned model evaluation method uses the recognition model to be evaluated to identify the evaluation sample pairs in the obtained evaluation sample set, and obtains the recognition results matching the evaluation sample pairs. The evaluation sample pairs include adversarial samples and adversarial samples. Based on the adversarial sample recognition results and the adversarial sample recognition results in the recognition results, the similarity of the recognition results matching the evaluation sample pairs is determined. When the similarity of the recognition results is greater than the similarity threshold, the attack results matching the evaluation sample pairs are determined to be successful attacks. Based on the attack results matching the evaluation sample pairs in the evaluation sample set, the attack success rate is calculated, and the model evaluation results are obtained according to the attack success rate. Since adversarial samples with high mobility and high generation efficiency are used in the model evaluation process, the success rate and efficiency of adversarial attacks can be improved.

[0144] In one embodiment, Figure 4 As shown, a flowchart is used to illustrate the adversarial sample generation method of the present application, and the adversarial sample generation method specifically includes the following steps:

[0145] Step 402: Identify the attack sample and the attacked sample respectively using a first identification model to obtain first identification results that match the attack sample and the attacked sample respectively.

[0146] Step 404: The attack sample and the attacked sample are respectively identified by a second identification model to obtain second identification results that match the attack sample and the attacked sample respectively. The second identification model and the first identification model have different identification tasks.

[0147] Step 406: Obtain feature loss of the first recognition model based on intermediate layer recognition features generated by the first recognition model when recognizing the attacking sample and the attacked sample.

[0148] Specifically, the terminal calculates the feature difference between the intermediate layer identification features that match the attack sample and the attacked sample, and obtains the feature loss of the first recognition model based on the feature difference. It should be noted that the intermediate layer identification features can be specifically a feature matrix that characterizes the attack sample and the attacked sample, and the feature difference between the intermediate layer identification features can be calculated by matrix subtraction. Furthermore, the binary norm of the feature difference can be used as the feature loss.

[0149] Step 408: Calculate the recognition result loss of the first recognition model based on the first recognition result.

[0150] Specifically, the terminal calculates the recognition result similarity between the first recognition results that match the attack sample and the attacked sample, and calculates the recognition result loss of the first recognition model based on the recognition result similarity. The first recognition results that match the attack sample and the attacked sample can be feature vectors that characterize the attack sample and the attacked sample, and the recognition result similarity between the first recognition results can be calculated by cosine distance. After obtaining the recognition result similarity, the similarity difference between the recognition result similarity and the similarity threshold can be used as the recognition result loss, and the similarity threshold can be set as needed.

[0151] Step 410, obtaining a first loss of a first recognition model according to the feature loss and the recognition result loss, and obtaining a second loss of a second recognition model based on the second recognition result.

[0152] Specifically, the terminal can obtain the first loss of the first recognition model by weighting the feature loss and the recognition result loss. The method of obtaining the second loss of the second recognition model based on the second recognition result is similar to the method of obtaining the first loss of the first recognition model based on the first recognition result in the above embodiment, and this embodiment will not be described here.

[0153] Step 412, optimizing the first recognition task based on the first loss to obtain an intermediate adversarial perturbation.

[0154] Specifically, the terminal will perform a gradient solution on the attack sample based on the first loss to optimize the first recognition task and obtain an intermediate adversarial disturbance.

[0155] Step 414, based on the intermediate adversarial perturbation and the second loss, the second recognition task is optimized to obtain the first iterative adversarial perturbation.

[0156] Specifically, the terminal will perform a gradient solution on the intermediate adversarial perturbation based on the second loss to optimize the second recognition task and obtain the first iterative adversarial perturbation.

[0157] Step 416, updating the attack sample according to the first iterative adversarial disturbance to obtain the first iterative attack sample.

[0158] Specifically, the terminal will superimpose the first-iteration adversarial disturbance on the attack sample to obtain the first-iteration attack sample.

[0159] Step 418, based on the initial iterative attack sample and the attacked sample, the first recognition task and the second recognition task are optimized and iterated.

[0160] Specifically, the terminal will identify the initial iterative attack sample and the attacked sample respectively through the first recognition model matching the first recognition task, and obtain the first recognition result that matches the initial iterative attack sample and the attacked sample respectively, and identify the initial iterative attack sample and the attacked sample respectively through the second recognition model matching the second recognition task, and obtain the second recognition result that matches the initial iterative sample and the attacked sample respectively, optimize the first recognition task and the second recognition task based on the first recognition result and the second recognition result, generate the secondary iterative adversarial perturbation, update the initial iterative attack sample according to the secondary iterative adversarial perturbation, obtain the secondary iterative attack sample, and then optimize and iterate the first recognition task and the second recognition task based on the secondary iterative attack sample and the attacked sample.

[0161] Step 420, when the number of iterations of the optimization iteration reaches the iteration number threshold, an adversarial sample matching the attack sample is obtained according to the latest generated adversarial perturbation.

[0162] Specifically, when the number of iterations of the optimization iteration reaches the iteration number threshold, the terminal updates the latest generated iterative attack sample according to the latest generated adversarial perturbation to obtain an adversarial sample that matches the attack sample.

[0163] In one embodiment, Figure 5 As shown, a flowchart is used to illustrate the adversarial sample generation method of the present application. The adversarial sample generation method can be applied to Figure 5 The two-branch model in .

[0164] Specifically, the input of the two-branch model is the attack sample (I a ) and the attacked sample (I v ), the dual-branch model includes three parts, namely, a shared encoder for feature extraction, a first recognition model, and a second recognition model. After the attack sample and the attacked sample are input into the dual-branch model, the shared encoder will first perform feature extraction on the attack sample and the attacked sample respectively to obtain matching attack sample features and attacked sample features. Subsequently, the attack sample features and the attacked sample features will be input into the first recognition model and the second recognition model respectively. Among them, the first recognition model will output the first recognition result ( and ), and generates intermediate layer recognition features when performing recognition ( and ), similarly, the second recognition model will output a second recognition result that matches the attack sample and the attacked sample respectively ( and ), and generates intermediate layer recognition features when performing recognition ( and ).

[0165] After obtaining the first recognition result and the second recognition result, the terminal will perform joint task optimization on the first recognition task and the second recognition task. For the first recognition model, the terminal will calculate the feature difference between the intermediate layer recognition features that match the attack sample and the attacked sample, and obtain the feature loss of the first recognition model based on the feature difference. The recognition result similarity between the first recognition results that match the attack sample and the attacked sample is calculated, and the recognition result loss of the first recognition model is calculated according to the recognition result similarity. The weighted feature loss and recognition result loss are added to obtain the first loss of the first recognition model. Similarly, for the second recognition model, the terminal will also use the intermediate layer recognition features to calculate the feature loss of the second recognition model. And use the second recognition result to calculate the recognition result loss of the second recognition model The weighted feature loss and the recognition result loss are added to obtain the second loss of the second recognition model.

[0166] Subsequently, by drawing on the idea of ​​meta-learning, a joint task optimization algorithm is designed to optimize and iterate the first recognition task and the second recognition task. In the first iteration, the algorithm first selects the first recognition task for optimization, optimizes the first recognition task based on the first loss, generates an intermediate adversarial perturbation, and then selects the second recognition task for optimization. Based on the intermediate adversarial perturbation and the second loss, the second recognition task is optimized to generate the first-iteration adversarial perturbation. In the second iteration, the second recognition task is first selected for optimization to generate the intermediate adversarial perturbation, and then the first recognition task is selected for optimization to generate the adversarial perturbation, that is, the first recognition task and the second recognition task are optimized alternately until the number of iterations of the optimization iteration reaches the iteration number threshold, and the adversarial sample (I) matching the attack sample is obtained according to the latest generated adversarial perturbation. adv ).

[0167] Furthermore, after generating the first iterative adversarial perturbation, the terminal will update the attack sample based on the first iterative adversarial perturbation to generate the first iterative attack sample. Subsequently, after each generation of the adversarial perturbation, the terminal will update the latest generated iterative attack sample based on the latest generated adversarial perturbation. When the number of optimization iterations reaches the iteration number threshold, an adversarial sample matching the attack sample can be obtained.

[0168] It should be noted that if the number of the second recognition model is 1, then during the optimization iteration, the odd-numbered iteration is the same as the first iteration, and the even-numbered iteration is the same as the second iteration. If the number of the second recognition model is at least two, then during the optimization iteration, the total number of the first recognition model and the second recognition model needs to be used as the cycle, and the optimization iteration is performed based on the cycle to ensure that the optimization order of the recognition task in two adjacent cycles is different.

[0169] In one embodiment, the first recognition model may be a face recognition model, and the second recognition model may be a face attribute recognition model. The face recognition model may output a feature vector representing the attack sample and the attacked sample by recognizing the attack sample and the attacked sample, i.e., a first recognition result. The first recognition result may be used to calculate whether the face matching the attack sample and the face matching the attacked sample represent the same person. When the first recognition model is a face recognition model and the second recognition model is a face attribute recognition model, the attack sample and the attacked sample are both face images, and the terminal may obtain the attack sample and the attacked sample by preprocessing the collected face images. Specifically, the terminal may first register the collected face images, i.e., detect the coordinates of the facial features in the collected face images through an algorithm, and then use the coordinates of the facial features returned by the algorithm to extract the face part from the entire image, and adjust the extracted face to a uniform preset size, which may be used as the attack sample and the attacked sample. Similarly, the adversarial sample in the evaluation sample pair in the evaluation sample set may also be obtained by the above-mentioned preprocessing method. Among them, in this embodiment, the algorithm for detecting the coordinates of the facial features is not limited, as long as the detection of the coordinates of the facial features can be achieved.

[0170] Furthermore, when the first recognition model is a face recognition model and the second recognition model is a face attribute recognition model, the recognition model to be evaluated can specifically be a face recognition model with a model structure different from that of the first recognition model. The adversarial sample generated by the above-mentioned adversarial sample generation method can be applied to perform efficient model evaluation on the recognition model to be evaluated, and determine the security of the recognition model to be evaluated. Furthermore, the adversarial sample generation method and model evaluation method in the present application can be applied to applications related to face recognition, and the face recognition models used in applications related to face recognition are evaluated. Among them, applications related to face recognition include but are not limited to terminal face unlocking, application face login, remote face verification, face access control system, offline face payment, automatic face clearance, etc.

[0171] It should be noted that the adversarial sample generation method in this application adopts a multi-task learning mode, uses the second recognition task as an auxiliary task of the first recognition task, makes full use of the additional information of the auxiliary task, and requires attacking the second recognition task while attacking the first recognition task. In the attack process, the idea of ​​meta-learning is used to alternately optimize the first recognition task and the second recognition task to generate adversarial samples. In this way, not only can the attack transferability of the final adversarial sample relative to the white-box attack be improved, but also the defect of too many queries in the black-box attack can be avoided. There is no need to perform massive queries, and the efficiency of generating adversarial samples is improved, thereby improving the efficiency of adversarial attacks. Among them, multi-task learning has been proven to effectively improve the generalization ability of the model training stage. Therefore, in this application, by adopting a multi-task learning mode when generating adversarial samples, the transferability of adversarial attacks can also be improved, and adversarial samples with strong transferability can be obtained, and adversarial samples with strong transferability can be used to improve the success rate of adversarial attacks.

[0172] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0173] Based on the same inventive concept, the embodiment of the present application also provides an adversarial sample generation device for implementing the adversarial sample generation method and a model evaluation device for implementing the model evaluation method involved above. The implementation solutions provided by the adversarial sample generation device and the model evaluation device are similar to the implementation solutions recorded in the above methods, so the specific limitations in the one or more adversarial sample generation devices and model evaluation device embodiments provided below can refer to the limitations of the adversarial sample generation method and the model evaluation method above, and will not be repeated here.

[0174] In one embodiment, Figure 6 As shown, an adversarial sample generation device is provided, including: a first recognition module 602, a second recognition module 604 and an adversarial sample generation module 606, wherein:

[0175] A first identification module 602, configured to identify the attack sample and the attacked sample respectively through a first identification model, and obtain a first identification result that matches the attack sample and the attacked sample respectively;

[0176] A second recognition module 604 is used to recognize the attack sample and the attacked sample respectively through a second recognition model to obtain a second recognition result that matches the attack sample and the attacked sample respectively, and the recognition task of the second recognition model is different from that of the first recognition model;

[0177] The adversarial sample generation module 606 is used to optimize the first recognition task matching the first recognition model and the second recognition task matching the second recognition model based on the first recognition result and the second recognition result, and obtain the adversarial sample matching the attack sample according to the adversarial perturbation generated during the optimization.

[0178] The above-mentioned adversarial sample generation device respectively identifies the attack sample and the attacked sample through the first recognition model to obtain a first recognition result that matches the attack sample and the attacked sample respectively, and respectively identifies the attack sample and the attacked sample through the second recognition model with a recognition task different from that of the first recognition model to obtain a second recognition result that matches the attack sample and the attacked sample respectively. Based on the first recognition result and the second recognition result, the first recognition task that matches the first recognition model and the second recognition task that matches the second recognition model are optimized, and the adversarial sample that matches the attack sample is obtained according to the adversarial perturbation generated during the optimization. The multi-task learning mode can be adopted in the adversarial attack to improve the transferability of the adversarial attack and obtain adversarial samples with strong transferability, and then the adversarial samples with strong transferability can be used to improve the success rate of the adversarial attack. At the same time, in the process of obtaining the adversarial sample, only optimization needs to be performed without massive query, which can improve the efficiency of generating adversarial samples and thus improve the efficiency of adversarial attacks.

[0179] In one embodiment, the adversarial sample generation module is also used to obtain a first loss of the first recognition model based on the first recognition result, and to obtain a second loss of the second recognition model based on the second recognition result. Based on the first loss and the second loss, the first recognition task matching the first recognition model and the second recognition task matching the second recognition model are optimized and iterated, and an adversarial sample matching the attack sample is obtained according to the adversarial perturbation generated during the optimization iteration.

[0180] In one embodiment, the adversarial sample generation module is also used to obtain the feature loss of the first recognition model based on the intermediate layer recognition features generated by the first recognition model when identifying the attack sample and the attacked sample, calculate the recognition result loss of the first recognition model based on the first recognition result, and obtain the first loss of the first recognition model according to the feature loss and the recognition result loss.

[0181] In one embodiment, the adversarial sample generation module is also used to optimize and iterate a first recognition task that matches the first recognition model and a second recognition task that matches the second recognition model based on a first loss and a second loss. When the number of optimization iterations reaches an iteration threshold, an adversarial sample that matches the attack sample is obtained based on the most recently generated adversarial perturbation, wherein the optimization order of the recognition tasks in two adjacent iterations in the optimization iteration is different.

[0182] In one embodiment, the adversarial sample generation module is also used to optimize the first recognition task and the second recognition task in sequence based on the first loss and the second loss to obtain the first iterative adversarial perturbation, update the attack sample according to the first iterative adversarial perturbation to obtain the first iterative attack sample, and optimize and iterate the first recognition task and the second recognition task based on the first iterative attack sample and the attacked sample.

[0183] In one embodiment, the adversarial sample generation module is also used to optimize the first recognition task based on the first loss to obtain an intermediate adversarial perturbation, and to optimize the second recognition task based on the intermediate adversarial perturbation and the second loss to obtain an initial iterative adversarial perturbation.

[0184] In one embodiment, Figure 7 As shown, a model evaluation device is provided, including: a model identification module 702, a similarity determination module 704, an attack analysis module 706, a statistics module 708 and a processing module 710, wherein:

[0185] A model identification module 702 is used to identify the evaluation sample pairs in the acquired evaluation sample set through the identification model to be evaluated, and obtain the identification results matching the evaluation sample pairs, where the evaluation sample pairs include adversarial samples and adversarial samples, and the adversarial samples are generated by the above-mentioned adversarial sample generation method;

[0186] A similarity determination module 704 is used to determine the similarity of the recognition result matching the evaluation sample pair based on the adversarial sample recognition result and the adversarial sample recognition result in the recognition result;

[0187] The attack analysis module 706 is used to determine the attack result matching the evaluation sample pair as a successful attack when the similarity of the recognition result is greater than the similarity threshold;

[0188] A statistics module 708, configured to calculate the attack success rate based on the attack results that match the evaluation sample pairs in the evaluation sample set;

[0189] The processing module 710 is used to obtain a model evaluation result according to the attack success rate.

[0190] The above-mentioned model evaluation device identifies the evaluation sample pairs in the acquired evaluation sample set through the recognition model to be evaluated, and obtains the recognition results matching the evaluation sample pairs, the evaluation sample pairs include adversarial samples and adversarial samples, and determines the similarity of the recognition results matching the evaluation sample pairs based on the adversarial sample recognition results and the adversarial sample recognition results in the recognition results. When the similarity of the recognition results is greater than the similarity threshold, the attack result matching the evaluation sample pair is determined to be a successful attack. Based on the attack results matching the evaluation sample pairs in the evaluation sample set, the attack success rate is counted, and the model evaluation results are obtained according to the attack success rate. Since adversarial samples with high mobility and high generation efficiency are used in the model evaluation process, the success rate and efficiency of adversarial attacks can be improved.

[0191] Each module in the above-mentioned adversarial sample generation device and model evaluation device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0192] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 8As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless method can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for generating adversarial samples and a model evaluation method are implemented. The display unit of the computer device is used to form a visually visible image, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse, etc.

[0193] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0194] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0195] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0196] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in the above-mentioned method embodiments.

[0197] It should be noted that the data involved in this application (including but not limited to data used for analysis, stored data, displayed data, etc.) are all data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0198] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0199] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0200] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for generating adversarial samples, characterized in that: The adversarial sample generation method comprises: The computer device recognizes the attack sample and the attacked sample respectively through a first recognition model to obtain a first recognition result that matches the attack sample and the attacked sample respectively; the first recognition model is a face recognition model; the attack sample refers to an unoptimized initial adversarial sample; The attack sample and the attacked sample are respectively identified by a second recognition model to obtain second recognition results that match the attack sample and the attacked sample respectively, wherein the second recognition model and the first recognition model have different recognition tasks; the second recognition model is a face attribute recognition model that has a different recognition task from the face recognition model; Obtaining feature loss of the first recognition model based on intermediate layer recognition features generated by the first recognition model when recognizing the attack sample and the attacked sample; Based on the first recognition result, calculating the recognition result loss of the first recognition model; Obtaining a first loss of the first recognition model according to the feature loss and the recognition result loss, and obtaining a second loss of the second recognition model based on the second recognition result; Based on the first loss and the second loss, optimizing and iterating a first recognition task matched with the first recognition model and a second recognition task matched with the second recognition model; When the number of iterations of the optimization iteration reaches an iteration number threshold, an adversarial sample matching the attack sample is obtained according to the latest generated adversarial perturbation.

2. The adversarial sample generation method according to claim 1, characterized in that: The optimization order of the recognition task in two adjacent iterations in the optimization iteration is different.

3. The adversarial sample generation method according to claim 1, characterized in that: The optimizing iteratively of the first recognition task matched with the first recognition model and the second recognition task matched with the second recognition model based on the first loss and the second loss includes: Based on the first loss and the second loss, the first recognition task and the second recognition task are optimized in sequence to obtain a first iterative adversarial perturbation; updating the attack sample according to the first iterative countermeasure disturbance to obtain a first iterative attack sample; Based on the initial iterative attack sample and the attacked sample, the first recognition task and the second recognition task are optimized and iterated.

4. The adversarial sample generation method according to claim 3, characterized in that: The optimizing the first recognition task and the second recognition task in sequence based on the first loss and the second loss to obtain the initial iterative adversarial disturbance comprises: Optimizing the first recognition task based on the first loss to obtain an intermediate adversarial perturbation; Based on the intermediate adversarial perturbation and the second loss, the second recognition task is optimized to obtain a first iterative adversarial perturbation.

5. A model evaluation method, characterized in that: The model evaluation method includes: Identify the evaluation sample pairs in the acquired evaluation sample set by the identification model to be evaluated, and obtain the identification results matching the evaluation sample pairs, wherein the evaluation sample pairs include adversarial samples and adversarial samples, and the adversarial samples are generated by the adversarial sample generation method according to any one of claims 1 to 4 above; Based on the adversarial sample recognition result and the adversarial sample recognition result in the recognition result, determining the similarity of the recognition result matching the evaluation sample pair; When the similarity of the recognition result is greater than a similarity threshold, determining the attack result matching the evaluation sample pair as a successful attack; Counting the attack success rate based on the attack results that match the evaluation sample pairs in the evaluation sample set; A model evaluation result is obtained according to the attack success rate.

6. An adversarial sample generation device, characterized in that: The adversarial sample generating device comprises: A first recognition module, configured to respectively recognize the attack sample and the attacked sample through a first recognition model, and obtain a first recognition result that matches the attack sample and the attacked sample respectively; the first recognition model is a face recognition model; the attack sample refers to an unoptimized initial adversarial sample; a second recognition module, configured to respectively recognize the attack sample and the attacked sample through a second recognition model, and obtain second recognition results that match the attack sample and the attacked sample respectively, wherein the second recognition model and the first recognition model have different recognition tasks; and the second recognition model is a face attribute recognition model that has a different recognition task from the face recognition model; An adversarial sample generation module is used to obtain the feature loss of the first recognition model based on the intermediate layer recognition features generated by the first recognition model when identifying the attack sample and the attacked sample, calculate the recognition result loss of the first recognition model based on the first recognition result, obtain the first loss of the first recognition model according to the feature loss and the recognition result loss, and obtain the second loss of the second recognition model based on the second recognition result, optimize and iterate the first recognition task matching the first recognition model and the second recognition task matching the second recognition model based on the first loss and the second loss, and when the number of iterations of the optimization iteration reaches an iteration number threshold, obtain an adversarial sample matching the attack sample according to the latest generated adversarial perturbation.

7. The adversarial sample generation device according to claim 6, characterized in that: The optimization order of the recognition task in two adjacent iterations in the optimization iteration is different.

8. The adversarial sample generation device according to claim 6, characterized in that: The adversarial sample generation module is also used to optimize the first recognition task and the second recognition task in turn based on the first loss and the second loss to obtain a first-iteration adversarial perturbation, update the attack sample according to the first-iteration adversarial perturbation to obtain a first-iteration attack sample, and optimize and iterate the first recognition task and the second recognition task based on the first-iteration attack sample and the attacked sample.

9. The adversarial sample generation device according to claim 8, characterized in that: The adversarial sample generation module is also used to optimize the first recognition task based on the first loss to obtain an intermediate adversarial perturbation, and to optimize the second recognition task based on the intermediate adversarial perturbation and the second loss to obtain an initial iterative adversarial perturbation.

10. A model evaluation device, characterized in that: The model evaluation device comprises: A model recognition module, used to recognize evaluation sample pairs in the acquired evaluation sample set through the recognition model to be evaluated, and obtain recognition results that match the evaluation sample pairs, wherein the evaluation sample pairs include adversarial samples and adversarial samples, and the adversarial samples are generated by the adversarial sample generation method according to any one of claims 1 to 4 above; A similarity determination module, configured to determine the similarity of a recognition result matching the evaluation sample pair based on the adversarial sample recognition result and the adversarial sample recognition result in the recognition result; An attack analysis module, configured to determine, when the similarity of the recognition result is greater than a similarity threshold, that the attack result matching the evaluation sample pair is a successful attack; A statistical module, used for calculating the attack success rate based on the attack results matching the evaluation sample pairs in the evaluation sample set; The processing module is used to obtain a model evaluation result according to the attack success rate.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.