Adversarial sample training method and device, equipment, storage medium and computer program

Abstract

The application discloses an adversarial sample training method and device, equipment, a storage medium and a computer program, and relates to the field of machine learning. The method comprises the following steps: acquiring an adversarial sample; in the process of the n-th adjustment of the adversarial sample, the adversarial sample is disturbed by a disturbance value in the n-th adjustment; based on the disturbance result of the n-th adjustment, a target loss value in the n-th adjustment is determined in the local neighborhood of the adversarial sample; based on the target loss value in the n-th adjustment, the disturbance value in the n-th adjustment is updated at least once to obtain a disturbance value in the n+1-th adjustment; the adversarial sample is adjusted by the n+1-th cyclic iteration by the disturbance value in the n+1-th adjustment to obtain a target adversarial sample. Through the above method, the migration of the adversarial sample can be improved, the risk of the target model being attacked can be more accurately evaluated, and the target model can be more comprehensively protected. The application can be applied to various scenes such as cloud technology, artificial intelligence and intelligent transportation.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of machine learning, and particularly relate to an adversarial sample training method and device, equipment, storage medium and computer program. BACKGROUND

[0002] Deep learning models have been widely used in many fields. Although the accuracy of the model is getting higher and higher, the model is also vulnerable to attacks, such as adversarial attacks. Adversarial attacks refer to a process of applying a slight perturbation to the original data set to generate an adversarial sample, and then deceiving the target model through the adversarial sample. Research on adversarial attacks can better judge the vulnerable parts of the machine learning model, thereby improving the robustness of the model.

[0003] In related technologies, improving the robustness of the model usually requires improving the transferability of the adversarial sample, so that the adversarial sample can attack more types of target models. In order to generate adversarial samples with stronger transferability, a momentum-based method is usually used to generate adversarial samples to improve the transferability of the adversarial samples.

[0004] However, in the black-box attack scenario where the model parameters are unknown and the data volume is small, the transferability of the adversarial sample is poor, and the performance of the transfer attack in the current black-box attack is not satisfactory. The above method cannot improve the transfer performance of the adversarial sample. SUMMARY

[0005] Embodiments of the present application provide an adversarial sample training method, device, equipment, storage medium and computer program, which can improve the transferability of the adversarial sample, accurately locate the model vulnerability of the target model, and thus protect the target model. The technical solution is as follows.

[0006] In one aspect, an adversarial sample training method is provided, the method comprising:

[0007] obtaining an adversarial sample, the adversarial sample being a sample to be adjusted;

[0008] In the process of adjusting the adversarial sample for the nth time, the adversarial sample is disturbed by a disturbance value in the nth adjustment, n being a positive integer;

[0009] Based on the disturbance result of the nth adjustment, a target loss value in the nth adjustment is determined in the local neighborhood after the adversarial sample is mapped to a vector space;

[0010] Based on the target loss value in the nth adjustment, the disturbance value in the nth adjustment is updated at least once to obtain a disturbance value in the (n+1)th adjustment;

[0011] performing the n+1th cyclic iteration adjustment on the adversarial sample by using the perturbation value in the n+1th adjustment until the training of the adversarial sample reaches a training condition, to obtain a target adversarial sample.

[0012] In another aspect, an adversarial sample training apparatus is provided, and the apparatus comprises:

[0013] an obtaining module configured to obtain an adversarial sample, the adversarial sample being a sample to be adjusted;

[0014] a perturbation module configured to, in the process of performing the n th adjustment on the adversarial sample, perturb the adversarial sample by using a perturbation value in the n th adjustment, n being a positive integer;

[0015] a determining module configured to determine a target loss value in the n th adjustment within a local neighborhood after the adversarial sample is mapped to a vector space based on a perturbation result of the n th adjustment;

[0016] an updating module configured to update the perturbation value in the n th adjustment at least once based on the target loss value in the n th adjustment, to obtain a perturbation value in an n+1th adjustment;

[0017] an adjusting module configured to perform the n+1th cyclic iteration adjustment on the adversarial sample by using the perturbation value in the n+1th adjustment until the training of the adversarial sample reaches a training condition, to obtain a target adversarial sample.

[0018] In another aspect, a computer device is provided, and the computer device comprises a processor and a memory, the memory storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by the processor to implement the adversarial sample training method according to any one of the above embodiments of the present application.

[0019] In another aspect, a computer readable storage medium is provided, and the storage medium stores at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by a processor to implement the adversarial sample training method according to any one of the above embodiments of the present application.

[0020] In another aspect, a computer program product or a computer program is provided, and the computer program product or the computer program comprises computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the adversarial sample training method according to any one of the above embodiments.

[0021] The beneficial effects of the technical solutions provided in this application include at least the following:

[0022] After perturbing the adversarial sample with the perturbation value from the nth adjustment, a target loss value is determined within the local neighborhood corresponding to the adversarial sample. The perturbation value from the nth adjustment is then updated based on the target loss value to obtain the perturbation value from the (n+1)th adjustment. This perturbation value from the (n+1)th adjustment is used to iteratively adjust the adversarial sample until the target adversarial sample is obtained. This method avoids the situation where the number and types of successfully attacked target models are limited when directly attacking the model with unprocessed adversarial samples. It improves the transferability of the obtained adversarial samples, allowing the adjusted target adversarial samples to attack more target models. This enables a more accurate assessment of the risk of attack on the target model and a more precise location of model vulnerabilities in different target models. Ultimately, this improves the security performance of the corresponding product, prevents attacks from attackers, and provides more comprehensive protection for the target model. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram of an implementation environment provided by an exemplary embodiment of this application;

[0025] Figure 2 This is a flowchart of an adversarial example training method provided in an exemplary embodiment of this application;

[0026] Figure 3 This is a schematic diagram illustrating an update of a disturbance value according to an exemplary embodiment of this application;

[0027] Figure 4 This is a schematic diagram illustrating the k-times updating of the perturbation value provided in an exemplary embodiment of this application;

[0028] Figure 5 This is a flowchart of an adversarial example training method provided in another exemplary embodiment of this application;

[0029] Figure 6 This is a flowchart of an adversarial example training method provided in another exemplary embodiment of this application;

[0030] Figure 7 This is a structural block diagram of an adversarial example training apparatus provided in an exemplary embodiment of this application;

[0031] Figure 8 This is a structural block diagram of an adversarial example training apparatus provided in another exemplary embodiment of this application;

[0032] Figure 9 This is a structural block diagram of a server provided in an exemplary embodiment of this application. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0034] First, a brief introduction to the terms used in the embodiments of this application will be given.

[0035] Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0036] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0037] Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instruction-based learning.

[0038] White-box attack and black-box attack: there are many methods for generating adversarial samples, and they can be classified according to different classification methods. According to the degree of understanding of the target model by the attacker, it can be divided into white-box attack and black-box attack. White-box attack refers to that the attacker knows the model structure, model parameters, objective function and other detailed information of the target model; while in the black-box attack scenario, the attacker can only access the input and output of the model, that is, the attacker does not know the structure and parameters of the target model. The attacker can estimate the gradient by observing the input and output of the target model to achieve the attack, or train a corresponding substitute model to improve the transferability of the adversarial sample and implement the attack.

[0039] In the related art, improving the robustness of the model usually requires improving the transferability of the adversarial sample, so that the adversarial sample can attack more types of target models. In order to generate adversarial samples with stronger transferability, a momentum-based method is usually used to generate adversarial samples to improve the transferability of the adversarial samples. However, in the black-box attack scenario where the model parameters are unknown and the data amount is small, the transferability of the adversarial sample is poor, and the performance of the transfer attack in the current black-box attack is not satisfactory, and the above method cannot improve the transfer performance of the adversarial sample.

[0040] In the embodiments of the present application, an adversarial sample training method is provided, which can improve the transferability of the adversarial sample, find the model vulnerabilities of different target models and protect the target model. The adversarial sample training method obtained by the present application includes at least one of the following scenarios when applied.

[0041] I. Applied to traffic sign recognition scenario

[0042] Illustratively, a vehicle terminal is configured in an autonomous vehicle, which can collect and analyze images of road traffic signs and control the vehicle operation according to the recognition result. For example, the road traffic sign is "speed limit 50km / h", and the recognition result of the vehicle terminal for the road traffic sign under normal circumstances is "speed limit 50km / h", and the vehicle terminal controls the vehicle to travel at a speed below 50km / h. However, when an attacker attacks the vehicle terminal, the road traffic sign image collected by the vehicle terminal is disturbed to obtain the adversarial sample of the road traffic sign image, so that the recognition result of the vehicle terminal for the road traffic sign is misjudged. For example, the road traffic sign is "speed limit 50km / h", and the recognition result of the vehicle terminal for the road traffic sign is "speed limit 110km / h", and the vehicle terminal may control the vehicle to travel at a speed of 80km / h, thereby greatly increasing the risk of traffic accidents. Using the above adversarial sample training method, while perturbing the adversarial sample, the perturbation value applied to the perturbation is updated to minimize the target loss value in the local neighborhood of the adversarial sample, and the adversarial sample is adjusted to obtain a target adversarial sample with stronger migration. Through the target adversarial sample, the model vulnerability of the vehicle terminal in judging the road traffic sign can be located, and the security of the weak part of the model is improved, thereby avoiding the case that the vehicle terminal makes a mistake in judgment due to a small disturbance of the road traffic sign, improving the safety of autonomous driving, and protecting road traffic safety.

[0043] II. Training scenario applied to classification model

[0044] Illustratively, the classification model is used to classify the input data, thereby better analyzing the data. However, when an attacker wants to intervene in the data analysis result, the attacker may disturb the data from the input process to attack the classification model. For example, the input data is an image, and the attacker applies a subtle disturbance that the human eye cannot perceive on the image through a pre-trained substitute model to obtain an adversarial sample. The adversarial sample can still be correctly recognized by the human system, but it will make the classification model output an incorrect classification result, thereby affecting the analysis result of the data. Using the above adversarial sample training method, while perturbing the adversarial sample, the perturbation value applied to the perturbation is updated to reduce the target loss value in the local neighborhood of the adversarial sample, and the adversarial sample is adjusted to obtain a target adversarial sample with stronger migration. Through the target adversarial sample attacking the classification model, the security vulnerability of the classification model can be found more accurately, and the security of the part is improved, thereby further improving the security performance of the classification model, and protecting the accuracy of the data analysis from the source of the data classification result.

[0045] It can be understood that in the specific embodiments of the present application, data related to user information is involved, and when the above embodiments of the present application are applied to specific products or technologies, user permission or consent needs to be obtained, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of countries and regions.

[0046] It is worth noting that the above application scenarios are only illustrative examples, and the adversarial sample training method provided in the present embodiment can also be applied to other scenarios, which are not limited by the present embodiment.

[0047] Secondly, the implementation environment involved in the embodiments of the present application is described, and schematically, please refer to Figure 1 The implementation environment involves a terminal 110, a server 120, and the terminal 110 and the server 120 are connected through a communication network 130.

[0048] In some embodiments, the terminal 110 is configured to send the adversarial sample to the server 120. In some embodiments, the terminal 110 is installed with an application program having an adversarial sample generation function.

[0049] The adversarial sample training method provided in the present embodiment can be implemented by the terminal 110 alone, or by the server 120, or by the terminal 110 and the server 120 through data interaction, which is not limited by the present embodiment. In the present embodiment, the terminal 110 obtains the adversarial sample through the application program having the adversarial sample generation function, and then sends the obtained adversarial sample to the server 120. The server 120 adjusts the obtained adversarial sample for the nth time, and the adjustment process is taken as an example for description.

[0050] Optionally, after receiving the adversarial sample sent by the terminal 110, the server 120 first disturbs the adversarial sample according to the disturbance value in the nth adjustment, determines the target loss value in the local neighborhood corresponding to the adversarial sample according to the disturbance result, updates the disturbance value in the nth adjustment according to the target loss value, obtains the disturbance value in the (n+1)th adjustment, and adjusts the adversarial sample according to the disturbance value in the (n+1)th adjustment. If the training of the adversarial sample reaches the training condition, the target adversarial sample is obtained; if the training of the adversarial sample has not reached the training condition, the adversarial sample is disturbed according to the disturbance value in the (n+1)th adjustment, and the above adjustment process is repeated, that is, the (n+1)th adjustment process is performed, until the training of the adversarial sample reaches the training condition, and the target adversarial sample is obtained.

[0051] It is worth noting that the terminal described above includes but is not limited to mobile terminals such as mobile phones, tablets, portable laptop computers, smart voice interaction devices, smart home appliances, vehicle-mounted terminals, etc., and can also be implemented as a desktop computer, etc. The server described above can be a standalone physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content distribution networks (CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

[0052] Among them, cloud technology refers to a hosting technology that unifies a series of resources such as hardware, applications, networks, etc. in a wide area network or local area network to realize data calculation, storage, processing and sharing. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on cloud computing business model application, which can form a resource pool, and can be used on demand, flexibly and conveniently. Cloud computing technology will become an important support. The background service of the technical network system needs a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the high development and application of the Internet industry, every item in the future may have its own identification mark and needs to be transmitted to the background system for logical processing. Different levels of data will be processed separately, and various industry data will need strong system support, which can only be realized through cloud computing.

[0053] In some embodiments, the server described above can also be implemented as a node in a blockchain system. Blockchain is a new application mode of distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm and other computer technologies. Blockchain is essentially a decentralized database, which is a series of data blocks associated using cryptography. Each data block contains information about a batch of network transactions, which is used to verify the validity (anti-fake) of the information and generate the next block. Blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer.

[0054] In combination with the above introduction of terms and application scenarios, the adversarial sample training method provided by the present application is described. As shown in Figure 2 , the method includes the following steps 210 to 250.

[0055] Step 210, obtaining an adversarial sample.

[0056] Among them, the adversarial sample is a sample to be adjusted.

[0057] The adversarial sample is used to indicate a sample obtained by adding a tiny perturbation that is imperceptible to the human visual system to the original sample. When the adversarial sample is input into the target model, the target model will obtain a wrong prediction result, that is, the original sample and the adversarial sample are extremely difficult to distinguish in visual difference, but will make the target model obtain different prediction results. Optionally, the "adversarial" in the adversarial sample is used to indicate an attack on the machine learning system (the target model).

[0058] Illustratively, the original sample is a panda image. After the panda image is input into the target model, the target model will identify the panda image. For example, the identification result of the target model is output in the form of probability, in which 78% possibility is a panda, and "panda" is taken as the identification result of the original sample by the target model.

[0059] Optionally, after the panda image is perturbed, a perturbed image is obtained. When the perturbed image is input into the target model, the target model will identify the perturbed image. For example, the identification result of the target model is 66% possibility of an elephant, and "elephant" is taken as the identification result of the perturbed sample by the target model. The perturbed image obtained by perturbing the panda image does not interfere with the human visual system, that is, the human visual system can still correctly identify the perturbed image (the identification result of the perturbed image by the human visual system is still "panda"), but the perturbed image can cause the target model to make a mistake, so the perturbed image is the adversarial sample of the target model.

[0060] In an optional embodiment, at least one candidate adversarial sample is generated by a substitute model.

[0061] Illustratively, the substitute model is a pre-trained model. In a black-box attack scenario, the target model is a model to be attacked by the attacker, and the attacker does not know the model parameters and the model algorithm of the target model. When the attacker wants to obtain an adversarial sample capable of attacking the target model, a candidate substitute model is trained by using a data set having the same or similar distribution as the target model, wherein the candidate substitute model is a model to be trained.

[0062] Optionally, in the process of training the candidate substitute model, the data in the data set having the similar distribution as the target model is perturbed to obtain a perturbed sample for training the candidate substitute model. Illustratively, the data set is an image set, and any image is selected from the image set as a sample image, for example, the sample image is a photo of a building A. After the pixels of the photo are perturbed, a perturbed sample is obtained. The perturbed sample still presents the state of the photo of the building A to the human visual system.

[0063] Optionally, the candidate substitute model is a classification model having a certain classification function, and when the photo of the building A is input into the candidate substitute model, the recognition result of the candidate substitute model is "building photo"; and when the perturbed sample after the photo of the building A is perturbed is input into the candidate substitute model, the recognition result of the candidate substitute model is "elephant photo", that is, the perturbed sample after the sample image is perturbed makes the candidate substitute model misjudge.

[0064] Optionally, the candidate substitute model is trained based on the output result of the candidate substitute model for the perturbed image and the difference between the sample image corresponding to the perturbed image, and finally the substitute model is obtained. Illustratively, the output form of the candidate substitute model is a probability form, and the probability of the classification result with a lower probability is increased as the target, the direction of the increased probability is repeatedly tested, and the corresponding direction is perturbed, so that the candidate substitute model outputs an incorrect classification result, and the input corresponding to the incorrect classification result is the sample that makes the candidate substitute model misjudge. For example: when the photo of the building A is input into the candidate substitute model, the recognition result of the candidate substitute model is "87.2% probability of building photo, 9.3% probability of person photo, and 3.5% probability of animal photo", the probability of the animal photo is increased as the target, the sample image (the photo of the building A) is perturbed, so that the candidate substitute model outputs an incorrect classification result, and the candidate substitute model is trained based on the perturbation by the gradient training method. Illustratively, the substitute model obtained by training can generate an adversarial sample that makes the target model misjudge, and the process of obtaining the adversarial sample is realized.

[0065] Optionally, the adversarial sample can also be obtained according to the original image by replacing the class variable in the adversarial perturbation with the class with the smallest recognition probability and then subtracting the class variable from the original image. It should be noted that the above is only an illustrative example, and the embodiments of the present application are not limited in this regard.

[0066] In an optional embodiment, the adversarial sample is obtained from at least one candidate adversarial sample.

[0067] Illustratively, after the substitute model generates a plurality of candidate adversarial samples, one of the plurality of candidate adversarial samples is randomly selected as the adversarial sample, that is, one of the plurality of candidate adversarial samples is randomly selected as the adversarial sample in an equal-probability selection manner.

[0068] In step 220, in the process of adjusting the adversarial sample for the nth time, the adversarial sample is perturbed by the perturbation value in the nth adjustment.

[0069] The transferability of the adversarial sample is used to indicate the attack range of the adversarial sample on different target models. The stronger the transferability of the adversarial sample is, the more target models can be misjudged by the adversarial sample. Illustratively, when evaluating the risk of the target model being attacked, improving the transferability of the adversarial sample helps to find the security vulnerabilities of the target model more quickly. Optionally, the adversarial sample is adjusted with the goal of improving the transferability of the adversarial sample. Illustratively, after obtaining the adversarial sample, the adversarial sample is adjusted in a loop iteration.

[0070] Optionally, the perturbation value is used to indicate the perturbation amplitude of the adversarial sample, and the adversarial sample is perturbed with the perturbation value in the nth adjustment during the nth adjustment of the adversarial sample.

[0071] Step 230, based on the perturbation result of the nth adjustment, determining the target loss value in the nth adjustment within the local neighborhood after the adversarial sample is mapped to the vector space.

[0072] Wherein, n is a positive integer.

[0073] Illustratively, after obtaining the adversarial sample, the adversarial sample is mapped to the vector space to obtain a sample vector corresponding to the adversarial sample, and a preset region around the sample vector is determined as the local neighborhood.

[0074] In an optional embodiment, the adversarial sample is a sample generated by a substitute model, the substitute model corresponds to a substitute loss function, and the substitute loss function is a loss function used by an attacker. Illustratively, the substitute loss function is maximized to generate a perturbation on the adversarial sample, and the perturbation is referred to as a reverse adversarial perturbation (RAP). Optionally, after the maximization operation of the substitute loss function, a maximum loss value within the local neighborhood after the adversarial sample is mapped to the vector space is obtained, and the maximum loss value is determined as the target loss value in the nth adjustment.

[0075] Wherein, the adversarial sample is a sample obtained by slightly perturbing the original sample. Illustratively, after the maximization operation of the substitute loss function, the formula for determining the perturbation value is as follows.

[0076]

[0077] Wherein, n adv Indicates the perturbation value; argmax is a function of parameter function (determines the set of independent variables according to the dependent variable); ∈ n Indicates the local search range of the reverse adversarial perturbation, wherein the local search range is a range preset in the vector space; L indicates the substitute loss function; M s (·, θ) indicates the substitute model; xadv for indicating an adversarial sample; y t for indicating a target class label.

[0078] Optionally, the adversarial sample generation algorithm is classified as target attack and non-target attack according to whether the attacker specifies the classification output category of the adversarial sample when generating the adversarial sample. The non-target attack refers to that the classification output category of the adversarial sample after passing through the target model is different from that of the original sample; while the target attack refers to that the output category of the adversarial sample is specified before the algorithm operation, and the classification output category of the adversarial sample after passing through the target model needs to be the same as the specified output category. The above formula is used to indicate the target attack, and when the above formula is applied to the non-target attack, the negative of the substitute loss function L is taken, and the target class label y is changed to the correct class label y, that is, when applied to the non-target attack, the classification output result of the target model is different from the correct class label y corresponding to the original sample. t

[0079] In an optional embodiment, at least one loss value in the local neighborhood after the adversarial sample is mapped to the vector space is determined based on the perturbation result of the nth adjustment.

[0080] The perturbation result of the nth adjustment is used to indicate the perturbation of the adversarial sample by the perturbation value in the nth adjustment, for example, the perturbation value in the nth adjustment is larger, and the perturbation amplitude of the adversarial sample is larger; the perturbation value in the nth adjustment is smaller, and the perturbation amplitude of the adversarial sample is smaller, etc.

[0081] Optionally, the loss values of different points in the local area are determined based on the perturbation result of the nth adjustment and the local neighborhood corresponding to the sample vector, and at least one loss value in the local neighborhood after the adversarial sample is mapped to the vector space is obtained. Illustratively, the target loss value in the nth adjustment is determined from the at least one loss value.

[0082] In an optional embodiment, the maximum loss value in the at least one loss value is taken as the target loss value in the nth adjustment. Illustratively, after obtaining the at least one loss value, the loss value with the maximum value in the at least one loss value is taken as the target loss value in the nth adjustment.

[0083] It is worth noting that the above is only an illustrative example, and the embodiments of the present application are not limited in this regard.

[0084] In step 240, the perturbation value in the nth adjustment is updated at least once based on the target loss value in the nth adjustment process, and the perturbation value in the nth+1 adjustment is obtained.

[0085] ​In the process of cyclically and iteratively adjusting the adversarial sample, the perturbation value applied to the perturbation of the adversarial sample is updated, and the adversarial sample is disturbed again with the updated perturbation value.

[0086] In an optional embodiment, the gradient operation is performed on the target loss value in the nth adjustment to determine the perturbation value update amount in the nth adjustment.

[0087] Illustratively, the gradient operation is performed on the target loss value in the nth adjustment to obtain a gradient operation value; within a pre-set target perturbation range, the step length for updating the perturbation value is determined; and based on the gradient operation value and the step length, the perturbation value update amount in the nth adjustment is determined.

[0088] The perturbation value is within the target perturbation range, the target perturbation range is a pre-set range, and the step length is used to indicate the moving distance of each update of the perturbation value.

[0089] Optionally, when the step length for updating the perturbation value is the same as the numerical value corresponding to the target perturbation range, it is used to indicate one update of the perturbation value; when the pre-set update times are T times, the step length for updating the perturbation value is equal to the quotient of the numerical value corresponding to the target perturbation range and the pre-set update times.

[0090] Illustratively, after the adversarial sample is determined, the perturbation value update amount in the nth adjustment is determined based on the gradient operation value obtained by performing the gradient operation on the target loss value in the nth adjustment and the step length.

[0091] In an optional embodiment, the perturbation value in the nth adjustment and the perturbation value update amount in the nth adjustment are combined to obtain the perturbation value in the (n+1)th adjustment.

[0092] Optionally, the perturbation value in the nth adjustment and the perturbation value update amount in the nth adjustment are summed to obtain the perturbation value in the (n+1)th adjustment. Illustratively, the formula for updating the perturbation value by using the above method is as follows.

[0093]

[0094] wherein, n adv is used to indicate the perturbation value, illustratively, n adv is used to indicate the updated perturbation value in the nth adjustment, n adv is used to indicate the perturbation value in the nth adjustment that has not been updated; α n is used to indicate the step length for updating the perturbation value; sign is a sign function used to simplify the integral operation; is used to indicate the gradient operation on the perturbation value; L is used to indicate the substitute loss function; Ms(·, θ) is used to indicate the substitute model; x advUsed to indicate adversarial examples; y t Used to indicate the target category label applied to a target attack.

[0095] In an optional embodiment, the perturbation value in the nth adjustment is updated at least once. That is, during the nth adjustment of the adversarial sample, the update of the perturbation value in the nth adjustment includes both a single update and multiple updates.

[0096] Optionally, such as Figure 3 As shown, during the nth adjustment of the adversarial sample 310, the perturbation value 320 in the nth adjustment is updated once. After perturbing the adversarial sample 310 with the perturbation value 320 in the nth adjustment, the target loss value in the nth adjustment is determined. Based on the target loss value and the step size, the perturbation value 320 in the nth adjustment is updated once using the above perturbation value update formula, resulting in the perturbation value 330 in the (n+1)th adjustment. Then, the adversarial sample 310 is adjusted based on the perturbation value 330 in the (n+1)th adjustment.

[0097] Optionally, the perturbation value in the nth adjustment is updated for the kth time using the target loss value in the nth adjustment, to obtain the perturbation value updated for the kth time in the nth adjustment, where k is a positive integer.

[0098] In an optional embodiment, the perturbation value in the (n+1)th adjustment is obtained based on the perturbation value updated in the kth adjustment in the nth adjustment.

[0099] Optionally, the perturbation value updated in the kth update of the nth adjustment is used to perturb the adversary sample to determine the loss value corresponding to the perturbation value updated in the kth update of the nth adjustment; based on the loss value corresponding to the perturbation value updated in the kth update of the nth adjustment, the perturbation value in the nth adjustment is updated for the (k+1)th time; in response to the perturbation value updated in the kth update of the nth adjustment reaching the convergence condition, the perturbation value in the (n+1)th adjustment is obtained.

[0100] Indicatively, the target loss value is the maximum loss value in the nth adjustment, such as... Figure 4As shown, in the process of the n th adjustment to the adversarial sample 410, the disturbance value 420 in the n th adjustment is updated for k times. After the adversarial sample 410 is disturbed by the first disturbance value 421 in the n th adjustment, the maximum loss value in the n th adjustment is determined, and based on the maximum loss value in the n th adjustment and the step length, the disturbance value update formula is used to update the first disturbance value 421 in the n th adjustment once, and the second disturbance value 422 in the n th adjustment after the first update in the n th adjustment is obtained. Alternatively, after the adversarial sample 410 is disturbed by the k-1 th disturbance value 423 in the n th adjustment, the k-1 th update is performed on the k-1 th disturbance value 423 in the n th adjustment based on the maximum loss value determined after the disturbance, and the k th disturbance value 424 in the n th adjustment is obtained. Wherein, the first disturbance value 421 in the n th adjustment, the second disturbance value 422 in the n th adjustment, the k-1 th disturbance value 423 in the n th adjustment and the k th disturbance value 424 in the n th adjustment belong to the disturbance value 420 in the n th adjustment. Alternatively, the adversarial sample 410 is adjusted by the k th disturbance value 424 in the n th adjustment.

[0101] Illustratively, after the k th update is performed on the k th disturbance value 424 in the n th adjustment, the disturbance value 430 in the n+1 th adjustment is obtained, and the process of updating the disturbance value 420 in the n th adjustment for k times is realized, that is, in the process of the n th adjustment to the adversarial sample 410, the disturbance value 420 in the n th adjustment can be updated for multiple times first.

[0102] In an optional embodiment, after the k th update is performed on the k th disturbance value 424 in the n th adjustment, the adversarial sample 410 is adjusted by the disturbance value 430 in the n+1 th adjustment after the k th update. The above is only an illustrative example, and the embodiments of the present application are not limited thereto.

[0103] Alternatively, when updating the disturbance value in the n+1 th adjustment, either the one-time update method or the above multiple update method can be used, that is, the number of times of updating the disturbance value in the process of the n th adjustment to the adversarial sample can be fixed or variable. The above is only an illustrative example, and the embodiments of the present application are not limited thereto.

[0104] Step 250: performing n+1 th cyclic iteration adjustment to the adversarial sample by the disturbance value in the n+1 th adjustment, until the training of the adversarial sample reaches the training condition, and obtaining the target adversarial sample.

[0105] Schematic: After obtaining the (n+1)th perturbation value, the adversarial sample is adjusted using the perturbation value from the (n+1)th adjustment, and it is determined whether the adjusted adversarial sample meets the training conditions. Optionally, when the adjusted adversarial sample meets the training conditions, a target adversarial sample is obtained, which has better transferability than the obtained adversarial sample; when the adjusted adversarial sample does not yet meet the training conditions, the adversarial sample is adjusted again using the perturbation value from the (n+1)th adjustment, that is: perturb the adversarial sample using the perturbation value from the (n+1)th adjustment, determine the target loss value from the (n+1)th adjustment, update the perturbation value from the (n+1)th adjustment at least once using the target loss value from the (n+1)th adjustment to obtain the perturbation value from the (n+2)th adjustment, and adjust the adversarial sample using the perturbation value from the (n+2)th adjustment, and again determine whether the adjusted adversarial sample meets the training conditions.

[0106] Optionally, in the process of obtaining the target adversarial sample based on the adversarial sample, it is necessary to judge the adjustment of the adversarial sample, that is, to judge whether the adjusted adversarial sample meets the training conditions, wherein the training conditions include at least one of the following.

[0107] (1) The training condition is to reach the preset number of updates.

[0108] In an optional embodiment, a preset number of adjustments is determined; in response to the preset number of adjustments to the adversarial sample being reached, a target adversarial sample is obtained.

[0109] In a schematic manner, the number of adjustments to the adversarial sample is preset to m. When the number of adjustments to the adversarial sample reaches the preset number m, the adjustment of the adversarial sample stops; that is, when n = m, the adjustment of the adversarial sample stops. Optionally, the adversarial sample obtained from the last adjustment is used as the target adversarial sample to realize the process of obtaining the target adversarial sample.

[0110] (2) The training condition is that it meets the convergence state.

[0111] In an optional embodiment, in response to the adversarial example reaching a convergence state, the adversarial example obtained in the most recent cyclic adjustment is used as the target adversarial example.

[0112] Indicatively, the convergence of adversarial examples indicates that the updated adversarial example no longer changes with the perturbation value or the change is less than a preset threshold. For example, if the adversarial example after the nth adjustment and the adversarial example after the (n+1)th adjustment change with the perturbation value at the same rate, it can be considered that the adversarial example has reached a convergence state. The adversarial example after the nth adjustment or the adversarial example after the (n+1)th adjustment can be used as the target adversarial example to achieve the process of obtaining the target adversarial example.

[0113] It is worth noting that the above is only an illustrative example, and the embodiments of the present application do not limit this.

[0114] To sum up, after the adversarial sample is disturbed by the disturbance value in the n th adjustment, the target loss value is determined in the local neighborhood corresponding to the adversarial sample, and the disturbance value in the n th adjustment is updated according to the target loss value to obtain the disturbance value in the n+1 th adjustment. The adversarial sample is iteratively adjusted by the disturbance value in the n+1 th adjustment, until the target adversarial sample is obtained. By the above method, the number and types of target models successfully attacked when the model is directly attacked by the untreated adversarial sample are avoided, the transferability of the adversarial sample can be improved, so that the target adversarial sample obtained by adjustment can attack more target models, so as to further accurately evaluate the risk of the target model being attacked, and find the model vulnerability of different target models, so as to improve the security performance of the model corresponding product, avoid being attacked by attackers, and more comprehensively protect the target model.

[0115] In an optional embodiment, the process of iteratively adjusting the adversarial sample is realized by the disturbance value. Illustratively, as shown in Figure 5 The step 250 in the embodiment shown in Figure 2 The step 250 in the embodiment shown in

[0116] Step 510, based on mapping the adversarial sample into a vector space, determine the sample vector corresponding to the adversarial sample.

[0117] Illustratively, after obtaining the adversarial sample, the adversarial sample is mapped into a vector space to obtain the sample vector corresponding to the adversarial sample.

[0118] Step 520, determine the sum of the disturbance value in the n+1 th adjustment and the sample vector in the n th adjustment as the target value.

[0119] Illustratively, after obtaining the disturbance value in the n+1 th adjustment, the sum of the disturbance value in the n+1 th adjustment and the sample vector corresponding to the adversarial sample in the n th adjustment is summed to obtain the target value, wherein the sum operation is used to indicate the disturbance applied to the adversarial sample in the n th adjustment, and the amplitude of the disturbance is the amplitude corresponding to the disturbance value in the n+1 th adjustment.

[0120] Step 530, based on the target value and the substitute model, the adversarial sample is adjusted for the n+1 th iteration.

[0121] Wherein, the substitute model is a model pre-trained to generate adversarial samples.

[0122] In an optional embodiment, a data enhancement operation is performed on the target value to obtain an enhanced target value; the enhanced target value is input into the substitute model to obtain a model output value; and the adversarial sample is adjusted for the n+1th cycle iteration based on the model output value and the substitute loss function.

[0123] By way of illustration, the data enhancement operation is used to indicate that a new sample data is generated after the target value is randomly increased in jitter or disturbance, and the label of the sample data is the same as that of the target value. For example, for a picture with a label of "dog", the picture is processed in a certain manner such as blurring, cropping, and deforming, and the category of the picture will not be changed. Alternatively, for picture data, the data enhancement method at least includes translation, rotation, scaling, cropping, shear, horizontal / vertical flipping, and the like.

[0124] Through the data enhancement operation, the limited data can generate a value equivalent to more data without substantially increasing the data, and the generalization capability of the model is improved. Alternatively, after the data enhancement operation is performed on the target value to obtain the enhanced target value, the enhanced data value is output into the substitute model, and the model output value of the substitute model is obtained while the generalization capability of the substitute model is improved.

[0125] In an optional embodiment, based on the model output value and the substitute loss function, a sample update amount corresponding to the adversarial sample is determined; and the adversarial sample is adjusted for the n+1th cycle iteration with the sample update amount in the local neighborhood of the adversarial sample after the adversarial sample is mapped to the vector space.

[0126] By way of illustration, the process of adjusting the adversarial sample for the n+1th cycle iteration by the disturbance value in the n+1th adjustment can be performed by using the following formula.

[0127]

[0128] wherein x adv is used to indicate the adversarial sample, by way of illustration, x adv is used to indicate the adversarial sample after the nth adjustment, x adv is used to indicate the disturbance value that has not been adjusted in the nth adjustment, B ∈ (x) is used to indicate the local neighborhood of the adversarial sample; Clip is a clipping function used to limit the adjusted adversarial sample to be located in the local neighborhood of the adversarial sample; n adv is used to indicate the disturbance value; a is used to indicate the step length of adjusting the adversarial sample, by way of illustration, a is a pre-set hyperparameter; sign is an indicator function used to simplify the integral operation; is used to indicate the gradient operation on the disturbance value; L is used to indicate the substitute loss function; M s (·, θ) is used to indicate the substitute model; y tThe target class label under the target attack is indicated.

[0129] In step 540, the training of the adversarial sample is performed until a training condition is reached, and a target adversarial sample is obtained.

[0130] Illustratively, after obtaining the (n+1)th perturbation value, the adversarial sample is adjusted with the perturbation value in the (n+1)th adjustment, and it is determined whether the adjusted adversarial sample meets the training condition.

[0131] Optionally, a preset number of updates is determined; in response to the number of updates of the adversarial sample reaching the preset number of updates, a target adversarial sample is obtained; or, in response to the adversarial sample reaching a convergence state, the adversarial sample obtained through the last cycle of adjustment is taken as the target adversarial sample.

[0132] It is worth noting that the above is only illustrative, and the embodiments of the present application are not limited in this regard.

[0133] In summary, after the adversarial sample is perturbed with the perturbation value in the nth adjustment, a target loss value is determined in the local neighborhood corresponding to the adversarial sample, and the perturbation value in the (n+1)th adjustment is updated according to the target loss value. The adversarial sample is iteratively adjusted with the perturbation value in the (n+1)th adjustment, until a target adversarial sample is obtained. Through the above method, the transferability of the adversarial sample can be improved, so as to further accurately find the model vulnerabilities of different target models, so as to improve the security performance of the model corresponding product.

[0134] In the embodiments of the present application, the process of adjusting the adversarial sample based on the perturbation value in the (n+1)th adjustment and then obtaining the target adversarial sample is described. Based on the mapping of the adversarial sample into the vector space, the sample vector corresponding to the adversarial sample is determined, the target value is obtained based on the perturbation value in the (n+1)th adjustment and the sample vector, and the adversarial sample is iteratively adjusted for the (n+1)th time based on the target value and the substitute model of the adversarial sample, until the training of the adversarial sample reaches the training adjustment, and the target adversarial sample is obtained. Through the above method, the adversarial sample is adjusted with different perturbation values after updating, which is conducive to assisting the adversarial sample to adapt to the influence of different perturbation amplitudes. When the transferability of the adversarial sample improves to a certain level, a target adversarial sample with stronger transferability is obtained, so as to attack more target models through the target adversarial sample, so as to find more model vulnerabilities of the target models and improve them, so as to more comprehensively protect the target models.

[0135] In an optional embodiment, the above training method of the adversarial sample is applied to a black box attack scenario. Illustratively, as shown in Figure 6 , the above Figure 2 The steps in the embodiments shown in the above can also be implemented as steps 610 to 640.

[0136] Step 610: Obtain adversarial examples.

[0137] In a black-box scenario, the attacker typically doesn't know the underlying architecture of the machine learning system (target model) they are attacking. That is, in a black-box attack, the model parameters of the target model are unknown to the attacker. For example, an attacker might want to attack an image recognition system, but they generally don't understand the neural network architecture or the front-end image processing components. Optionally, the attacker uses a local substitute model to approximate the image recognition system, and then generates attack samples on the substitute model. In a illustrative case, the attack samples generated on this substitute model are the original images with slight perturbations.

[0138] Alternatively, adversarial transferability from the surrogate model to the target model is analogous to model generalization between the training and test sets. The former studies overfitting to the surrogate model when constructing adversarial examples, while the latter focuses on overfitting to the training set when training the model.

[0139] Illustratively, effective techniques for improving model generalization can be used to enhance the adversarial transferability of adversarial examples. For example, based on data augmentation methods, a series of transfer attack methods focus on strengthening the robustness of adversarial examples to perturbations during the generation of adversarial perturbations and exhibit good transferability. Optionally, model generalization can be correlated with the flatness (or sharpness) of the loss landscape in the weight space, by implicitly or explicitly finding a flatter minimum in the vector space to improve the model's generalization ability.

[0140] Schematic: Adversarial examples located in flatter regions of the landscape where the alternative model loss occurs may exhibit better adversarial transferability, i.e., not only adversarial examples x adv The loss function value itself is low, and the adversarial example x adv The local neighborhood also exhibits a low loss value. Optionally, minimizing the adversarial example x is employed. adv The maximum loss value method within a local neighborhood achieves the above objective.

[0141] Step 620: Apply perturbation to the adversarial sample.

[0142] Indicative, in order to use adversarial sample x adv Find the maximum loss value within the local neighborhood, and then maximize the loss function in x. adv Generate disturbance n adv This is referred to as the Counter-Attack Perturbation (RAP). Where x adv Used to indicate the adversarial sample obtained after adding perturbations to the original sample; n adv Used to indicate the perturbation added to the adversarial sample, i.e.: n adv and xadv are all for minor changes on the picture to be identified.

[0143] Step 630, the maximum loss value in the local neighborhood determined according to the perturbation result is minimized, and the perturbation value is updated.

[0144] Optionally, the maximum loss value is minimized to obtain the following optimization scheme.

[0145]

[0146]

[0147] Wherein, B ∈ (x) is used to indicate the local neighborhood corresponding to the adversarial sample; L is used to indicate the substitute loss function; M s (·, θ) is used to indicate the substitute model; G is used to indicate the data enhancement operation; y t is used to indicate the target class label in the target attack scenario; argmax is used to indicate a function of determining a set of independent variables according to dependent variables; ∈ n is used to indicate the preset local search range in the backdoor adversarial perturbation.

[0148] Illustratively, the above optimization scheme involves a double-layer optimization problem, that is, the parameters of one optimization problem are limited by another optimization problem, and the two problems affect each other. The double-layer optimization problem includes an inner optimization problem and an outer optimization problem. The inner optimization problem will make inner decision optimization according to its own objective function, constraint condition, decision variable, etc. The optimal decision in the outer optimization problem will depend on the inner optimization problem, and make outer decision optimization according to its own objective function, constraint condition, decision variable, etc.

[0149] Optionally, the inner and outer target functions are iteratively solved by alternating optimization. Illustratively, for each iteration, given the adversarial sample x adv , the inner optimization problem about n adv is solved by the projected gradient method, and the perturbation value n adv is updated as follows:

[0150]

[0151] Step 640, a double-layer optimization method is used to adjust the adversarial sample according to the perturbation value, and obtain a target adversarial sample.

[0152] Illustratively, after solving the inner optimization problem of the perturbation value n adv by the projected gradient method, the perturbation value n adv is updated. Based on the obtained updated perturbation value n adv, the known adversarial sample x in the outer-layer optimization problem adv , and the attack algorithm adjusts the adversarial sample to obtain a target adversarial sample with stronger transferability. Optionally, the projection gradient method is used to adjust the adversarial sample x adv , the formula for adjusting the adversarial sample x adv is as follows:

[0153]

[0154] In an optional embodiment, the step length for updating the perturbation value in the inner-layer optimization problem and the step length for adjusting the adversarial sample in the outer-layer optimization problem are pre-set, and the number of times of updating the perturbation value during one adjustment of the adversarial sample can be one or multiple, that is, after the step length for updating the perturbation value is determined, when the perturbation value is updated based on the inner-layer optimization problem, one-step updating or multi-step updating can be performed.

[0155] Illustratively, when the perturbation value is updated by one step based on the inner-layer optimization problem, the updating formula of the perturbation value when solving the inner-layer optimization problem is used to update the perturbation value by one step, where the step length a n is equal to ∈ n / T, ∈ n is used to indicate the local search range of the reverse adversarial perturbation; T is used to indicate the number of updates, that is, the total number of updates of the perturbation value n adv After the perturbation value is updated by one step, the adjustment formula of the adversarial sample when solving the outer-layer optimization problem is used to adjust the adversarial sample, and the process of "updating the perturbation value by one step in the inner layer-adjusting the adversarial sample by one step in the outer layer" is repeated until the target adversarial sample is obtained.

[0156] Illustratively, when the perturbation value is updated by multiple steps based on the inner-layer optimization problem, the updating formula of the perturbation value when solving the inner-layer optimization problem is used to update the perturbation value by multiple steps, for example, ten steps. After the perturbation value is updated by ten steps, the adjustment formula of the adversarial sample when solving the outer-layer optimization problem is used to adjust the adversarial sample by one step, and the process of "updating the perturbation value by ten steps in the inner layer-adjusting the adversarial sample by one step in the outer layer" is repeated until the target adversarial sample is obtained.

[0157] Optionally, the number of times of adjusting the adversarial sample also includes one or multiple, that is, after the step length for adjusting the adversarial sample is determined, when the adversarial sample is adjusted based on the outer-layer optimization problem, one-step updating or multi-step updating can be performed. It is worth noting that the number of times of updating the perturbation value and the number of times of adjusting the adversarial sample are only illustrative examples, and the embodiments of the present application are not limited in this regard.

[0158] In an optional embodiment, considering the number of iterations when the disturbance value reaches a convergence state and enhancing the attack effect of the adversarial sample at the beginning of iteration, a late-start method is used to adjust the adversarial sample.

[0159] Optionally, the late-start method is used to indicate that when solving the double-layer optimization problem, the adjustment of the update of the disturbance value in the inner-layer optimization and the adjustment of the adversarial sample in the outer-layer optimization is adjusted. Illustratively, first, at least one adjustment is made to the adversarial sample based on the initial disturbance value (for example, the initial disturbance value is 0), for example, 50 adjustments are made to the adversarial sample by using the adjustment formula of the adversarial sample when solving the outer-layer optimization problem, and then the update formula of the disturbance value when solving the inner-layer optimization problem is used to update the disturbance value. That is, in the above late-start process, the objective function corresponding to the outer-layer optimization problem is used first, and then the objective function corresponding to the inner-layer optimization problem is used, so that the inner-layer optimization problem is performed after the outer-layer optimization problem. The above is only an illustrative example, and the embodiments of the present application are not limited in this regard.

[0160] In summary, after the adversarial sample is disturbed by the disturbance value in the nth adjustment, the target loss value is determined in the local neighborhood corresponding to the adversarial sample, and the disturbance value in the n+1 adjustment is obtained according to the target loss value, and the adversarial sample is iteratively adjusted by the disturbance value in the n+1 adjustment, until the target adversarial sample is obtained. Through the above method, the transferability of the adversarial sample can be improved, so as to further accurately find the model vulnerability of different target models, so as to improve the security performance of the model corresponding product, and more comprehensively protect the target model.

[0161] In the embodiments of the present application, the application of the above adversarial sample training method to the black-box attack scene is described. After obtaining the adversarial sample, in order to improve the transferability of the adversarial sample, not only the adversarial sample x advThe loss function value of the target model is low, and the local neighborhood corresponding to the adversarial sample also needs to have a low loss value. A perturbation is applied to the adversarial sample to determine the maximum loss value in the local neighborhood of the adversarial sample. The perturbation value of the applied perturbation is updated to minimize the maximum loss value, and the updated perturbation value is used to adjust the adversarial sample, thereby obtaining a target adversarial sample with improved transferability. In addition, during the iterative solving process of the alternating optimization of the inner and outer layer target functions, the adversarial sample can be adjusted after the perturbation value is updated multiple times during the inner layer optimization, or the alternating process of the inner and outer layer optimization can be performed after at least one outer layer optimization. Through the above method, the perturbation value of the inner layer optimization can be converged more quickly, and the attack effect of the adversarial sample at the beginning of the iteration can be improved, further improving the transfer performance of the adversarial sample, thereby more accurately positioning the model vulnerability of the target model and protecting the target model.

[0162] Figure 7 is a structural block diagram of an adversarial sample training device provided by an exemplary embodiment of the present application, as shown in Figure 7 The device includes the following parts:

[0163] The acquisition module 710 is configured to acquire an adversarial sample, the adversarial sample being a sample to be adjusted.

[0164] The perturbation module 720 is configured to, in the process of adjusting the adversarial sample for the nth time, perturb the adversarial sample with a perturbation value in the nth adjustment, n being a positive integer.

[0165] The determination module 730 is configured to determine a target loss value in the nth adjustment based on a perturbation result of the nth adjustment in a local neighborhood after the adversarial sample is mapped to a vector space.

[0166] The update module 740 is configured to update the perturbation value in the nth adjustment at least once based on the target loss value in the nth adjustment, to obtain a perturbation value in the (n+1)th adjustment.

[0167] The adjustment module 750 is configured to adjust the adversarial sample for the (n+1)th cyclic iteration with the perturbation value in the (n+1)th adjustment, until the training of the adversarial sample reaches a training condition, to obtain a target adversarial sample.

[0168] In an optional embodiment, the determination module 730 is further configured to determine at least one loss value in the local neighborhood after the adversarial sample is mapped to the vector space based on the perturbation result of the nth adjustment; and determine the target loss value in the nth adjustment from the at least one loss value.

[0169] In an optional embodiment, the determining module 730 is further configured to determine the maximum loss value in the at least one loss value as the target loss value in the n th adjustment.

[0170] As shown in FIG. 7, in an optional embodiment, the updating module 740 further includes: Figure 8

[0171] The operation unit 741 is configured to perform gradient operation on the target loss value in the n th adjustment to determine the perturbation value update amount in the n th adjustment.

[0172] The combination unit 742 is configured to combine the perturbation value in the n th adjustment and the perturbation value update amount in the n th adjustment to obtain the perturbation value in the (n+1) th adjustment.

[0173] In an optional embodiment, the operation unit 741 is further configured to perform gradient operation on the target loss value in the n th adjustment to obtain a gradient operation value; determine a step length for updating the perturbation value in the target perturbation range, the perturbation value being within the target perturbation range; and determine the perturbation value update amount in the n th adjustment based on the gradient operation value and the step length.

[0174] In an optional embodiment, the adjustment module 750 is further configured to determine a sample vector corresponding to the adversarial sample based on the mapping of the adversarial sample into the vector space; determine a target value as a sum of the perturbation value in the (n+1) th adjustment and the sample vector in the n th adjustment; and perform the (n+1) th cyclic iterative adjustment on the adversarial sample based on the target value and a substitute model, the substitute model being a model pre-trained to generate the adversarial sample.

[0175] In an optional embodiment, the substitute model corresponds to a substitute loss function.

[0176] The adjustment module 750 is further configured to perform a data augmentation operation on the target value to obtain an augmented target value; input the augmented target value into the substitute model to obtain a model output value; and perform the (n+1) th cyclic iterative adjustment on the adversarial sample based on the model output value and the substitute loss function.

[0177] In an optional embodiment, the adjustment module 750 is further configured to determine a sample update amount corresponding to the adversarial sample based on the model output value and the substitute loss function; and perform the (n+1) th cyclic adjustment on the adversarial sample in the local neighborhood after the mapping of the adversarial sample into the vector space, with the sample update amount.

[0178] ​In an optional embodiment, the adjustment module 750 is further configured to determine a preset number of updates; in response to the number of updates of the adversarial sample reaching the preset number of updates, obtain a target adversarial sample; or, in response to the adversarial sample reaching a convergence state, take the adversarial sample obtained through the last cycle of adjustment as the target adversarial sample.

[0179] In an optional embodiment, the obtaining module 710 is further configured to generate at least one candidate adversarial sample through a substitute model; and obtain the adversarial sample from the at least one candidate adversarial sample.

[0180] In an optional embodiment, the updating module 740 is further configured to update the perturbation value in the nth adjustment for the kth time using the target loss value in the nth adjustment, to obtain a perturbation value updated for the kth time in the nth adjustment, k being a positive integer; and obtain the perturbation value in the n+1th adjustment based on the perturbation value updated for the kth time in the nth adjustment.

[0181] In an optional embodiment, the updating module 740 is further configured to perturb the adversarial sample using the perturbation value updated for the kth time in the nth adjustment, to determine a loss value corresponding to the perturbation value updated for the kth time in the nth adjustment; update the perturbation value in the nth adjustment for the k+1th time based on the loss value corresponding to the perturbation value updated for the kth time in the nth adjustment; and obtain the perturbation value in the n+1th adjustment in response to the perturbation value updated for the kth time in the nth adjustment reaching a convergence condition.

[0182] In summary, after the adversarial sample is perturbed using the perturbation value in the nth adjustment, a target loss value is determined in a local neighborhood corresponding to the adversarial sample, and the perturbation value in the nth adjustment is updated according to the target loss value to obtain the perturbation value in the n+1th adjustment, and the adversarial sample is iteratively adjusted using the perturbation value in the n+1th adjustment, until a target adversarial sample is obtained. Through the above device, the situation that the number and types of target models successfully attacked are less when the adversarial sample is directly used to attack the model without processing can be avoided, the transferability of the adversarial sample can be improved, the target adversarial sample obtained through adjustment can attack more target models, so as to further accurately evaluate the risk of the target model being attacked, and find the model vulnerabilities of different target models, so as to improve the security performance of the product corresponding to the model, avoid being attacked by attackers, and more comprehensively protect the target model.

[0183] It should be noted that the adversarial sample training device provided in the above embodiments is only exemplified by the division of the above functional modules. In actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the adversarial sample training device and the adversarial sample training method provided in the above embodiments belong to the same concept, and the specific implementation process is described in the method embodiments, which will not be repeated here.

[0184] Figure 9 A structural diagram of a server provided by an example embodiment of the present application is shown. The server 900 includes a central processing unit (CPU) 901, a system memory 904 including a random access memory (RAM) 902 and a read only memory (ROM) 903, and a system bus 905 connecting the system memory 904 and the central processing unit 901. The server 900 also includes a mass storage device 906 for storing an operating system 913, application programs 914, and other program modules 915.

[0185] The mass storage device 906 is connected to the central processing unit 901 through a mass storage controller (not shown) connected to the system bus 905. The mass storage device 906 and its associated computer readable medium provide non-volatile storage for the server 900. That is, the mass storage device 906 can include a computer readable medium (not shown) such as a hard disk or a compact disc read only memory (CD-ROM) drive.

[0186] Without loss of generality, the computer readable medium can include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM, ROM, Erasable Programmable Read Only Memory (EPROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other solid state memory technology, CD-ROM, Digital Versatile Disc (DVD), or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. It should be understood by those skilled in the art that computer storage media does not limit to the above-mentioned several types. The system memory 904 and the mass storage device 906 mentioned above can be collectively referred to as memory.

[0187] According to various embodiments of the present application, the server 900 can also run on a remote computer connected to the network through a network connection such as the Internet. That is, the server 900 can be connected to the network 912 through the network interface unit 911 connected to the system bus 905, or can be connected to other types of networks or remote computer systems (not shown) using the network interface unit 911.

[0188] The above-mentioned memory further includes one or more programs, one or more programs are stored in the memory and are configured to be executed by the CPU.

[0189] Embodiments of the present application also provide a computer device including a processor and a memory, the memory storing at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to implement the adversarial sample training method provided by the above-mentioned method embodiments.

[0190] Embodiments of the present application also provide a computer readable storage medium, the computer readable storage medium storing at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to implement the adversarial sample training method provided by the above-mentioned method embodiments.

[0191] The embodiment of the present application further provides a computer program product or computer program, which comprises computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the adversarial sample training method in any of the above-mentioned embodiments.

[0192] Optionally, the computer readable storage medium can include a read only memory (ROM), a random access memory (RAM), a solid state disk (SSD), an optical disk, or the like. The random access memory can include a resistance random access memory (ReRAM) and a dynamic random access memory (DRAM). The above-mentioned serial numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0193] Those of ordinary skill in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or by a program instructing relevant hardware to complete, and the program can be stored in a computer readable storage medium. The above-mentioned storage medium can be a read only memory, a magnetic disk or an optical disk.

[0194] The above-mentioned is only the optional embodiment of the present application, and does not limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for training against samples, characterized in that, The method comprises: obtaining an adversarial sample image, the adversarial sample image being a sample to be adjusted; in the process of adjusting the adversarial sample image for the nth time, the adversarial sample image is disturbed by a disturbance value in the nth adjustment, n being a positive integer; after the maximum substitute loss function operation, the maximum loss value in the local neighborhood after the adversarial sample image is mapped into the vector space is obtained, and is determined as the target loss value in the nth adjustment; the substitute loss function is the loss function corresponding to the substitute model, and the substitute model is a model for generating the adversarial sample image obtained by pre-training; based on the target loss value in the nth adjustment, the disturbance value in the nth adjustment is updated at least once to obtain the disturbance value in the (n+1)th adjustment; the adversarial sample image is adjusted for the (n+1)th cyclic iteration by the disturbance value in the (n+1)th adjustment until the training of the adversarial sample image reaches the training condition, and a target adversarial sample image is obtained.

2. The method of claim 1, wherein, The method comprises: performing gradient operation on the target loss value in the nth adjustment to determine the disturbance value update amount in the nth adjustment; combining the disturbance value in the nth adjustment and the disturbance value update amount in the nth adjustment to obtain the disturbance value in the (n+1)th adjustment.

3. The method of claim 2, wherein, The method comprises: performing gradient operation on the target loss value in the nth adjustment to obtain a gradient operation value; determining a step length for updating the disturbance value within a pre-set target disturbance range, the disturbance value being within the target disturbance range; based on the gradient operation value and the step length, determining the disturbance value update amount in the nth adjustment.

4. The method according to any one of claims 1 to 3, characterized in that, The method comprises: based on the adversarial sample image being mapped into the vector space, determining a sample vector corresponding to the adversarial sample image; determining a target value as the sum of the disturbance value in the (n+1)th adjustment and the sample vector in the nth adjustment; based on the target value and the substitute model, adjusting the adversarial sample image for the (n+1)th cyclic iteration.

5. The method of claim 4, wherein, The substitute model corresponds to a substitute loss function; The method comprises: performing data enhancement operation on the target value to obtain an enhanced target value; inputting the enhanced target value into the substitute model to obtain a model output value; based on the model output value and the substitute loss function, adjusting the adversarial sample image for the (n+1)th cyclic iteration.

6. The method of claim 5, wherein, The method comprises: determine a sample update quantity corresponding to the adversarial sample image based on the model output value and the substitute loss function; perform an (n+1)th cycle adjustment on the adversarial sample image in the local neighborhood after the adversarial sample image is mapped to a vector space, by using the sample update quantity.

7. The method according to any one of claims 1 to 3, characterized in that, The training of the adversarial sample image until a training condition is reached to obtain a target adversarial sample image, comprising: determining a preset update number; and obtaining the target adversarial sample image in response to the number of updates on the adversarial sample image reaching the preset update number; or, obtaining the adversarial sample image in response to the adversarial sample image reaching a convergence state, and taking the adversarial sample image obtained through the last cycle adjustment as the target adversarial sample image.

8. The method according to any one of claims 1 to 3, characterized in that, The obtaining of the adversarial sample image comprises: generating at least one candidate adversarial sample image through a substitute model, the substitute model being a model pre-trained to generate the candidate adversarial sample image; and obtaining the adversarial sample image from the at least one candidate adversarial sample image.

9. The method according to any one of claims 1 to 3, characterized in that, The at least one update on the perturbation value in the nth adjustment based on the target loss value in the nth adjustment to obtain a perturbation value in an (n+1)th adjustment, comprises: performing the kth update on the perturbation value in the nth adjustment based on the target loss value in the nth adjustment to obtain a perturbation value updated for the kth time in the nth adjustment, k being a positive integer; obtaining the perturbation value in the (n+1)th adjustment based on the perturbation value updated for the kth time in the nth adjustment.

10. The method of claim 9, wherein, The at least one update on the perturbation value in the nth adjustment based on the target loss value in the nth adjustment to obtain a perturbation value in an (n+1)th adjustment, comprises: performing the kth update on the perturbation value in the nth adjustment based on the target loss value in the nth adjustment to obtain a perturbation value updated for the kth time in the nth adjustment, k being a positive integer; obtaining the perturbation value in the (n+1)th adjustment based on the perturbation value updated for the kth time in the nth adjustment. The at least one update on the perturbation value in the nth adjustment based on the target loss value in the nth adjustment to obtain a perturbation value in an (n+1)th adjustment, comprises:

11. An adversarial sample training apparatus, comprising: performing the kth update on the perturbation value in the nth adjustment based on the target loss value in the nth adjustment to obtain a perturbation value updated for the kth time in the nth adjustment, k being a positive integer; obtaining the perturbation value in the (n+1)th adjustment based on the perturbation value updated for the kth time in the nth adjustment. The device comprises: an obtaining module configured to obtain an adversarial sample image, the adversarial sample image being a sample to be adjusted; a perturbation module configured to perturb the adversarial sample image by using a perturbation value in an nth adjustment in a process of performing the nth adjustment on the adversarial sample image, n being a positive integer; a determining module configured to obtain a maximum loss value in a local neighborhood after the adversarial sample image is mapped to a vector space through a maximization substitute loss function operation, and determine the maximum loss value as a target loss value in the nth adjustment; the substitute loss function being a loss function corresponding to a substitute model, the substitute model being a model pre-trained to generate the adversarial sample image; an updating module configured to perform at least one update on the perturbation value in the nth adjustment based on the target loss value in the nth adjustment to obtain a perturbation value in an (n+1)th adjustment; and An adjusting module is configured to perform an (n+1)th loop iteration adjustment on the adversarial sample image with the disturbance value in the (n+1)th adjustment until the training of the adversarial sample image reaches a training condition, and obtain a target adversarial sample image.

12. A computer device, comprising: The computer device comprises a processor and a memory, and the memory stores at least one program, which is loaded and executed by the processor to implement the adversarial sample training method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The storage medium stores at least one program, which is loaded and executed by the processor to implement the adversarial sample training method according to any one of claims 1 to 10.

14. A computer program product, characterised in that, The storage medium stores at least one program, which is loaded and executed by the processor to implement the adversarial sample training method according to any one of claims 1 to 10.

Images