Method and related equipment for generating adversarial sample Trojans for neural network models

Abstract

The present application provides a method and related equipment for generating an adversarial sample Trojan for a neural network model, comprising: generating K first perturbations that meet predetermined requirements, wherein the predetermined requirements include objective function requirements and perturbation rate requirements for L sample images, wherein the dimension of the sample images is higher than the dimension of the first perturbation; for each of the K first perturbations, iteratively performing an operation that causes the first perturbation to decrease in the gradient direction of its objective function until a preset iteration end condition is satisfied to obtain an optimized perturbation of the first perturbation; calculating the first value of the objective function for each optimized perturbation; determining the optimized perturbation corresponding to the minimum value among the first values ​​as the target perturbation; and performing data dimensionality increase processing on the target perturbation using a trained neural network model to obtain the adversarial sample Trojan. This method improves the speed and practicality of the algorithm and accelerates the efficiency of constructing adversarial sample Trojans for neural network models.

Description

Technical Field

[0001] The present application relates to the field of machine learning technology, and in particular to a method and related equipment for generating adversarial sample Trojans for neural network models. Background Art

[0002] Neural networks are the most important machine learning models in modern artificial intelligence. They are widely used in various fields today and demonstrate performance that approaches or even surpasses human intelligence. However, the emergence of neural network Trojans has raised questions about their security. To establish a defense system against neural network Trojans, in-depth research on their characteristics is necessary. Constructing adversarial Trojans is a key component of this defense system.

[0003] Constructing adversarial Trojans provides technical support for research on neural network defense systems. Many existing construction methods often generate perturbations with excessively large norms, making them difficult to hide from human eyes. Some iterative construction methods, while achieving high success rates on neural networks trained using defensive distillation methods, still have significant limitations in terms of runtime and controllability, indicating a critical need for improvement. Summary of the Invention

[0004] In view of this, the purpose of this application is to propose a method and related equipment for generating adversarial sample Trojans for neural network models.

[0005] Based on the above objectives, this application provides a method for generating an adversarial sample Trojan for a neural network model, comprising:

[0006] Generate K first perturbations that meet predetermined requirements, wherein the predetermined requirements include objective function requirements and perturbation rate requirements for L sample images, the dimension of the sample images is a, the dimension of the first perturbation is n, K, L, a, and n are all positive integers, and a is greater than n;

[0007] For each of the K first perturbations, iteratively performing an operation of decreasing the first perturbation in the gradient direction of its objective function until a preset iteration end condition is satisfied, so as to obtain an optimized perturbation of the first perturbation;

[0008] Calculating a first value of the objective function of each of the K optimized disturbances obtained;

[0009] Determine the optimized disturbance corresponding to the minimum value among the calculated K first values ​​as the target disturbance;

[0010] The trained neural network model is used to perform data dimensionality increase processing on the target disturbance to obtain the adversarial sample Trojan.

[0011] Furthermore, the objective function includes an L2 norm.

[0012] Furthermore, generating K first disturbances that meet predetermined requirements includes repeatedly performing the following operations until K first disturbances are generated:

[0013] Use the random number generation function to randomly generate a random perturbation of dimension n;

[0014] Using the neural network model to increase the dimension of the random perturbation to a dimension to obtain a-dimensional random perturbation, and calculating the perturbation rates of the L sample images under the increased-dimensional random perturbation based on the fitting function of the sample images;

[0015] Calculating the L2 norm of the random perturbation;

[0016] In response to determining that the calculated perturbation rate is greater than a first threshold and the calculated L2 norm is less than a second threshold, the random perturbation is used as the first perturbation.

[0017] Furthermore, the iteratively performing the operation of decreasing the first disturbance in the gradient direction of its objective function until a preset iteration end condition is satisfied to obtain an optimized disturbance of the first disturbance includes:

[0018] Iteratively perform the following first update operation on the first perturbation until the perturbation rate of the L sample images under the first perturbation is less than or equal to the first threshold, stop the first update operation, and use the object of the last execution of the first update operation as the second perturbation:

[0019] Calculating a first gradient of the L2 norm of the first perturbation;

[0020] Updating the first disturbance by decreasing the first disturbance in the direction of the calculated first gradient at a preset learning rate to obtain an updated first disturbance;

[0021] Using the neural network model, the dimension of the updated first disturbance is increased to a dimension to obtain the increased-dimensional updated first disturbance, and based on the fitting function of the sample images, the disturbance rates of the L sample images under the increased-dimensional updated first disturbance are calculated;

[0022] Iteratively perform the following second updating operation on the second perturbation until the perturbation rate of the L sample images under the second perturbation is greater than the first threshold, so as to obtain an optimized perturbation of the first perturbation:

[0023] Calculating a second gradient of the L2 norm of the second perturbation;

[0024] reducing the learning rate by a predetermined ratio to update the learning rate;

[0025] Updating the second perturbation by decreasing the second perturbation in the direction of the calculated second gradient at the updated learning rate to obtain an updated second perturbation;

[0026] The neural network model is used to increase the dimension of the updated second disturbance to a dimension to obtain the increased-dimensional updated second disturbance, and based on the fitting function of the sample image, the disturbance rates of the L sample images under the increased-dimensional updated second disturbance are calculated.

[0027] Furthermore, the predetermined ratio is 50%.

[0028] Furthermore, the first threshold is 80%.

[0029] Furthermore, the trained neural network model includes an autoencoder;

[0030] The using of the trained neural network model to perform data dimensionality increase processing on the target disturbance includes: using the decoder end of the autoencoder to perform dimensionality increase processing on the target disturbance.

[0031] Furthermore, the autoencoder is trained using L1 regularized mean square error as the loss function.

[0032] Based on the same inventive concept, the present application also provides a device for generating an adversarial sample Trojan for a neural network model, comprising:

[0033] a generation module configured to generate K first perturbations that meet predetermined requirements, wherein the predetermined requirements include objective function requirements and perturbation rate requirements for L sample images, the dimension of the sample images is a, the dimension of the first perturbation is n, and K, L, a, and n are all positive integers, with a greater than n;

[0034] an optimization module configured to iteratively perform, for each of the K first perturbations, an operation of decreasing the first perturbation in a gradient direction of its objective function until a preset iteration end condition is satisfied, so as to obtain an optimized perturbation of the first perturbation;

[0035] a calculation module configured to calculate a first value of the objective function of each of the K optimized disturbances obtained by the optimization module;

[0036] a determination module configured to determine the optimized disturbance corresponding to the minimum value among the calculated K first values ​​as a target disturbance;

[0037] The dimension-increasing module is configured to use a trained neural network model to perform data dimension-increasing processing on the target disturbance to obtain the adversarial sample Trojan.

[0038] Based on the same inventive concept, the present application also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the method described above when executing the computer program.

[0039] From the above, it can be seen that the method and related equipment for generating adversarial sample Trojans for neural network models provided by this application randomly generate and screen a certain number of first perturbations with dimensions lower than the sample image, iteratively perform descent in the gradient direction on these first perturbations, quickly optimize them, obtain optimized perturbations, and screen out the target perturbation with the smallest norm based on the optimized perturbations, and perform dimensionality-raising processing on them to obtain adversarial sample Trojans. This method reduces the complexity of the algorithm by generating low-dimensional perturbations, and the perturbations of this method are randomly generated, which has a wider range of applicability. Iterative gradient optimization of multiple first perturbations can prevent the results from falling into local optimal solutions. After each iteration, the updated perturbation rate is judged to ensure that the optimization algorithm does not stop prematurely. This method improves the speed and practicality of the algorithm and accelerates the efficiency of constructing adversarial sample Trojans for neural network models. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in this application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are merely embodiments of this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0041] Figure 1 A flowchart of a method for generating an adversarial sample Trojan for a neural network model according to an embodiment of the present application;

[0042] Figure 2 A schematic diagram of the structure of an apparatus for generating an adversarial sample Trojan of a neural network model according to an embodiment of the present application;

[0043] Figure 3 This is a schematic structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the objectives, technical solutions and advantages of this application more clear, this application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0045] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should have the usual meanings understood by people with ordinary skills in the field to which this application belongs. The "first", "second" and similar words used in the embodiments of the present application do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0046] As mentioned in the background, neural networks are one of the most important models in machine learning, and the emergence of Trojans significantly impacts the security of neural networks. Constructing adversarial Trojans is a key step in establishing a defense system against neural network Trojans. Related technologies often generate perturbations as adversarial examples, but these perturbations often have large norms, are clearly targeted, and are easily identified. However, these perturbations still have limitations in terms of generation efficiency, runtime, and controllability.

[0047] This application first randomly generates low-dimensional perturbations with a lower dimensionality than the sample image. Based on certain conditions, a certain number of first perturbations are selected. These first perturbations are iteratively optimized using the gradient descent method. While maintaining a certain perturbation rate, the perturbations have a smaller norm. The perturbation with the smallest norm is selected as the target perturbation, and this perturbation is subjected to dimensionality-enhancing processing to generate adversarial sample Trojans. This method significantly reduces the amount of computation and improves computational efficiency, resulting in higher-quality adversarial sample Trojans with a wider range of applications.

[0048] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0049] This application provides a method for generating adversarial sample Trojans for neural network models, referring to Figure 1 , including the following steps:

[0050] Step S101, generating K first perturbations that meet predetermined requirements, wherein the predetermined requirements include objective function requirements and perturbation rate requirements for L sample images, the dimension of the sample images is a, the dimension of the first perturbation is n, and K, L, a, and n are all positive integers, with a greater than n;

[0051] Specifically, in this embodiment, several perturbations with n-dimensional data dimensions are randomly generated, and the several perturbations are screened based on predetermined requirements to obtain K first perturbations that meet the predetermined requirements. The predetermined requirements include the objective function requirements of the perturbation and the perturbation rate requirements of L a-dimensional sample images under the perturbation. The K, L, a, and n are all positive integers, and a is greater than n.

[0052] Step S102: for each of the K first perturbations, iteratively perform an operation of decreasing the first perturbation in the gradient direction of its objective function until a preset iteration end condition is satisfied, so as to obtain an optimized perturbation of the first perturbation;

[0053] Specifically, in this embodiment, for each of the K first perturbations, the perturbation is iteratively decreased in the gradient direction of its objective function at a preset learning rate until a preset iteration end condition is met, thereby obtaining the optimized perturbation corresponding to each first perturbation.

[0054] Step S103, calculating the first value of the objective function of each of the K optimized disturbances obtained;

[0055] Specifically, in this embodiment, for each of the K optimized disturbances, the objective function value of the optimized disturbance is calculated as the first value of the optimized disturbance.

[0056] Step S104, determining the optimized disturbance corresponding to the minimum value among the calculated K first values ​​as the target disturbance;

[0057] Specifically, in this embodiment, K optimized disturbances correspond to K first values, and the optimized disturbance corresponding to the minimum value among the K first values ​​is determined as the target disturbance.

[0058] Step S105: Use the trained neural network model to perform data dimensionality upscaling on the target disturbance to obtain the adversarial sample Trojan.

[0059] Specifically, in this embodiment, a trained neural network model is used to perform data dimensionality upscaling on the target disturbance to the same dimension as the sample image, thereby obtaining an adversarial sample Trojan.

[0060] This method generates perturbations with lower dimensions than the sample image. When calculating the objective function of the perturbation, it greatly reduces the amount of computation, improves computational efficiency, and reduces the complexity of the algorithm. In addition, the perturbations of this method are randomly generated and have a wider range of applicability. Iterative gradient optimization of multiple first perturbations can prevent the results from falling into local optimal solutions.

[0061] In some embodiments, the objective function comprises an L2 norm.

[0062] Specifically, in this embodiment, the objective function is the L2 norm.

[0063] In some embodiments, generating K first perturbations that meet predetermined requirements includes repeatedly performing the following operations until K first perturbations are generated: using a random number generation function to randomly generate a random perturbation of dimension n; using the neural network model to increase the dimension of the random perturbation to dimension a to obtain a dimension-increased random perturbation, and based on the fitting function of the sample image, calculating the perturbation rate of the L sample images under the dimension-increased random perturbation; calculating the L2 norm of the random perturbation; in response to determining that the calculated perturbation rate is greater than a first threshold and the calculated L2 norm is less than a second threshold, using the random perturbation as the first perturbation.

[0064] Specifically, in this embodiment, a random number generation function is used to randomly generate a random perturbation ε=(x1, x2, x3..., x n ); using the trained neural network model to increase the data dimension of the random perturbation to a dimension (a>n), to obtain a dimension-increased random perturbation; for L a-dimensional sample images (h1, h2, h3..., h L ), based on the pre-trained neural network classifier model, the perturbation rate of the L sample images under the dimensionality-increasing random perturbation is calculated, wherein the calculation formula of the perturbation rate is: Where P = {h i |F(h i +ε)≠F(h i )}, F(h) is the fitting function for classifying sample images in the classifier, |P| is the number of sample images that meet the P condition in L sample images; calculate the L2 norm of the random perturbation, In response to determining that the calculated disturbance rate R is greater than 80% and the calculated L2 norm is less than the second threshold M, the random disturbance is used as the first disturbance; otherwise, the random disturbance is eliminated. The above steps are repeated until K first disturbances that meet the requirements are obtained, and the K first disturbances are each iteratively optimized to prevent the result from falling into a local optimal solution.

[0065] In some embodiments, the iteratively performing the operation of decreasing the first perturbation in the gradient direction of its objective function until a preset iteration termination condition is satisfied to obtain an optimized perturbation of the first perturbation includes: iteratively performing the following first update operation on the first perturbation until the perturbation rate of the L sample images under the first perturbation is less than or equal to the first threshold, stopping the first update operation, and using the object of the last execution of the first update operation as the second perturbation:

[0066] Calculating a first gradient of the L2 norm of the first perturbation; updating the first perturbation by decreasing the first perturbation in the direction of the calculated first gradient at a preset learning rate to obtain an updated first perturbation; increasing the dimension of the updated first perturbation to a dimension using the neural network model to obtain an increased-dimensional updated first perturbation; and calculating perturbation rates of the L sample images under the increased-dimensional updated first perturbation based on a fitting function of the sample images;

[0067] Iteratively perform the following second updating operation on the second perturbation until the perturbation rate of the L sample images under the second perturbation is greater than the first threshold, so as to obtain an optimized perturbation of the first perturbation:

[0068] Calculate the second gradient of the L2 norm of the second perturbation; reduce the learning rate by a predetermined ratio to update the learning rate; update the second perturbation by decreasing the second perturbation in the direction of the calculated second gradient at the updated learning rate to obtain an updated second perturbation; use the neural network model to increase the dimension of the updated second perturbation to a dimension to obtain a dimension-increased updated second perturbation, and calculate the perturbation rates of the L sample images under the dimension-increased updated second perturbation based on the fitting function of the sample images.

[0069] Specifically, in this embodiment, for each of the first disturbances, the first gradient is calculated based on the L2 norm of the first disturbance. Let the first perturbation be the first gradient g i The first disturbance ε′=ε-g is updated by decreasing the learning rate lr in the direction of i *lr, using a trained neural network model to increase the data dimension of the updated first disturbance ε′ to a dimension to obtain the dimension-updated updated first disturbance; based on the fitting function for classifying sample images in a pre-trained neural network classifier model, calculating the disturbance rate R′ of L sample images under the dimension-updated updated first disturbance, in response to determining that R′ is greater than 80%, continuing to iteratively optimize and update the updated first disturbance ε′ until the disturbance rate of the L sample images under the dimension-updated updated first disturbance is less than or equal to 80%, stopping the first update operation, and taking the executed object of the last execution of the first update operation as the second disturbance ε″.

[0070] Iteratively perform the following second update operation on the second perturbation ε″, including calculating the L2 norm of the second perturbation ε″, calculating the second gradient g based on the L2 norm of the second perturbation ε″ i ′, and reduce the learning rate lr by 50% to update the learning rate, and decrease the second perturbation ε″ in the second gradient direction at a learning rate of lr / 2 to obtain the updated second perturbation ε III =ε″-g i′*lr / 2, using the trained neural network model to update the second disturbance ε III The data dimension is increased to a dimension to obtain the second disturbance of the dimensionality increase update; based on the fitting function for classifying sample images in the pre-trained neural network classifier model, the disturbance rate R″ of the L sample images under the second disturbance of the dimensionality increase update is calculated, and in response to determining that R″ is less than or equal to 80%, the learning rate is further reduced by 50% and updated to lr / 4, and the second disturbance ε″ is decreased at a learning rate of lr / 4 in the second gradient direction until the disturbance rate of the L sample images under the second disturbance of the dimensionality increase update is greater than 80%, the second update operation is stopped, and the disturbance after the second update operation is used as the optimized disturbance. This embodiment judges the updated disturbance rate after each iteration to ensure that the optimization algorithm does not stop prematurely.

[0071] For example, the data dimension of the first perturbation ε is n, the data dimension of the sample image is a, the first perturbation ε is obtained by the first gradient descent at a preset learning rate lr, and the data dimension of ε1 is increased to a dimension using the trained neural network model to obtain ε 1a , if at this time L sample images are in ε 1a If the perturbation rate R1 is greater than 80%, the second gradient descent is performed on ε1 with the preset learning rate lr to obtain ε2. The trained neural network model is used to increase the data dimension of ε2 to a dimension to obtain ε 2a , if at this time L sample images are in ε 2a If the perturbation rate R2 is greater than 80%, the third gradient descent is performed on ε2 with the preset learning rate lr to obtain ε3. The data dimension of ε3 is increased to a dimension using the trained neural network model to obtain ε 3a , if at this time L sample images are in ε 3a If the perturbation rate R3 is less than or equal to 80%, the iteration is stopped and the object of the third gradient descent, ε2, is used as the second perturbation. ε2 is subjected to gradient descent with a reduced learning rate lr / 2 to obtain ε4. The data dimension of ε4 is increased to a dimension using the trained neural network model to obtain ε 4a , if at this time L sample images are in ε 4a If the perturbation rate R4 under ε is greater than 80%, ε4 is used as the optimized perturbation of the first perturbation ε; if the L sample images are in ε 4a If the perturbation rate R4 is less than or equal to 80%, the learning rate is further reduced to lr / 4, and ε2 is gradient-descended at the reduced learning rate lr / 4 to obtain ε5. The trained neural network model is used to increase the data dimension of ε5 to a dimension to obtain ε 5a , if at this time L sample images are in ε 5aIf the perturbation rate R5 is greater than 80%, ε5 is used as the optimized perturbation of the first perturbation ε. Otherwise, the learning rate is further reduced to lr / 8 until the perturbation rate is greater than 80%.

[0072] According to the above method, the optimized perturbations corresponding to the K first perturbations are obtained respectively, the L2 norms of the optimized perturbations are calculated based on the K optimized perturbations respectively, and the optimized perturbation with the smallest L2 norm value is selected as the target perturbation.

[0073] In some embodiments, the trained neural network model includes an autoencoder; the use of the trained neural network model to perform data dimensionality increase processing on the target disturbance includes: using the decoder end of the autoencoder to perform dimensionality increase processing on the target disturbance.

[0074] In some embodiments, the autoencoder is trained using L1 regularized mean squared error as the loss function.

[0075] Specifically, in this embodiment, the form of the autoencoder is:

[0076]

[0077] Where x is the input vector, t is the reduced dimensionality vector output by the hidden layer, and x′ is the result of increasing the dimensionality of the reduced dimensionality vector, which has the same dimension as x. f(x) is the dimensionality reduction function on the encoder side, and g(t) is the dimensionality increase function on the decoder side. In order to make x and x′ as similar as possible, the mean square error E||xx′|| is used. 2 As a loss function, in order to increase the generalization ability of the model, it is necessary to add L1 regularization to the loss function to prevent overfitting, so the final loss function is as follows:

[0078] L(x, x′)=E||xx′|| 2 +λ∑ i |c i |

[0079] where c i is the excitation value of the i-th neuron. The hidden layer activation function uses the rectified linear unit function Relu, and the reconstruction layer activation function uses the analytical function Softplus, which is a smooth approximation of Relu;

[0080] In this embodiment, after obtaining the target perturbation, the target perturbation is subjected to dimensionality increasing processing using the decoder-side dimensionality increasing function g(t) of the autoencoder, and the data dimension of the target perturbation is increased to the same dimension as the sample image to obtain an adversarial sample Trojan.

[0081] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied in a distributed scenario and performed by multiple devices working together. In such a distributed scenario, one of the multiple devices may only perform one or more steps of the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the method.

[0082] It should be noted that the above description is limited to some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in an order different from that described in the above embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0083] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a device for generating an adversarial sample Trojan for a neural network model.

[0084] refer to Figure 2 The device for generating an adversarial sample Trojan of a neural network model includes:

[0085] A generation module 201 is configured to generate K first perturbations that meet predetermined requirements, wherein the predetermined requirements include an objective function requirement and a perturbation rate requirement for L sample images, the dimension of the sample images is a, the dimension of the first perturbation is n, and K, L, a, and n are all positive integers, with a greater than n;

[0086] The optimization module 202 is configured to iteratively perform an operation of decreasing the first perturbation in the gradient direction of the objective function for each of the K first perturbations until a preset iteration end condition is satisfied, so as to obtain an optimized perturbation of the first perturbation;

[0087] A calculation module 203 is configured to calculate a first value of the objective function of each of the K optimized disturbances obtained by the optimization module;

[0088] A determination module 204 is configured to determine the optimized disturbance corresponding to the minimum value among the calculated K first values ​​as a target disturbance;

[0089] The dimension-increasing module 205 is configured to perform data dimension-increasing processing on the target disturbance using a trained neural network model to obtain the adversarial sample Trojan.

[0090] For the convenience of description, the above devices are described as being divided into various modules according to their functions. Of course, when implementing this application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0091] The device of the above embodiment is used to implement the method of generating an adversarial sample Trojan of a neural network model corresponding to any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0092] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, it implements the method for generating an adversarial sample Trojan of a neural network model as described in any of the above embodiments.

[0093] Figure 3 10 is a schematic diagram showing a more specific hardware structure of an electronic device provided in this embodiment. The device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are communicatively connected to each other within the device via the bus 1050.

[0094] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0095] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage devices, dynamic storage devices, etc. The memory 1020 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0096] The input / output interface 1030 is used to connect input / output modules to implement information input and output. The input / output modules can be configured as components within the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and output devices may include a display, speaker, vibrator, indicator light, etc.

[0097] The communication interface 1040 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via a wired method (such as USB, network cable, etc.) or a wireless method (such as mobile network, WiFi, Bluetooth, etc.).

[0098] The bus 1050 comprises a path for transmitting information between the various components of the device (eg, the processor 1010 , the memory 1020 , the input / output interface 1030 , and the communication interface 1040 ).

[0099] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040, and the bus 1050, in a specific implementation, the device may also include other components necessary for normal operation. In addition, it will be understood by those skilled in the art that the above device may only include the components necessary to implement the embodiments of this specification, and does not necessarily include all the components shown in the figure.

[0100] The electronic device of the above embodiment is used to implement the method of generating an adversarial sample Trojan of a neural network model corresponding to any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0101] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute the method of generating an adversarial sample Trojan of a neural network model as described in any of the above embodiments.

[0102] The computer-readable media of this embodiment include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0103] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the method of generating an adversarial sample Trojan of a neural network model as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0104] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. Within the scope of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0105] In addition, for simplicity of description and discussion, and in order not to make the embodiment of the application difficult to understand, the known power supply / ground connection with integrated circuit (IC) chip and other components may or may not be shown in the accompanying drawings provided. In addition, the device can be shown in the form of a block diagram to avoid making the embodiment of the application difficult to understand, and this also takes into account the following fact, that is, the details of the embodiment of these block diagram devices are highly dependent on the platform to be implemented in the embodiment of the application (that is, these details should be fully within the scope of understanding of those skilled in the art). When specific details (for example, circuit) are set forth to describe exemplary embodiments of the application, it will be apparent to those skilled in the art that the embodiment of the application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0106] Although the present invention has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those skilled in the art based on the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may utilize the embodiments discussed.

[0107] The embodiments of the present application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of this application.

Claims

1. A method for generating an adversarial Trojan horse for a neural network model, characterized in that: The adversarial Trojan is an image with perturbations added to it. The method includes: Generate K first perturbations that meet predetermined requirements, wherein the predetermined requirements include objective function requirements and perturbation rate requirements for L sample images, the dimension of the sample images is a, the dimension of the first perturbation is n, K, L, a, and n are all positive integers, and a is greater than n; For each of the K first perturbations, iteratively performing an operation of decreasing the first perturbation in the gradient direction of its objective function until a preset iteration end condition is satisfied, so as to obtain an optimized perturbation of the first perturbation; Calculating a first value of the objective function of each of the K optimized disturbances obtained; Determine the optimized disturbance corresponding to the minimum value among the calculated K first values ​​as the target disturbance; The trained neural network model is used to perform data dimensionality increase processing on the target disturbance to obtain the adversarial sample Trojan.

2. The method according to claim 1, characterized in that The objective function includes an L2 norm.

3. The method according to claim 2, characterized in that Generating K first disturbances that meet predetermined requirements includes repeatedly performing the following operations until K first disturbances are generated: Use the random number generation function to randomly generate a random perturbation of dimension n; Using the neural network model to increase the dimension of the random perturbation to a dimension to obtain a-dimensional random perturbation, and calculating the perturbation rates of the L sample images under the increased-dimensional random perturbation based on the fitting function of the sample images; Calculating the L2 norm of the random perturbation; In response to determining that the calculated perturbation rate is greater than a first threshold and the calculated L2 norm is less than a second threshold, the random perturbation is used as the first perturbation.

4. The method according to claim 3, characterized in that The iteratively performing an operation of decreasing the first disturbance in the gradient direction of its objective function until a preset iteration end condition is satisfied to obtain an optimized disturbance of the first disturbance includes: Iteratively perform the following first update operation on the first perturbation until the perturbation rate of the L sample images under the first perturbation is less than or equal to the first threshold, stop the first update operation, and use the object of the last execution of the first update operation as the second perturbation: Calculating a first gradient of the L2 norm of the first perturbation; Updating the first disturbance by decreasing the first disturbance in the direction of the calculated first gradient at a preset learning rate to obtain an updated first disturbance; Using the neural network model, the dimension of the updated first disturbance is increased to a dimension to obtain the increased-dimensional updated first disturbance, and based on the fitting function of the sample images, the disturbance rates of the L sample images under the increased-dimensional updated first disturbance are calculated; Iteratively perform the following second update operation on the second perturbation until the perturbation rate of the L sample images under the second perturbation is greater than the first threshold, so as to obtain an optimized perturbation of the second perturbation: Calculating a second gradient of the L2 norm of the second perturbation; reducing the learning rate by a predetermined ratio to update the learning rate; Updating the second perturbation by decreasing the second perturbation in the direction of the calculated second gradient at the updated learning rate to obtain an updated second perturbation; The neural network model is used to increase the dimension of the updated second disturbance to a dimension to obtain the increased-dimensional updated second disturbance, and based on the fitting function of the sample image, the disturbance rates of the L sample images under the increased-dimensional updated second disturbance are calculated.

5. The method according to claim 4, characterized in that The predetermined ratio is 50%.

6. The method according to any one of claims 3 to 5, characterized in that The first threshold is 80%.

7. The method according to any one of claims 1 to 5, characterized in that The trained neural network model includes an autoencoder; The using of the trained neural network model to perform data dimensionality increase processing on the target disturbance includes: using the decoder end of the autoencoder to perform dimensionality increase processing on the target disturbance.

8. The method according to claim 7, characterized in that The autoencoder is trained using L1 regularized mean squared error as the loss function.

9. A device for generating an adversarial Trojan horse for a neural network model, characterized in that: The adversarial sample Trojan is an image with disturbance added, and the device includes: a generation module configured to generate K first perturbations that meet predetermined requirements, wherein the predetermined requirements include objective function requirements and perturbation rate requirements for L sample images, the dimension of the sample images is a, the dimension of the first perturbation is n, and K, L, a, and n are all positive integers, with a greater than n; an optimization module configured to iteratively perform, for each of the K first perturbations, an operation of decreasing the first perturbation in a gradient direction of its objective function until a preset iteration end condition is satisfied, so as to obtain an optimized perturbation of the first perturbation; a calculation module configured to calculate a first value of the objective function of each of the K optimized disturbances obtained by the optimization module; a determination module configured to determine the optimized disturbance corresponding to the minimum value among the calculated K first values ​​as a target disturbance; The dimension-increasing module is configured to use a trained neural network model to perform data dimension-increasing processing on the target disturbance to obtain the adversarial sample Trojan.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein: The processor implements the method according to any one of claims 1 to 8 when executing the computer program.

Images