Counterattack method and device based on three-dimensional dynamic interactive scene

Abstract

The present invention discloses a method and device for adversarial attack based on a three-dimensional dynamic interactive scene. In the present invention, firstly, the attention mechanism is combined to select the frame to be attacked from the historical frame of the intelligent body; for the environmental objects of the frame to be attacked, a guideable rendering process is introduced to generate adversarial samples; finally, the adversarial samples are used to conduct adversarial attacks on the intelligent body, thereby avoiding the problems in traditional static scene attacks and making the attack effect in the three-dimensional dynamic interactive scene far superior to the traditional attack method.

Description

[0001] This invention application is a divisional application of the invention application entitled “Anti-attack method and device based on three-dimensional dynamic interactive scene” filed on May 11, 2020 and application number 202010394266.4. Technical Field

[0002] The present invention relates to a counter-attack method based on a three-dimensional dynamic interactive scene, and also relates to a corresponding counter-attack device, belonging to the technical field of machine learning. Background Art

[0003] There has long been a technical challenge in the field of artificial intelligence: how to create an intelligent agent that can perceive the surrounding environment, conduct human-computer interaction, and complete a given task through a specific sequence of actions. In recent years, deep learning has achieved outstanding results in many challenging fields such as computer vision and natural language processing, and it has also become a core module for building environmental interactive intelligent agents. Researchers have combined deep learning models in the fields of computer vision and natural language processing to try to build an intelligent agent that meets the above requirements.

[0004] Currently, researchers have conducted a lot of research on interactive agent navigation and question-answering tasks using deep learning. In this task, agents are randomly placed in a virtual environment. Given a question described in natural language, they need to navigate intelligently in the environment from a first-person perspective, reach the target location, and complete the specific task given. Researchers have enabled agents to achieve good performance in virtual environments by allowing them to learn active perception, common sense reasoning, and score allocation in virtual environments.

[0005] On the other hand, adversarial examples have gradually become a research hotspot in the field of deep learning. Since the decision-making of environmental interactive agents relies on deep learning models, their robustness is also threatened by adversarial examples. Therefore, in scenarios where adversarial examples exist, there are great risks in the security and robustness of agents. For three-dimensional (3D) scenarios, researchers have proposed many methods to attack deep learning models. However, most of those attack methods only focus on static space problems.

[0006] The interactive agent navigation and question-answering task is different from the deep learning task of static scenes. Its special feature is that the agent can move, interact, navigate and communicate with humans in the virtual environment. If it is treated the same as the traditional static scene problem, the agent will easily ignore the noise in the environment due to the change of noise content caused by factors such as the angle and distance at which the agent observes the object, which leads to poor attack results. Summary of the invention

[0007] The primary technical problem to be solved by the present invention is to provide a method for countering attacks based on a three-dimensional dynamic interactive scene.

[0008] Another technical problem to be solved by the present invention is to provide an anti-attack device based on a three-dimensional dynamic interactive scene.

[0009] In order to achieve the above-mentioned purpose of the invention, the present invention adopts the following technical solutions:

[0010] According to a first aspect of an embodiment of the present invention, a method for countering attacks based on a three-dimensional dynamic interactive scene is provided, comprising the following steps:

[0011] Combined with the attention mechanism, select the frame to be attacked from the historical frames of the agent;

[0012] For the environmental objects in the frame to be attacked, a guideable rendering process is introduced to generate adversarial samples;

[0013] Use adversarial examples to conduct adversarial attacks on intelligent agents.

[0014] Preferably, the attention mechanism is combined to select the frame to be attacked from the historical frames of the intelligent agent, specifically including:

[0015] Calculate the contribution of each historical frame to the agent's decision-making;

[0016] Normalize the decision contribution of each historical frame to obtain the weight of each historical frame;

[0017] The weights are sorted from roughly small to roughly large, and the first M (M>0) historical frames are designated as frames to be attacked.

[0018] Preferably, the calculation formula of the decision contribution is:

[0019]

[0020] In formula (1), y is the correct category, Z is the feature of the current historical frame at a specific layer of the neural network, Represents the value of the ith row and jth column of the nth neuron in the current network feature. u and v represent the height and width of the feature of a specific layer of the neural network, respectively.

[0021] Preferably, the weight calculation formula is:

[0022]

[0023] In formula (2), μ is the mean of the weights of each historical frame, σ is the variance of each historical frame, and ∈ is a very small non-zero value to avoid the situation where the denominator is zero.

[0024] The preferred ones include:

[0025] Obtaining the three-dimensional properties of environmental objects appearing in the frame to be attacked;

[0026] Construct a differentiable renderer, use the gradient descent algorithm, and combine the agent's loss function to optimize the three-dimensional anti-noise;

[0027] Three-dimensional adversarial noise is added to the three-dimensional attributes of the environmental objects appearing in the frame to be attacked, thereby forming corresponding adversarial samples.

[0028] Preferably, after adding three-dimensional anti-noise to the three-dimensional attributes of the environmental objects appearing in the frame to be attacked, the method further includes:

[0029] When the agent’s loss function reaches the preset condition, stop generating adversarial samples.

[0030] Preferably, the loss function is expressed as:

[0031]

[0032] In formula (3), x m represents the mth environmental object in the current frame to be attacked, y is the true label, λ is used to weigh the attack effect and visual effect, x adv According to x m Generated adversarial environment objects, is the decision model of the intelligent agent, is the bootable renderer, c is the current environment variable, C is the distribution of environment variables, E c～C Indicates that the environment variable corresponds to the expectation.

[0033] Preferably, the expression of the adversarial sample is:

[0034]

[0035] In formula (4), φ m represents the 3D attributes of the mth environmental object in the current historical frame, S k represents all objects in the current frame to be attacked, K is the total number of objects in the current historical frame, and the Φ() function represents the environmental objects selected from the K objects in the current historical frame. is the weight of the current history frame.

[0036] Preferably, the attack effect loss function is defined as:

[0037]

[0038] The visual effect loss function is defined as:

[0039]

[0040] Among them, P represents the probability of the agent's decision, and S represents the frame to be attacked.

[0041] According to a second aspect of an embodiment of the present invention, there is provided an anti-attack device based on a three-dimensional dynamic interactive scene, comprising a processor and a memory, wherein the processor reads a computer program in the memory to perform the following operations:

[0042] Combined with the attention mechanism, select the frame to be attacked from the historical frames of the agent;

[0043] For the environmental objects in the frame to be attacked, a guideable rendering process is introduced to generate adversarial samples;

[0044] Use adversarial examples to conduct adversarial attacks on intelligent agents.

[0045] An embodiment of the present invention provides an adversarial attack method based on a three-dimensional dynamic interactive scene. In combination with the attention mechanism, the frame to be attacked is selected from the historical frames of the intelligent agent, so as to determine the optimal attack position; then a guideable rendering process is introduced to generate adversarial samples, and finally the adversarial samples are used to perform an adversarial attack, thereby avoiding the problems that occur in traditional static scene attacks and making the attack effect in the three-dimensional dynamic interactive scene far superior to the traditional attack method. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A flowchart of the anti-attack method provided by the present invention;

[0047] Figure 2 This is a structural example diagram of the anti-attack device provided by the present invention. DETAILED DESCRIPTION

[0048] The technical content of the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0049] At present, deep learning has achieved outstanding results in many challenging fields such as computer vision and natural language processing. In the field of intelligent agent-related research, deep learning is an indispensable tool. For example, the core modules of environmental interactive intelligent agent navigation and question-answering tasks are composed of deep learning models. In this task, the intelligent agent is randomly placed in a simulated environment. Given a question described in natural language, it needs to perform intelligent navigation in the environment based on the first-person perspective, reach the target location, and complete the specific task given. Although the intelligent agent can achieve good performance in a virtual environment, the virtual environment often lacks the inevitable noise in the real environment, such as adversarial samples.

[0050] Adversarial samples are samples that are slightly adjusted from the original samples. They have no effect on human cognition and object recognition, but they can mislead deep neural networks to make wrong decisions, which poses a serious security threat to the application of machine learning in the digital and physical worlds in practice. Since the decision-making of three-dimensional dynamic interactive agents relies on deep learning models, their robustness is also threatened by adversarial samples. Adversarial samples refer to a very finely designed noise that is indistinguishable to the human eye, but devastating to deep learning models:

[0051] F θ (x adv )≠y st||xx adv ||<ε

[0052] Among them, x represents a normal sample, x adv represents adversarial examples. x and x adv The images are visually similar and the distance is less than ε, but the deep learning model F will misclassify them.

[0053] like Figure 1 As shown, the anti-attack method based on a three-dimensional dynamic interactive scene provided by an embodiment of the present invention includes the following steps:

[0054] 101. Combine the attention mechanism to select the frame to be attacked from the historical frames of the intelligent agent;

[0055] Specifically:

[0056] 1011. Calculate the contribution of each historical frame to the decision-making of the intelligent agent;

[0057] The calculation formula of the decision contribution is:

[0058]

[0059] In formula (1), y is the correct category, Z is the feature of the current historical frame at a specific layer of the neural network, Represents the value of the ith row and jth column of the nth neuron in the current network feature. u and v represent the height and width of the feature of a specific layer of the neural network, respectively.

[0060] 1012. Normalize the decision contribution of each historical frame to obtain the weight of each historical frame;

[0061] The calculation formula of the weight is:

[0062]

[0063] In formula (2), μ is the mean of the weights of each historical frame, σ is the variance of each historical frame, and ∈ is a very small non-zero value to avoid the situation where the denominator is zero.

[0064] 1013. Sort the weights from roughly small to large, and designate the first M (M>0) historical frames as frames to be attacked.

[0065] In existing methods, the 3D physical properties of objects are often modified in static scenes, or in non-interactive scenes, operations are performed on information obtained from static perspectives. If the particularities of 3D dynamic interactive agents in time and space are ignored and treated the same as traditional static scene problems, then due to the changes in noise content caused by factors such as the angle and distance at which the agent observes the object, the agent will easily ignore the noise in the environment, resulting in poor attack results.

[0066] In the embodiment of the present invention, when making the current decision, the intelligent agent in the three-dimensional dynamic interactive scene not only relies on the current frame, but also relies on the historical frames that it has seen before. Therefore, we need to find the most important frame for the current decision in the historical frames that the intelligent agent has passed through to attack. The assessment of "most important" needs to be realized through the decision degree or weight of each historical frame. Assume that the frames to be attacked are the 3rd, 6th and 9th frames in the historical frames.

[0067] 102. For the environmental objects in the frame to be attacked, introduce a guideable rendering process to generate adversarial samples;

[0068] Specifically:

[0069] 1021. Obtain the three-dimensional properties of the environmental objects appearing in the frame to be attacked;

[0070] In the embodiment of the present invention, the environmental objects are all objects appearing in the frame to be attacked; the three-dimensional attributes are the physical attributes of each object, such as shape, texture, etc. The physical attributes of all objects appearing in the 3rd frame, the 6th frame and the 9th frame are obtained.

[0071] 1022. Construct a derivative renderer, use the gradient descent algorithm, and combine the loss function of the intelligent agent to optimize the three-dimensional anti-noise;

[0072] In the embodiment of the present invention, the gradient descent algorithm is simple and convenient, and can quickly optimize the three-dimensional properties of environmental objects. Since the agent sees a 2D image, when we change the three-dimensional properties of environmental objects, we need to render the three-dimensional objects into 2D images through a guideable renderer.

[0073] The expression of the loss function is:

[0074]

[0075] In formula (3), x mrepresents the mth environmental object in the current frame to be attacked, y is the true label, λ is used to weigh the attack effect and visual effect, x adv According to x m Generated adversarial environment objects, is the decision model of the intelligent agent, is the bootable renderer, c is the current environment variable, C is the distribution of environment variables, E c～C Indicates that the environment variable corresponds to the expectation.

[0076] An environmental object with adversarial aggressiveness is an environmental object with added noise. Assume that the pattern of the table in the third frame is modified, then the modified table in the third frame is an environmental object with adversarial aggressiveness.

[0077] The attack effect loss function is defined as:

[0078]

[0079] The visual effect loss function is defined as:

[0080]

[0081] Among them, P represents the probability of the agent's decision, and S represents the frame to be attacked.

[0082] The role of the attack effect loss function is to mislead the model of the 3D dynamic interactive scene agent to make wrong decisions. The role of the visual effect loss function is to ensure that the attributes of the attack sample are similar to those before the attack, thereby ensuring that the visual effect of the adversarial sample remains basically unchanged before and after the attack.

[0083] 1023. Add three-dimensional adversarial noise to the three-dimensional attributes of the environmental objects appearing in the frame to be attacked, thereby forming a corresponding adversarial sample;

[0084] The expression of the adversarial sample is:

[0085]

[0086] In formula (4), φ m represents the 3D attributes of the mth environmental object in the current historical frame, S k represents all objects in the current frame to be attacked, K is the total number of objects in the current historical frame, and the Φ() function represents the environmental objects selected from the K objects in the current historical frame. is the weight of the current history frame.

[0087] 1024. When the loss function of the agent reaches the preset condition, stop generating adversarial samples.

[0088] In the embodiment of the present invention, the three-dimensional attributes are changed during the optimization process to minimize the loss function. The loss function is calculated through an iterative algorithm, and when the number of iterations reaches a predetermined threshold, or the value of the added noise reaches a preset upper limit, it is necessary to stop generating adversarial samples.

[0089] 103. Use adversarial samples to conduct adversarial attacks on intelligent agents.

[0090] In an embodiment of the present invention, the "most important" frame in the historical frame for the intelligent agent to make a decision is used as the frame to be attacked. The frame to be attacked represents the best attack position, and the attack is optimized for each object in the frame to be attacked, thereby ensuring the attack effect to the greatest extent. In the process of optimizing the attack, the three-dimensional properties of the environmental objects are changed by the derivative renderer and the gradient descent algorithm, so as to increase the three-dimensional adversarial noise for the environmental objects of the frame to be attacked; and the three-dimensional adversarial noise is constrained by the loss function. In this way, the problems in the traditional static scene attack are avoided, and the attack effect in the three-dimensional dynamic interactive scene is far better than the traditional attack method.

[0091] Furthermore, the present invention also provides an anti-attack device based on a three-dimensional dynamic interactive scene, such as Figure 2 As shown, it includes a processor 22 and a memory 21, and may further include a communication component, a sensor component, a power component, a multimedia component and an input / output interface according to actual needs. Among them, the memory, communication component, sensor component, power component, multimedia component and input / output interface are all connected to the processor 22. As mentioned above, the memory 21 in the node device can be a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, etc., and the processor can be a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a digital signal processing (DSP) chip, etc. Other communication components, sensor components, power components, multimedia components, etc. can be implemented using common components in existing smart phones, which will not be described in detail here.

[0092] On the other hand, in the above-mentioned anti-attack device based on three-dimensional dynamic interactive scene, the processor 22 reads the computer program in the memory 21 to perform the following operations:

[0093] Combined with the attention mechanism, select the frame to be attacked from the historical frames of the agent;

[0094] For the environmental objects in the frame to be attacked, a guideable rendering process is introduced to generate adversarial samples;

[0095] Use adversarial examples to conduct adversarial attacks on intelligent agents.

[0096] In the embodiment of the present invention, the most important frame for the intelligent agent to make a decision is found as the frame to be attacked; then the three-dimensional attributes of each environmental object in the frame to be attacked are attacked, so as to achieve the purpose of deceiving the intelligent agent. Furthermore, the robustness and stability of the intelligent agent in the three-dimensional dynamic interaction scene are increased.

[0097] The above is a detailed description of the anti-attack method and device based on the three-dimensional dynamic interactive scene provided by the present invention. For those skilled in the art, any obvious changes made to it without departing from the essence of the present invention will constitute an infringement of the patent right of the present invention and will bear the corresponding legal liability.

Claims

1. A counterattack method based on a three-dimensional dynamic interactive scene, characterized in that The steps include: Combined with the attention mechanism, the frames to be attacked are selected from the historical frames of the intelligent agent; the contribution of each historical frame to the decision of the intelligent agent is calculated; the decision contribution of each historical frame is normalized to obtain the weight of each historical frame; the weights are sorted from large to small, and the first M historical frames are designated as the frames to be attacked, where M>0; Acquire the three-dimensional properties of the environmental objects appearing in the frame to be attacked; construct a derivable renderer, change the three-dimensional properties of the environmental objects through the derivable renderer and the gradient descent algorithm in the process of optimizing the attack, add three-dimensional adversarial noise to the three-dimensional properties of the environmental objects appearing in the frame to be attacked, and constrain the three-dimensional adversarial noise through the loss function of the intelligent agent, so as to generate adversarial samples; wherein, when the three-dimensional properties of the environmental objects are changed, the three-dimensional objects are rendered into 2D pictures through the derivable renderer; The adversarial sample is used to conduct adversarial attacks on the intelligent agent.

2. The anti-attack method according to claim 1, characterized in that: The calculation formula of the decision contribution is: Among them, y is the correct category, Z is the feature of the current historical frame at a specific layer of the neural network, Represents the value of the ith row and jth column of the nth neuron in the current network feature. u and v represent the height and width of the feature of a specific layer of the neural network, respectively.

3. The anti-attack method according to claim 2, characterized in that: The calculation formula of the weight is: in, is the mean value of the weights of each historical frame, is the variance of each historical frame, is a very small non-zero value used to avoid the case where the denominator is zero.

4. The method for resisting attacks as claimed in claim 1, characterized in that After adding the three-dimensional adversarial noise to the three-dimensional attributes of the environmental objects appearing in the frame to be attacked, it also includes: When the agent’s loss function reaches the preset condition, stop generating adversarial samples.

5. The method for resisting attacks according to claim 4, characterized in that: The expression of the loss function is: in, represents the mth environmental object in the current frame to be attacked, y is the true label, Used to balance attack effect and visual effect. Indicates based on Generated adversarial environment objects, is the decision model of the intelligent agent, is the bootable renderer, c is the current environment variable, C is the distribution of environment variables, Indicates that the environment variables correspond to expectations, is the attack effect loss function, is the visual effect loss function.

6. The method for resisting attacks according to claim 5, characterized in that: The expression of the adversarial sample is: in, Represents the three-dimensional properties of the mth environmental object in the current historical frame, Represents all objects in the current frame to be attacked, is the total number of objects in the current history frame, The function represents the The environmental objects selected from the objects, is the weight of the current history frame.

7. The method for resisting attacks according to claim 6, characterized in that: The attack effect loss function for: The visual effect loss function for: Among them, P represents the probability of the agent's decision, and S represents the frame to be attacked.

8. A device for countering attacks based on a three-dimensional dynamic interactive scene, characterized in that The invention comprises a processor and a memory, wherein the processor reads a computer program in the memory to execute the anti-attack method according to any one of claims 1 to 7.

Images