An infrared adversarial patch generation method and device based on reinforcement learning

Abstract

The present application relates to a kind of infrared countermeasure patch generation method and device based on reinforcement learning, by the decision network of reinforcement learning is trained for multiple rounds to obtain the decision network of trained infrared countermeasure patch that can obtain suitable pasting area and suitable gray scale in infrared image, and gray scale block is pasted, wherein, multiple time steps are pasted gray scale block in each round training process, the action of next time step is obtained according to the coordinates and gray scale of last time step gray scale block when pasting gray scale block. Finally, infrared countermeasure patch can be obtained according to the decision network of trained infrared countermeasure patch pasting gray scale block in patch area.The structure and texture of infrared countermeasure patch generated by the present application are simple, operability is strong, and attack effectiveness is high.

Description

Technical Field

[0001] This invention relates to the field of computer vision community technology, and in particular to a method and apparatus for generating infrared adversarial patches based on reinforcement learning. Background Technology

[0002] Most object detectors are trained using deep neural networks (DNNs). These trained DNNs are susceptible to slight perturbations. Adding invisible noise to images, or adding attributes such as color, texture, and camouflage, can make it difficult for DNNs to identify target images. Currently, the robustness of object detectors is typically improved by generating adversarial examples through adversarial training, such as adding attributes like color, texture, and camouflage to the images.

[0003] Patch-based adversarial attacks are defined as attacks that use carefully crafted adversarial patches to deceive DNNs and are often applied to physical attacks. Patch-based adversarial attack methods replace local regions of the threat image with patches, without considering perturbation constraints. Current patch-based adversarial attack methods mainly focus on designing special structures and textures for visible light adversarial patches. These patches are mostly color images from visible light scenes, resulting in adversarial patches with high detail, making adversarial attack effects easier to achieve. However, infrared grayscale images, unlike visible light images, lack a large amount of feature information, making visible light adversarial attack methods unsuitable for infrared.

[0004] In existing technologies, the following two methods are commonly used to generate adversarial patches in infrared scenes: one is to use complex infrared textures similar to visible light adversarial patches, and the other is to use traditional heuristic algorithms to generate irregularly shaped patches or scattered patch blocks, and use various materials to simulate patch grayscale in the physical world.

[0005] However, using infrared countermeasures patches with complex textures is difficult to achieve in physical infrared imaging, resulting in high costs and poor practical operability. Furthermore, infrared countermeasures patches obtained using heuristic algorithms require long training times, and the physical world may not perfectly match the irregular patch shapes trained on them, or the patch positions may be scattered, the patch grayscale may be uniform, and the effectiveness of countermeasures is low. Summary of the Invention

[0006] Therefore, it is necessary to provide a method and apparatus for generating infrared countermeasures patches based on reinforcement learning to address the above-mentioned technical problems. The generated infrared countermeasures patches have simple structures and textures, are highly operable, and have high attack effectiveness.

[0007] In a first aspect, the present invention provides an infrared adversarial patch generation method based on reinforcement learning, comprising the following steps:

[0008] Initialize the patch to obtain an initial process sample;

[0009] The coordinates and grayscale of the initial grayscale block are input as state information into the decision network of reinforcement learning to obtain the action to be executed at the next time step;

[0010] Update the state information based on the action to be executed in the next time step, and obtain the process sample of the next time step;

[0011] Determine if the number of time steps exceeds the preset time step threshold. If yes, proceed to the next step. Otherwise, input the updated state information into the reinforcement learning decision network to obtain the updated action to be executed in the next time step, and return to the step of updating the state information according to the action to be executed in the next time step.

[0012] The total reward value corresponding to the reinforcement learning decision network is calculated based on all process samples;

[0013] Establish a convergence curve based on the total reward value, determine whether the convergence curve has converged, if so, use the reinforcement learning decision network corresponding to the convergence curve as the decision network of the trained infrared adversarial patch and execute the next step; otherwise, update the reinforcement learning decision network and return to the initialization patch step.

[0014] Infrared countermeasures patches are obtained by pasting grayscale blocks in the patch area based on the decision network of the trained infrared countermeasures patch.

[0015] In one embodiment, the next time step action is to select the coordinates and grayscale of the grayscale block to be pasted in the patch area in the next time step.

[0016] In one embodiment, initializing the patch to obtain an initial process sample includes:

[0017] Identify the patch area;

[0018] Build initial grayscale blocks in the patch area;

[0019] The initial grayscale block is pasted onto the coordinates of the initial grayscale block in the patch area to obtain the initial process sample.

[0020] In one embodiment, constructing the initial grayscale block in the patch area includes:

[0021] Choose any coordinate in the patch area as the coordinate of the initial grayscale block;

[0022] Set the grayscale and size of the initial grayscale block.

[0023] In one embodiment, updating the state information based on the action to be performed at the next time step includes:

[0024] In the patch area, select a coordinate as the coordinate of the grayscale block for the next time step based on the action to be executed in the next time step;

[0025] The grayscale and size of the grayscale block in the next time step are set according to the action to be executed in the next time step, wherein the size of the grayscale block in the next time step is the same as the size of the initial grayscale block.

[0026] In one embodiment, the total reward value for the reinforcement learning decision network, calculated based on all process samples, includes:

[0027] All process samples are input into the target detector to obtain the target confidence score for each process sample;

[0028] Calculate the difference in target confidence scores for process samples at all adjacent time steps;

[0029] The reward value for all differences is calculated using the reward function;

[0030] The total reward value is obtained by summing the reward values ​​corresponding to all the differences.

[0031] In one embodiment, the reinforcement learning decision network is updated based on the experience pool;

[0032] The experience pool includes the experience at each time step. The format of the experience is <status information, action performed, reward value, update status information>.

[0033] Secondly, the present invention also provides an apparatus for generating infrared countermeasures patches based on reinforcement learning, comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-mentioned infrared countermeasures patch generation method based on reinforcement learning.

[0034] The beneficial effects of this invention are as follows: This invention uses grayscale blocks as the basic unit of infrared countermeasures patches. The infrared countermeasures patches are formed by sequentially piecing together grayscale blocks, simplifying the structure and texture of the patches and enhancing their operability. Furthermore, the patch unit, i.e., the grayscale block, is determined by a reinforcement learning decision network of a trained infrared countermeasures patch, which identifies the appropriate placement area and grayscale level in the infrared image and then applies the grayscale blocks. This reduces the confidence level of the target in the infrared image, achieving a better countermeasures effect. Attached Figure Description

[0035] Figure 1 This is a flowchart illustrating an infrared adversarial patch generation method based on reinforcement learning provided in an embodiment of the present invention.

[0036] Figure 2 This is a schematic diagram of an infrared adversarial patch generation framework based on reinforcement learning provided in an embodiment of the present invention;

[0037] Figure 3 This is a flowchart illustrating the initialization patch provided in an embodiment of the present invention;

[0038] Figure 4 This is a schematic diagram of an intelligent agent performing actions provided in an embodiment of the present invention;

[0039] Figure 5 This is a schematic diagram of the process for calculating the total reward value of the reinforcement learning decision network based on all process samples provided in this embodiment of the invention;

[0040] Figure 6 These are images showing the target detector's detection results after performing adversarial attacks using different adversarial patches. Figure 6 (a) is a graph showing the target detector's detection results after performing an adversarial attack using the patch Bulb obtained with a light bulb. Figure 6 (b) is a graph showing the target detector's detection results after performing an adversarial attack using a patch QR code similar to a QR code. Figure 6 (c) is a graph showing the target detector's detection results after the patched AIP is executed to perform an adversarial attack, obtained through aggregation regularization. Figure 6 (d) is a graph showing the target detector's detection results after performing an adversarial attack using the distributed patch AdvIB obtained through heuristic search. Figure 6 (e) is the target detector detection result after the patched HCB is subjected to an adversarial attack using the particle swarm optimization algorithm. Figure 6 (f) is a diagram showing the target detector detection results after the infrared countermeasure patch generated by the method of the present invention performs a countermeasure attack. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0042] In infrared target detection, an infrared image dataset needs to be established first. This dataset includes several labeled training and validation images. Images containing target labels are selected for adversarial patch generation training. Then, the images from the dataset are input into a target detector, and the detector output is obtained, yielding the confidence level of the target in the image. The infrared adversarial patch generated by the reinforcement learning-based infrared adversarial patch generation method of this invention makes it difficult for the infrared target detector to detect targets in infrared images.

[0043] This invention addresses infrared target detection by considering a dataset containing clean input samples X and labels Y. A pre-trained target detection model f(·) for thermal infrared images can predict labels that match the true labels Y. Each image in the samples contains several "target labels," where the label Y is determined by the position Y' of the target object's bounding box. p Confidence score of the object C = f c Composed of (·). The generated adversarial examples are usually represented as X. adv =X+δ, by adding an adversarial perturbation δ to the image, we can deceive a trained object detector, lowering the object confidence score so that it cannot recognize the object, even if f(X) adv )≠Y.

[0044] This invention proposes an infrared adversarial patch generation method based on reinforcement learning. By patching or replacing local regions of the original image with infrared adversarial patches P to create perturbations, the adversarial examples are then generated as X. adv =X+P. The training set consists of (X) i ,Y i The sequence consists of i = 1, 2, ..., N, where X... i Y represents the input training sample. i This represents the corresponding real-world label, where i represents the index of the target in the training set. The target description can be as follows:

[0045]

[0046] Where i represents the sequence number of the target to be detected in the input image, and the smaller the maximum confidence score of the target, the better.

[0047] In one embodiment, such as Figure 1 and Figure 2 As shown, Figure 1 This is one of the flowcharts illustrating the infrared adversarial patch generation method based on reinforcement learning provided in this embodiment of the invention. Figure 2 This is a schematic diagram of the infrared adversarial patch generation framework based on reinforcement learning of the present invention. The infrared adversarial patch generation method based on reinforcement learning includes the following steps:

[0048] S101. Initialize the patch and obtain the initial process sample.

[0049] Specifically, such as Figure 3 As shown, the initialization process for the patch, obtaining an initial sample, includes:

[0050] S301. Determine the patch area.

[0051] S302. Construct an initial grayscale block in the patch area. Specifically, constructing an initial grayscale block in the patch area includes: arbitrarily selecting a coordinate in the patch area as the coordinate of the initial grayscale block; and setting the grayscale and size of the initial grayscale block.

[0052] In this embodiment, the grayscale block is composed of B. i The representation is based on the basic unit, where the set of positions for each grayscale block is W = {(x1,y1),(x2,y2),…,(x...}. m ,y m} represents the coordinates of the top-left vertex of each patch unit, i.e., the grayscale block, where m represents the upper limit of the total number of patch units. All grayscale blocks should be confined within a reasonable area of ​​the target bounding box. The grayscale value of each grayscale block is represented by G. Infrared images have only one monochrome channel from 0 to 255, so the grayscale values ​​are normalized to 0 to 1. Since it is difficult to accurately map the grayscale values ​​of infrared imaging using a certain material in physical space, several discrete grayscale values ​​are used as representatives: G = {g1, g2, ... g}. n Furthermore, it cannot generate grayscale values ​​of 0 and 255. To be more realistic, g1 = 0.1, g n =0.9.

[0053] S203. Paste the initial grayscale block to the initial grayscale block coordinates of the patch area to obtain the initial process sample.

[0054] S102. Input the coordinates and grayscale of the initial grayscale block as state information into the decision network of reinforcement learning to obtain the action to be executed at the next time step.

[0055] It should be noted that since the process of forming infrared countermeasures patches is discrete, each step of grayscale block placement may cause changes in the output of the target detector. By learning the most effective grayscale block placement position, a patch that can produce countermeasures effect can be finally formed. Therefore, this embodiment uses a reinforcement learning decision network to learn the most effective grayscale block placement position.

[0056] In the process of combining grayscale blocks, grayscale blocks can be regarded as process patches. The grayscale block combination process can be formalized as follows: Process patch Each change depends only on the state of the current time step and the action to be performed in the next time step.

[0057] Specifically, the action performed in the next time step is to select the coordinates and grayscale of the grayscale block to be pasted in the patch area in the next time step. The grayscale of grayscale blocks in two adjacent time steps is different.

[0058] S103. Update the state information according to the action to be executed in the next time step, and obtain the process sample of the next time step.

[0059] Specifically, updating the status information based on the action executed in the next time step includes: selecting a coordinate in the patch area as the coordinate of the grayscale block in the next time step based on the action executed in the next time step; setting the grayscale and size of the grayscale block in the next time step based on the action executed in the next time step, wherein the size of the grayscale block in the next time step is the same as the size of the initial grayscale block.

[0060] S104. Determine whether the number of time steps exceeds the preset time step threshold. If yes, execute the next step. Otherwise, input the updated state information into the reinforcement learning decision network to obtain the updated next time step execution action, and return to the step of updating the state information according to the next time step execution action.

[0061] S105. Calculate the total reward value corresponding to the decision network of reinforcement learning based on all process samples.

[0062] S106. Establish a convergence curve based on the total reward value, and determine whether the convergence curve has converged. If it has, use the reinforcement learning decision network corresponding to the convergence curve as the decision network of the trained infrared adversarial patch and execute the next step. Otherwise, update the reinforcement learning decision network and return to the initialization patch step.

[0063] Specifically, the reinforcement learning decision network is updated based on the experience pool. The experience pool includes the experience at each time step, and the format of the experience is <state information, action to be performed, reward value, updated state information>.

[0064] S107. Based on the decision network of the trained infrared countermeasures patch, grayscale blocks are pasted in the patch area to obtain the infrared countermeasures patch.

[0065] Grayscale blocks are used as the basic unit of infrared countermeasures patches. The infrared countermeasures patches are formed by sequentially splicing grayscale blocks, which simplifies the structure and texture of the infrared countermeasures patches and enhances their operability.

[0066] In this embodiment, the decision-making process of the trained infrared countermeasures patch decision network consists of a series of discrete actions A = {a1, a2, ..., a...} n Here, we define an agent as moving up, down, left, and right to select the placement position of a grayscale block, while simultaneously performing grayscale allocation. This means that the agent selects both the grayscale value and the position. If two grayscale values ​​are available, there will be 8 actions. After each action 'a', the agent will move from state 's'... t Move to the next state s t+1 Furthermore, since the state is related to the time step t, this embodiment sets a maximum time step t. m This is to avoid any single action taking too long. The state includes the block's current position (x...). t ,y tThe state is represented by the tuple (w, g) and the gray level of the block. A patch converter is used to represent the state on the image. The agent performs actions as follows: Figure 4 As shown.

[0067] It's worth noting that the agent moves exactly the length of a grayscale block each time, ensuring that the blocks are connected. Although the entire patch is irregular, it forms a complete connection, which helps in aligning and determining the position of tiles in the physical environment. This is equivalent to grayscale blocks moving within a grid, such as... Figure 4 As shown, the agent applies a specific grayscale to the grid it passes through. Each action of the agent changes the environment: when the agent moves to another location, the previous grayscale block does not disappear but becomes part of the new environment. On the one hand, this mechanism can, to some extent, suppress the repeated selection of a certain location; on the other hand, it can achieve the effect of a whole block rather than a single block.

[0068] In this embodiment, the target bounding box region in the infrared image is used as the reinforcement learning environment. An agent is defined to perform grayscale block pasting. The position and grayscale of the grayscale block are used as the state. The action space is defined by adjusting the position of the patch unit. After the agent performs an action, it obtains the action to be performed at the next time step and obtains a reward composed of a reward function built from the target detection results. The agent continuously accumulates experience during exploration. This experience is used to update the policy network, so that the agent can eventually learn how to perform actions according to the current state to obtain the maximum reward, thereby achieving the effect of resisting attacks.

[0069] In one embodiment, such as Figure 5 As shown, the total reward value for the reinforcement learning decision network, calculated based on all process samples, includes:

[0070] S501. Input all process samples into the target detector to obtain the target confidence score for each process sample.

[0071] S502. Calculate the difference in target confidence scores for all adjacent time steps of the process samples.

[0072] S503. Calculate the reward value for all differences using the reward function.

[0073] Specifically, the reward function in this embodiment is:

[0074]

[0075] S504. Summing up the reward values ​​corresponding to all differences yields the total reward value. The total reward value is calculated based on the target confidence level, and the convergence curve established by the total reward value can intuitively reflect whether the reinforcement learning decision network has been trained successfully.

[0076] In one specific embodiment, the performance of the reinforcement learning-based infrared adversarial patch generation method is verified using the Teledyne FLIR Free ADAS Thermal Dataset v2, targeting pedestrians.

[0077] The dataset was acquired using a thermal imager and a visible light camera mounted on a vehicle. Thermal images were obtained using a TeledyneFLIR Tau 2 13mm f / 1.0 lens with a horizontal field of view (HFOV) of 45 degrees and a vertical field of view (VFOV) of 37 degrees. The infrared thermal imager operated in T-linear mode. Time-synchronized acquisition was performed using Teledyne FLIR's Guardian software. A wide sampling range of segmented frames was used for training / validation images to ensure high diversity in both training and validation videos. The original dataset was filtered to better target pedestrian-based adversarial attacks, with the filtering criteria being: images containing the "person" category and a person's body height exceeding 120 pixels. A total of 1255 images were ultimately obtained, with 877 in the training set (1192 matching "person" labels) and 378 in the test set (618 matching "person" labels).

[0078] The goal of a successful attack is to prevent people in an image from being detected by the detector. To this end, Average Precision (AP) is used to evaluate the detector's performance on the threat dataset. A lower AP indicates a more effective attack. Furthermore, Attack Success Rate (ASR) is used to evaluate the effectiveness of the adversarial attack method, and is formalized as follows: Where tp is the number of times the target is correctly bounded, and N is the number of all correct labels detected in the dataset when not under attack. An attack is considered successful when the confidence level drops below 0.5. The higher the attack success rate, the more effective the counter-attack method is.

[0079] This embodiment was conducted on an Intel(R) Xeon(R) Bronze 3206R CPU and an Nvidia RTX3090 GPU. YOLOv5 was primarily used as the object detector to train the infrared pedestrian detector. To conform to model specifications, the image size was adjusted to 640×640, and pre-trained weights from YOLOv5s were used. The model was then fine-tuned using the selected dataset mentioned above. The resulting infrared pedestrian detector achieved average accuracy scores of 93.1% and 93.7% on the training and test sets, respectively, meeting the requirements for normal pedestrian object detection. During adversarial patch training, this embodiment used a batch size of 64 images, with each batch running for 500 rounds. The random greedy coefficient epsilon decayed exponentially with the number of training rounds, changing from 0.5 to 0.01, to ensure that reinforcement learning could explore as much as possible. The patch size was set to occupy a maximum of 30% of the target bounding box, with the top-left corner of the patch search area positioned relative to the top-left corner of the target bounding box at (0.2*h, 0.2*h), where h is the actual height of each target bounding box. The experimental patch block grayscale was set to 0.1 and 0.9. Under these settings, the patch block would explore the target area to cover certain locations and achieve an attack effect. During the experiment, the YOLO v5, acting as the target detector, could output AP, tp, and N, and the ASR could then be calculated from tp and N.

[0080] To further verify the performance of the method of the present invention, this embodiment fairly compares the method of the present invention with various existing methods capable of generating patches (Bulb, QR, AIP, AdvIB, and HCB) while ensuring the same patch coverage area. The target detector for each method is Yolo v5, and the obtained AP and ASR are shown in Table 1. The target detector detection results after performing adversarial attacks on the generated patches are shown in Table 1. Figure 6 As shown.

[0081] Table 1 shows the AP and ASR obtained using various patching methods.

[0082] Methods / Indicators AP ASR Clean sample 93.88 \ Bulb 70.43 37.7 QR 57.42 55.99 AIP 74.20 36.41 AdvIB 70.38 42.88 HCB 38.44 74.27 This invention 31.66 76.21

[0083] As shown in Table 1, when generating infrared countermeasures patches, the method of this invention achieves the lowest AP value (31.66) and the highest ASR value (76.21) compared to other methods. Therefore, the patch generated by the method of this invention has a strong attack effect and a high attack success rate. Figure 6It is known that after the infrared countermeasure patch generated by the method of the present invention performs the countermeasure attack, the target detector (Yolov5) cannot select the target person, the countermeasure attack has a high success rate, and the infrared countermeasure patch is concentrated in the square area on the upper half of the target person, and the patches are connected to each other, which makes it easier to determine the position of the patch in actual deployment.

[0084] Furthermore, this embodiment also validates the method of the present invention on four typical and mainstream detectors: YOLOv3, YOLOv5, Faster R-CNN, and DETR. ASR and AP are still used to evaluate attack performance. The results are shown in Table 2.

[0085] Table 2 Attack performance of different detectors

[0086]

[0087] As shown in Table 2, despite the differences between single-stage and two-stage detection models, the method of this invention is equally effective. For typical single-stage models in the YOLO series, an ASR of 80.1% was first achieved for the classic YOLOv3, and good performance was still obtained for the upgraded YOLOv5. Furthermore, for the two-stage detection models Faster R-CNN and DETR, even higher ASR attack effectiveness was achieved, demonstrating the versatility of the method of this invention across different detection systems.

[0088] Based on the same invention, the present invention also provides an apparatus for generating infrared countermeasures patches based on reinforcement learning, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the infrared countermeasures patch generation method based on reinforcement learning in the above embodiments.

[0089] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0090] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A method for generating infrared adversarial patches based on reinforcement learning, characterized in that, Includes the following steps: Initialize the patch to obtain an initial process sample; The coordinates and grayscale of the initial grayscale block are input as state information into the decision network of reinforcement learning to obtain the action to be executed at the next time step; Update the state information based on the action to be executed in the next time step, and obtain the process sample of the next time step; Determine if the number of time steps exceeds the preset time step threshold. If yes, proceed to the next step. Otherwise, input the updated state information into the reinforcement learning decision network to obtain the updated action to be executed in the next time step, and return to the step of updating the state information according to the action to be executed in the next time step. The total reward value corresponding to the decision network of the reinforcement learning is calculated based on all process samples; Establish a convergence curve based on the total reward value, determine whether the convergence curve has converged, if yes, use the reinforcement learning decision network corresponding to the convergence curve as the decision network of the trained infrared adversarial patch and execute the next step; otherwise, update the reinforcement learning decision network and return to the initialization patch step. The infrared countermeasures patch is obtained by pasting grayscale blocks in the patch area according to the trained infrared countermeasures patch decision network; The action to be performed at the next time step is to select the coordinates and grayscale of the grayscale block to be pasted in the patch area at the next time step, and the grayscale of the grayscale blocks in two adjacent time steps are different. The status information is updated based on the action to be performed at the next time step, including: In the patch area, a position is selected as the coordinate of the grayscale block for the next time step based on the action to be performed in the next time step; The grayscale and size of the grayscale block in the next time step are set according to the action to be executed in the next time step, wherein the size of the grayscale block in the next time step is the same as the size of the initial grayscale block; The total reward value for the reinforcement learning decision network is calculated based on all process samples, including: All process samples are input into the target detector to obtain the target confidence score for each process sample; Calculate the difference in target confidence scores for process samples at all adjacent time steps; The reward value for all differences is calculated using the reward function; The total reward value is obtained by summing the reward values ​​corresponding to all differences. When the trained infrared countermeasures patch decision network applies grayscale blocks to the patch area, the target bounding box area in the infrared image is used as the reinforcement learning environment. It is defined as an agent that moves up, down, left, and right to select the grayscale block application position and performs grayscale allocation at the same time. Grayscale blocks are used as the basic unit of the infrared countermeasures patch, and the infrared countermeasures patch is formed by sequentially splicing grayscale blocks.

2. The infrared adversarial patch generation method based on reinforcement learning according to claim 1, characterized in that, Initializing the patch and obtaining an initial process sample includes: Identify the patch area; Construct an initial grayscale block in the patch area; The initial grayscale block is pasted onto the initial grayscale block coordinates of the patch area to obtain the initial process sample.

3. The infrared adversarial patch generation method based on reinforcement learning according to claim 2, characterized in that, Constructing the initial grayscale block in the patch area includes: Arbitrarily select a coordinate in the patch area as the coordinate of the initial grayscale block; Set the grayscale and size of the initial grayscale block.

4. The infrared adversarial patch generation method based on reinforcement learning according to claim 1, characterized in that, Update the reinforcement learning decision network based on the experience pool; The experience pool includes the experience at each time step, and the format of the experience is <status information, executed action, reward value, updated status information>.

5. An infrared adversarial patch generation device based on reinforcement learning, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

Images