Target full-coverage confrontation texture generation and physical implementation method applicable to multiband image

Abstract

The invention provides a target full-coverage confrontation texture generation and physical implementation method applicable to a multiband image, and relates to the technical field of image processing. According to the method, an end-to-end digital domain multi-band adversarial sample generation framework is provided, micro 3D rendering is adopted, the influence of an infrared band implementation material on a visible light adversarial sample is fully considered, joint optimization of multiple band adversarial samples is achieved, generated adversarial textures cover the surface of an object, the adversarial has robustness on an observation angle, and the adversarial effect is good. Actual application requirements are met; meanwhile, the EOT data enhancement technology is used, and the network is guided to be balanced and optimized in combination with multiple types of loss; and the generated multiband confrontation texture is physically realized by covering the surface of an object with an infrared high-emissivity material and a low-emissivity material with different colors or covering the printed visible light confrontation sample and then covering the transparent infrared high-emissivity material and the low-emissivity material.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for generating target full-coverage adversarial textures applicable to multi-band images and its physical implementation method. Background Art

[0002] The current adversarial sample attack method is mainly aimed at attacking single-band images. However, today's optoelectronic intelligent perception systems, such as target detection and target tracking intelligent systems, often use multi-band optoelectronic acquisition devices (such as visible light, near-infrared, mid-infrared, far-infrared, etc.).

[0003] The research focus on multi-band intelligent models mainly concentrates on the interaction and fusion of information from different bands. For data from multiple bands, the fusion methods can be roughly divided into early fusion, mid-fusion, and late fusion. In the field of image processing, early fusion, mid-fusion, and late fusion can also be respectively referred to as pixel-level fusion, feature-level fusion, and decision-level fusion. Early fusion directly fuses the original data before feature extraction. Mid-fusion performs information fusion after extracting certain features. Late fusion only performs result integration after the output results of each band. Early fusion is suitable for tasks where the data between different bands has temporal and spatial consistency. For example, the data returned by different bands are all image information of the same field of view. Mid-fusion is to solve the problem of spatial inconsistency of the original data in each band, and it can extract features from each band separately and then fuse them. Late fusion only fuses the output results of different bands and has the lowest requirement for data consistency. When there are large differences in the data of each band, such as large differences in dimension and sampling rate, late fusion is more suitable. The three fusion methods each have their own advantages and disadvantages. Early fusion and mid-fusion can better capture the relationship between data and features, but are prone to overfitting the training data. Late fusion can better handle overfitting but is difficult to capture relevant information in different dimensions. With the successive emergence and development of various band acquisition devices, intelligent detection systems have tended to be the comprehensive results of single-band, multi-band composite, and multi-band fusion. How to achieve "one-to-many" confrontation against various band combinations (single, composite, fusion) is a difficult problem to be solved for the attack method, which has attracted a group of researchers to conduct exploratory experiments.

[0004] With the rapid development and wide application of artificial intelligence and deep network technologies, computer security issues have become increasingly prominent, and adversarial attacks have emerged as a result. An adversarial attack refers to adding imperceptible tiny perturbations to the original input to make the network output incorrect results. According to whether the network's structural parameters are visible, adversarial attacks can be divided into white-box attacks and black-box attacks. In white-box attacks, the attacker can access the network model parameters and use gradient-based adversarial attack algorithms; in black-box attacks, the network model parameters are not visible, and the attacker can only obtain the network's output and can create a surrogate network by continuously querying the input and output. Different from single-band adversarial attacks, the object of multi-band adversarial attacks combines multiple band information and has stronger robustness, making multi-band adversarial attacks more challenging. Therefore, how to generate multi-band adversarial samples has become a technical problem that urgently needs to be solved. Summary of the Invention

[0005] To solve the above technical problems, the present invention proposes a method for generating target-full-coverage adversarial textures applicable to multi-band images and its physical implementation.

[0006] The first aspect of the present invention discloses a method for generating target-full-coverage adversarial textures applicable to multi-band images and its physical implementation, the method comprising:

[0007] Step S1, randomly initialize the visible light adversarial texture and the infrared adversarial texture to obtain the initial visible light adversarial texture T tv and the initial infrared adversarial texture T ir ;

[0008] Step S2, use a 3D differentiable renderer to render the initial visible light adversarial texture T tv and the initial infrared adversarial texture T ir onto the surface of the target object model respectively to obtain the visible light target object adversarial sample and the infrared target object adversarial sample

[0009] Step S3, merge the visible light target object adversarial sample with the original visible light image to obtain a visible light adversarial image, and merge the infrared target object adversarial sample with the original infrared image to obtain an infrared adversarial image;

[0010] Step S4, perform EOT data augmentation on the visible light adversarial image and the infrared adversarial image respectively, and input the data-augmented visible light adversarial image and infrared adversarial image into a multi-band detector to obtain detection results;

[0011] Step S5, calculate a multi-loss function based on the detection results, and perform gradient backpropagation to optimize and update the visible light adversarial texture and the infrared adversarial texture;

[0012] Step S6: Repeat steps S2 to S5 until the function value of the multi-loss function no longer decreases, and then implement physical realization based on the generated visible light adversarial texture and infrared adversarial texture.

[0013] The specific steps of step S2 include:

[0014] Given the target object model and rendering parameters, create a visible light 3D differentiable renderer R tv and an infrared 3D differentiable renderer R ir ;

[0015] Use the visible light 3D differentiable renderer R tv to combine with the visible light image shooting parameters to render the initial visible light adversarial texture T tv onto the surface of the target object model to obtain a fully covered visible light target object adversarial sample Use the infrared 3D differentiable renderer R ir to combine with the infrared image shooting parameters to render the initial infrared adversarial texture T ir onto the surface of the target object model to obtain an infrared target object adversarial sample

[0016] The specific steps of step S3 include:

[0017] Train and apply the image segmentation network U-Net to segment the target regions and background regions in the original visible light image and the original infrared image;

[0018] Generate matrices with the same width and height dimensions as the original visible light image and the original infrared image, and with 1 channel respectively. Set the values at the corresponding positions of the target regions segmented on the matrices to 1, and the values at the corresponding positions of the background regions to 0, to obtain the first mask m1 and the second mask m2;

[0019] Execute the merging operation of the visible light target object adversarial sample and the original visible light image according to the formula where, represents the visible light adversarial image, the symbol ⊙ represents the matrix dot product operation, and I tv represents the original visible light image;

[0020] Execute the merging operation of the infrared target object adversarial sample and the original infrared image according to the formula where, represents the infrared adversarial image, and I ir represents the original infrared image.

[0021] The specific steps of step S4 include:

[0022] Perform EOT data augmentation on the visible light adversarial image, that is:

[0023]

[0024] Among them, p is a weight factor with a value range of [0, 1], is an image transformation operation;

[0025] For the infrared countermeasure image perform EOT data augmentation, that is:

[0026]

[0027] In the step S4, by adopting the Gaussian probability distribution, after dynamically and randomly adjusting the color and grayscale ranges of the visible light countermeasure image and the mid-infrared countermeasure image, perform the EOT data augmentation operation respectively.

[0028] In step S5, the multi-loss function L is:

[0029] L = αL cls + βL score + γL dist

[0030] Among them, α, β, γ are weight parameters, L cls is the classification adversarial loss, L score is the classification confidence adversarial loss, L dist is the grayscale loss, and the calculation method is: according to the color limitation of the visible light physical implementation material, set the color value set P, and then calculate the grayscale loss: r(i) represents the pixel value at the i-th position of the target adversarial image, p j is the j-th value in the color value set P, and n is the total number of pixels of the adversarial texture.

[0031] In step S6, implement the physical implementation based on the generated visible light adversarial texture and infrared adversarial texture, specifically including:

[0032] According to the generated visible light adversarial texture, print the color blocks at the corresponding positions of the visible light adversarial texture on the surface of the target object;

[0033] Set the judgment threshold θ. If the pixel value at a certain position of the infrared adversarial texture is greater than this threshold θ, it is considered the corresponding position of the high infrared emissivity. If it is less than the threshold θ, it is considered the corresponding position of the low infrared emissivity;

[0034] According to the generated infrared adversarial texture, cover the transparent infrared low emissivity material and the transparent infrared high emissivity material at the corresponding positions on the surface of the target object respectively.

[0035] Step S2 also includes:

[0036] Generate the corresponding solid color image I according to the actual color of the infrared material color, to adjust the adversarial samples of visible light target objects for adjustment, that is:

[0037]

[0038] wherein, is the adjusted adversarial sample of the visible light target object; a and b are weight parameters.

[0039] In step S6, physical implementation is carried out based on the generated visible light adversarial texture and infrared adversarial texture, specifically including:

[0040] Design materials with a variety of different structures, each structure of the material having different colors and emissivities, and according to the generated visible light adversarial texture and infrared adversarial texture, cover the materials on different local positions of the target object surface.

[0041] The second aspect of the present invention discloses a system for generating and physically implementing target full-coverage adversarial textures applicable to multi-band images, the system including:

[0042] The first processing module is configured to randomly initialize the visible light adversarial texture and the infrared adversarial texture to obtain the initial visible light adversarial texture T tv and the initial infrared adversarial texture T ir ;

[0043] The second processing module is configured to use a 3D differentiable renderer to render the initial visible light adversarial texture T tv and the initial infrared adversarial texture T ir onto the surface of the target object model respectively to obtain the visible light target object adversarial sample and the infrared target object adversarial sample

[0044] The third processing module is configured to merge the visible light target object adversarial sample with the visible light original image to obtain a visible light adversarial image, and merge the infrared target object adversarial sample with the infrared original image to obtain an infrared adversarial image;

[0045] The fourth processing module is configured to perform EOT data enhancement on the visible light adversarial image and the infrared adversarial image respectively, and input the data-enhanced visible light adversarial image and infrared adversarial image into a multi-band detector to obtain a detection result;

[0046] The fifth processing module is configured to calculate a multi-loss function based on the detection result, perform gradient backpropagation, optimize and update the visible light adversarial texture and the infrared adversarial texture, and when the function value of the multi-loss function no longer decreases, perform physical implementation based on the generated visible light adversarial texture and infrared adversarial texture.

[0047] In summary, the solution proposed by the present invention has the following technical effects: The end-to-end digital domain multi-band adversarial sample generation framework proposed by this method uses differentiable 3D rendering, fully considers the influence of materials in the infrared band on visible light adversarial samples, realizes the joint optimization of adversarial samples in multiple bands, generates adversarial textures that cover the object surface, and the adversarial property is robust to the observation angle, meeting the actual application requirements. At the same time, the present invention uses the EOT data augmentation technology to increase the diversity of training data, so as to enhance the robustness of the generated adversarial samples in different environments, jointly guides the network for balanced optimization with classification loss, confidence loss and grayscale loss, and physically realizes the generated multi-band adversarial textures by covering the object surface with infrared high-emissivity materials and low-emissivity materials of different colors or first covering the printed visible light adversarial samples and then covering transparent infrared high-emissivity materials and low-emissivity materials. Description of the Drawings

[0048] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0049] Figure 1 It is a flow chart of end-to-end multi-band target full-coverage adversarial texture generation proposed according to an embodiment of the present invention;

[0050] Figure 2 It is an end-to-end multi-band target full-coverage adversarial texture generation network architecture proposed according to an embodiment of the present invention;

[0051] Figure 3 It is a schematic diagram of dynamic data augmentation based on Gaussian probability proposed according to an embodiment of the present invention;

[0052] Figure 4 It is a structural diagram of an electronic device according to an embodiment of the present invention. Detailed Embodiments

[0053] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, rather than all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0054] It will be appreciated that the terms "first", "second", etc. used in the present application may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of the present application, the first image may be referred to as the second image, and similarly, the second image may be referred to as the first image. Both the first image and the second image are images, but they are not the same image.

[0055] The method of the present invention is based on digital image processing technology and artificial intelligence technology, integrating deep learning technology and 3D rendering technology. The present invention applies a data augmentation method to enhance the diversity of training samples to obtain more robust adversarial textures, and proposes an end-to-end digital domain multi-band adversarial texture generation algorithm. Through a differentiable 3D rendering method combined with the design of multiple loss functions, the network is guided to jointly optimize the multi-band adversarial textures, realizing the generation of robust adversarial textures with full target coverage and multi-band applicability. And the generated multi-band adversarial textures are physically implemented by covering the surface of an object with infrared high-emissivity materials and low-emissivity materials of different colors, or by first covering the printed visible light adversarial samples and then covering the transparent infrared high-emissivity materials and low-emissivity materials.

[0056] According to an embodiment of the present invention, in a first aspect, the present invention discloses a method for generating and physically implementing full target coverage adversarial textures applicable to multi-band images. Please refer to Figure 1 and Figure 2 , the method includes:

[0057] Step S1, randomly initialize the visible light adversarial texture and the infrared adversarial texture to obtain the initial visible light adversarial texture T tv and the initial infrared adversarial texture T ir ;

[0058] In this step, randomly generate the initial visible light adversarial texture T tv , and the pixel values are randomly taken in the range of 0-1; randomly generate the binary initial infrared adversarial texture T ir , and the pixel values are randomly taken in the range of 0-1.

[0059] Step S2, use a 3D differentiable renderer to render the initial visible light adversarial texture T tv and the initial infrared adversarial texture T ir onto the surface of the target object model respectively to obtain the visible light target object adversarial sample and the infrared target object adversarial sample

[0060] The specific steps of step S2 include:

[0061] Given the target object model and rendering parameters, create a visible light 3D differentiable renderer Rtv and the infrared 3D differentiable renderer R ir (executed only in the first iteration);

[0062] Use the visible light 3D differentiable renderer R tv Combine the initial visible light adversarial texture T with the visible light image shooting parameters tv Render it to the surface of the target object model to obtain a fully covered visible light target object adversarial sample Use the infrared 3D differentiable renderer R ir Combine the initial infrared adversarial texture T with the infrared image shooting parameters ir Render it to the surface of the target object model to obtain an infrared target object adversarial sample

[0063] Step S3: Combine the visible light target object adversarial sample with the original visible light image to obtain a visible light adversarial image, and combine the infrared target object adversarial sample with the original infrared image to obtain an infrared adversarial image;

[0064] The specific steps of step S3 include:

[0065] Train and apply the image segmentation network U-Net to segment the target area and background area in the original visible light image and the original infrared image;

[0066] Generate matrices with the same width and height dimensions as the original visible light image and the original infrared image respectively, and with the number of channels being 1. Set the values at the corresponding positions of the target area segmented on the matrix to 1, and the values at the corresponding positions of the background area to 0, to obtain the first mask m1 and the second mask m2;

[0067] According to the formula Perform the merging operation of the visible light target object adversarial sample with the original visible light image, where represents the visible light adversarial image, the symbol ⊙ represents the matrix dot product operation, and I tv represents the original visible light image;

[0068] According to the formula Perform the merging operation of the infrared target object adversarial sample with the original infrared image, where represents the infrared adversarial image, and I ir represents the original infrared image.

[0069] Step S4: Perform EOT data augmentation on the visible light adversarial image and the infrared adversarial image respectively, and input the data-augmented visible light adversarial image and infrared adversarial image into a multi-band detector to obtain detection results;

[0070] The specific steps of step S4 include;

[0071] Perform EOT data augmentation on the visible light countermeasure image, that is:

[0072]

[0073] where p is a weight factor with a value range of [0, 1], is an image transformation operation;

[0074] Perform EOT data augmentation on the infrared countermeasure image That is:

[0075]

[0076] In step S4, in order to further improve the robustness of the countermeasure samples, the Gaussian probability dynamic data augmentation technology is used. That is, on the basis of traditional EOT, the Gaussian probability distribution is adopted to dynamically and randomly adjust the color and gray scale ranges of the visible light and mid-infrared digital domain countermeasure textures, so as to ensure that the countermeasure samples can adapt to the influence of various environment and equipment changes after being migrated to the physical domain. For details, please refer to Figure 3 , Dynamically generate a visible light parameter and an infrared parameter according to the Gaussian probability distribution, and respectively perform dynamic random adjustment on the color and gray scale ranges of the visible light countermeasure image and the mid-infrared countermeasure image by using the visible light parameter and the infrared parameter, and then perform EOT data augmentation operations respectively.

[0077] Step S5: Calculate the multi-loss function based on the detection results, and perform gradient backpropagation to optimize and update the visible light countermeasure texture and the infrared countermeasure texture;

[0078] In step S5, the multi-loss function L is:

[0079] L = αL cls + βL score + γL dist

[0080] where α, β, γ are weight parameters, L cls is the classification countermeasure loss, L score is the classification confidence countermeasure loss, L dist is the gray scale loss, and the calculation method is: according to the color limit of the visible light physical implementation material, set the color value set P, and then calculate the gray scale loss: r(i) represents the pixel value at the i-th position of the target countermeasure image, p j is the j-th value in the color value set P, and n is the total number of pixels of the countermeasure texture.

[0081] Among them, minimizing the classification adversarial loss reduces the probability that a sample is detected as the correct class, minimizing the confidence adversarial loss reduces the confidence score of positive samples, and minimizing the grayscale loss constrains the color of the visible light adversarial texture to be close to the color that the actual physical implementation material can present; through the gradient descent algorithm, the network is guided to jointly optimize the multi-band adversarial texture.

[0082] Step S6: Repeat Step S2 to Step S5 until the function value of the multi-loss function no longer decreases, then execute Step S7;

[0083] Step S7: Implement physical realization based on the generated visible light adversarial texture and infrared adversarial texture.

[0084] Step S7 specifically includes:

[0085] According to the generated visible light adversarial texture, print color blocks at the corresponding positions of the visible light adversarial texture on the surface of the target object;

[0086] Set a decision threshold θ. If the pixel value at a certain position of the infrared adversarial texture is greater than this threshold θ, it is considered the corresponding position of high infrared emissivity. If it is less than this threshold θ, it is considered the corresponding position of low infrared emissivity;

[0087] According to the generated infrared adversarial texture, cover the corresponding positions on the surface of the target object with transparent low infrared emissivity materials and transparent high infrared emissivity materials respectively.

[0088] For this physical implementation method, in order to eliminate the influence of the infrared material on the visible light adversarial texture, Step S2 also includes:

[0089] Generate a corresponding solid color image I color for adjusting the visible light target object adversarial sample as follows:

[0090]

[0091] Among them, is the adjusted visible light target object adversarial sample; a and b are weight parameters.

[0092] Or,

[0093] Step S7 specifically includes:

[0094] Design materials with various different structures. Each structure of the material has different colors and emissivities. According to the generated visible light adversarial texture and infrared adversarial texture, cover different local positions on the surface of the target object with the materials. Such materials with various different structures include visible light materials, low infrared emissivity materials, and high infrared emissivity materials.

[0095] The second aspect of the present invention discloses a system for generating target full-coverage adversarial textures applicable to multi-band images and its physical implementation, the system comprising:

[0096] A first processing module, configured to randomly initialize a visible light adversarial texture and an infrared adversarial texture to obtain an initial visible light adversarial texture T tv and an initial infrared adversarial texture T ir ;

[0097] A second processing module, configured to use a 3D differentiable renderer to render the initial visible light adversarial texture T tv and the initial infrared adversarial texture T ir onto the surface of the target object model respectively to obtain a visible light target object adversarial sample and an infrared target object adversarial sample

[0098] A third processing module, configured to merge the visible light target object adversarial sample with the original visible light image to obtain a visible light adversarial image, and merge the infrared target object adversarial sample with the original infrared image to obtain an infrared adversarial image;

[0099] A fourth processing module, configured to perform EOT data enhancement on the visible light adversarial image and the infrared adversarial image respectively, and input the data-enhanced visible light adversarial image and infrared adversarial image into a multi-band detector to obtain a detection result;

[0100] A fifth processing module, configured to calculate a multi-loss function based on the detection result, perform gradient backpropagation, optimize and update the visible light adversarial texture and the infrared adversarial texture, and when the function value of the multi-loss function no longer decreases, implement physical implementation based on the generated visible light adversarial texture and infrared adversarial texture.

[0101] The third aspect of the present invention discloses an electronic device. The electronic device includes a memory and a processor. When the processor executes a computer program, the steps in a method for generating target full-coverage adversarial textures applicable to multi-band images and its physical implementation according to any one of the first aspects of the present disclosure are implemented.

[0102] Figure 4 is a structural diagram of an electronic device according to an embodiment of the present invention, as Figure 3As shown, the electronic device includes a processor, a memory, a communication interface, a display screen, and an input device connected via a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface of the electronic device is used to communicate with external terminals in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a carrier network, near field communication (NFC), or other technologies. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the electronic device, or an external keyboard, touchpad, or mouse, etc.

[0103] Those skilled in the art can understand that Figure 4 the structure shown in is only a structural diagram of a part related to the technical solution of the present disclosure, and does not constitute a limitation on the electronic device to which the solution of the present application is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0104] The fourth aspect of the present invention discloses a computer-readable storage medium. A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps in a method for generating a target full-coverage adversarial texture applicable to a multi-band image and its physical implementation in any one of the first aspects of the present disclosure are implemented.

[0105] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that the technical solutions described in the foregoing embodiments can still be modified, or some or all of the technical features can be equivalently replaced, and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating full-coverage adversarial texture for multi-band images and its physical implementation method, characterized in that: The method comprises: Step S1: Randomly initialize the visible light adversarial texture and infrared adversarial texture to obtain the initial visible light adversarial texture T tv and the initial infrared adversarial texture T ir ; Step S2: Use a 3D differentiable renderer to transform the initial visible light adversarial texture T tv and the initial infrared adversarial texture T ir Render them to the surface of the target object model respectively to obtain the visible light target object adversarial sample Adversarial samples with infrared target objects Step S3: Use the visible light target object against the sample The visible light original image is merged with the visible light image to obtain the visible light adversarial image, and the infrared target object adversarial sample is Merge with the original infrared image to obtain the infrared adversarial image; Step S4, performing EOT data enhancement on the visible light confrontation image and the infrared confrontation image respectively, and inputting the data-enhanced visible light confrontation image and the infrared confrontation image into a multi-band detector to obtain a detection result; Step S5, calculating multiple loss functions based on the detection results, and performing gradient backpropagation to optimize and update the visible light adversarial texture and the infrared adversarial texture; Step S6, repeating steps S2 to S5 until the function value of the multi-loss function no longer decreases, and performing physical realization based on the generated visible light adversarial texture and infrared adversarial texture.

2. The method according to claim 1, characterized in that: The step S2 specifically includes: Given the target object model and rendering parameters, create a visible light 3D differentiable renderer R tv and infrared 3D differentiable renderer R ir ; Using the Visible Light 3D Differentiable Renderer R tv Combined with the visible light image shooting parameters, the initial visible light adversarial texture T tv Rendering to the surface of the target object model to obtain a fully covered visible light target object adversarial sample Using the Infrared 3D Differentiable Renderer R ir Combined with the infrared image shooting parameters, the initial infrared adversarial texture T ir Rendering to the surface of the target object model to obtain the infrared target object adversarial sample 3. The method according to claim 2, characterized in that The step S3 specifically includes: Train and apply the image segmentation network U-Net to segment the target area and background area in the visible light original image and infrared original image; Generate matrices with the same width and height as the visible light original image and the infrared original image, and the number of channels is 1, set the corresponding position values ​​of the target area segmented on the matrix to 1, and set the corresponding position values ​​of the background area to 0, to obtain the first mask m1 and the second mask m2; By formula Execute visible light object adversarial examples Merge operation with the visible light original image, where represents the visible light adversarial image, the symbol ⊙ represents the matrix dot multiplication operation, I tv Represents the original visible light image; By formula Execute infrared target object adversarial samples Merge operation with the original infrared image, where: represents the infrared adversarial image, I ir Indicates the original infrared image.

4. The method according to claim 3, characterized in that Step S4 specifically includes: Perform EOT data enhancement on the visible light adversarial image, namely: Among them, p is a weight factor with a value interval of [0, 1], is the image transformation operation; Infrared countermeasure image Perform EOT data augmentation, namely:

5. The method according to claim 4, characterized in that In step S4, Gaussian probability distribution is used to dynamically and randomly adjust the color and grayscale range of the visible light adversarial image and the mid-infrared adversarial image, and then EOT data enhancement operations are performed respectively.

6. The method according to claim 4, characterized in that In step S5, the multi-loss function L is: L=αL cls +βL score +γL dist Among them, α, β, γ are weight parameters, L cls is the classification adversarial loss, L score is the classification confidence adversarial loss, L dist is the grayscale loss, which is calculated as follows: according to the color limit of the material implemented by visible light physics, the color value set P is set, and then the grayscale loss is calculated: r(i) represents the pixel value at the i-th position of the target adversarial image, p j is the jth value in the color value set P, and n is the total number of pixels of the adversarial texture.

7. The method according to claim 6, characterized in that In step S6, physical implementation is performed based on the generated visible light adversarial texture and infrared adversarial texture, specifically including: According to the generated visible light confrontation texture, a color block at a position corresponding to the visible light confrontation texture is printed on the surface of the target object; Set a judgment threshold θ. If the pixel value at a certain position of the infrared adversarial texture is greater than this threshold θ, it is considered to be a position corresponding to infrared high emissivity. If it is less than this threshold θ, it is considered to be a position corresponding to infrared low emissivity. According to the generated infrared countermeasure texture, the corresponding positions on the surface of the target object are covered with transparent infrared low emissivity material and transparent infrared high emissivity material respectively.

8. The method according to claim 7, characterized in that Step S2 also includes: Generate the corresponding pure color image I according to the actual color of the infrared material color , to counter samples of visible light target objects Make adjustments, namely: in, is the adjusted visible light target object adversarial sample; a and b are weight parameters.

9. The method according to claim 6, characterized in that In step S6, physical implementation is performed based on the generated visible light adversarial texture and infrared adversarial texture, specifically including: Materials with a variety of different structures are designed, and each structure of the material has different colors and emissivity. According to the generated visible light countermeasure texture and infrared countermeasure texture, different local positions on the surface of the target object are covered.

10. A target full coverage adversarial texture generation and physical implementation system applicable to multi-band images, characterized in that: The system comprises: The first processing module is configured to randomly initialize the visible light adversarial texture and the infrared adversarial texture to obtain an initial visible light adversarial texture T tv and the initial infrared adversarial texture T ir ; The second processing module is configured to use a 3D differentiable renderer to transform the initial visible light into a texture T tv and the initial infrared adversarial texture T ir Render them to the surface of the target object model respectively to obtain the visible light target object adversarial sample Adversarial samples with infrared target objects The third processing module is configured to transform the visible light target object into an adversarial sample The visible light original image is merged with the visible light image to obtain the visible light adversarial image, and the infrared target object adversarial sample is Merge with the original infrared image to obtain the infrared adversarial image; A fourth processing module is configured to perform EOT data enhancement on the visible light confrontation image and the infrared confrontation image respectively, and input the data-enhanced visible light confrontation image and the infrared confrontation image into the multi-band detector to obtain a detection result; The fifth processing module is configured to calculate multiple loss functions based on the detection results, perform gradient backpropagation, optimize and update the visible light adversarial texture and the infrared adversarial texture, until the function value of the multiple loss function no longer decreases, and implement physical realization based on the generated visible light adversarial texture and infrared adversarial texture.

Images