Target full-coverage confrontation texture generation and physical implementation method for multiband image collaborative optimization

Abstract

The invention provides a target full-coverage confrontation texture generation and physical implementation method for multiband image collaborative optimization, and relates to the technical field of image processing. According to the end-to-end digital domain multi-band adversarial sample generation framework provided by the method, micro 3D rendering is adopted, interaction of different bands is fully considered, collaborative optimization of multiple band adversarial samples is realized, the generated adversarial texture covers the surface of an object, the adversarial has robustness to an observation angle, and actual application requirements are met; meanwhile, the EOT data enhancement technology is used, the diversity of training data is increased, the robustness of the generated adversarial samples in different environments is enhanced, the network is guided to be balanced and optimized by combining classification loss, confidence loss and gray loss, and the robustness of the network is improved. And the generated multiband confrontation texture is physically realized by using an infrared low-emissivity material and infrared high-emissivity materials with different colors.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular, to a method for generating target full-coverage adversarial textures for multi-band image collaborative optimization and its physical implementation. Background Art

[0002] Currently, the adversarial sample attack method mainly targets single-band images. However, modern optoelectronic intelligent perception systems, such as target detection and target tracking intelligent systems, often use multi-band optoelectronic acquisition devices (such as visible light, near-infrared, mid-infrared, far-infrared sensors, etc.).

[0003] The research focus on multi-band intelligent models mainly concentrates on the interaction and fusion of information from different bands. For data from multi-band optoelectronic sensors, the fusion methods can be roughly divided into early fusion, mid-fusion, and late fusion. In the field of image processing, early fusion, mid-fusion, and late fusion can also be respectively referred to as pixel-level fusion, feature-level fusion, and decision-level fusion. Early fusion directly fuses the original data before feature extraction. Mid-fusion performs information fusion after extracting certain features. Late fusion only integrates the results after the output of each band. Early fusion is suitable for tasks where the data between different bands has temporal and spatial consistency, such as when the data returned by different bands are all image information of the same field of view. Mid-fusion is to solve the problem of spatial inconsistency of the original data in each band, and it can extract features from each band separately and then fuse them. Late fusion only fuses the output results of different bands and has the lowest requirement for data consistency. When there are large differences in the data of each band, such as large differences in dimension and sampling rate, late fusion is more suitable. The three fusion methods each have their own advantages and disadvantages. Early fusion and mid-fusion can better capture the relationship between data and features, but are prone to overfitting the training data. Late fusion can better handle overfitting but is difficult to capture relevant information in different dimensions. With the successive emergence and development of various band acquisition devices, intelligent detection systems have tended to be the comprehensive result of single-band, multi-band composite, and multi-band fusion. How to implement the same adversarial sample against the intelligent perception models of multi-band acquisition devices is a difficult problem to be solved in the field of intelligent perception deception, attracting a group of researchers to conduct exploration experiments.

[0004] With the rapid development and wide application of artificial intelligence and deep network technologies, computer security issues have become increasingly prominent, and adversarial attacks have emerged as a result. An adversarial attack refers to adding imperceptible tiny perturbations to the original input to make the network output incorrect results. According to whether the network's structural parameters are visible, adversarial attacks can be divided into white-box attacks and black-box attacks. In a white-box attack, the attacker can access the network model parameters and can use gradient-based adversarial attack algorithms; in a black-box attack, the network model parameters are not visible, and the attacker can only obtain the network's output and can create a surrogate network by continuously querying the input and output. Different from single-band adversarial attacks, the object of multi-band adversarial attacks combines multiple band information and has stronger robustness, making multi-band adversarial attacks more challenging. Therefore, how to generate multi-band adversarial samples has become a technical problem that urgently needs to be solved. Summary of the Invention

[0005] To solve the above technical problems, the present invention proposes a method for generating target-full-coverage adversarial textures with multi-band image collaborative optimization and its physical implementation.

[0006] The first aspect of the present invention discloses a method for generating target-full-coverage adversarial textures with multi-band image collaborative optimization and its physical implementation, and the method includes:

[0007] Step S1, randomly initialize the visible light adversarial texture and the infrared adversarial texture to obtain the initial visible light adversarial texture T tv and the initial infrared adversarial texture T ir ;

[0008] Step S2, divide the initial infrared adversarial texture T ir into an infrared high-emissivity adversarial texture and an infrared low-emissivity adversarial texture according to the determination threshold θ, and set the part of the initial visible light adversarial texture T tv corresponding to the position of the infrared low-emissivity adversarial texture to a specific color;

[0009] Step S3, use a 3D differentiable renderer to render the initial visible light adversarial texture T tv and the initial infrared adversarial texture T ir onto the surface of the target object model respectively to obtain a visible light target object adversarial sample and an infrared target object adversarial sample

[0010] Step S4, merge the visible light target object adversarial sample with the visible light original image to obtain a visible light adversarial image, and merge the infrared target object adversarial sample with the infrared original image to obtain an infrared adversarial image;

[0011] Step S5: Perform EOT data augmentation on the visible light countermeasure image and the infrared countermeasure image respectively, and input the data-augmented visible light countermeasure image and infrared countermeasure image into the multi-band detector to obtain the detection result;

[0012] Step S6: Calculate the multi-loss function based on the detection result, perform gradient backpropagation, and optimize and update the visible light countermeasure texture and the infrared countermeasure texture;

[0013] Step S7: Repeat Step S2 to Step S6 until the function value of the multi-loss function no longer decreases. Then, based on the generated infrared high emissivity countermeasure texture and infrared low emissivity countermeasure texture, cover the entire surface of the target object with a layer of infrared low emissivity material, and based on the generated visible light countermeasure texture, cover the color blocks at the corresponding positions of the visible light countermeasure texture at the positions of the infrared high emissivity countermeasure texture.

[0014] In the said Step S2, the initial infrared countermeasure texture T is divided into an infrared high emissivity countermeasure texture and an infrared low emissivity countermeasure texture according to the determination threshold θ, specifically including: ir Set the determination threshold θ. If the pixel value at a certain position of the initial infrared countermeasure texture is greater than this threshold θ, it is considered as the corresponding position of the infrared high emissivity. If it is less than the threshold θ, it is considered as the corresponding position of the infrared low emissivity;

[0015] Traverse all positions of the initial infrared countermeasure texture to obtain the infrared high emissivity countermeasure texture and the infrared low emissivity countermeasure texture.

[0016]

[0017] The said Step S3 specifically includes:

[0018] Given the target object model and rendering parameters, create a visible light 3D differentiable renderer R tv and an infrared 3D differentiable renderer R ir ;

[0019] Use the visible light 3D differentiable renderer R tv to render the initial visible light countermeasure texture T tv onto the surface of the target object model to obtain a fully covered visible light target object countermeasure sample Use the infrared 3D differentiable renderer R ir to render the initial infrared countermeasure texture T ir onto the surface of the target object model to obtain an infrared target object countermeasure sample

[0020] The said Step S4 specifically includes:

[0021] ​Train and apply the image segmentation network U-Net to segment the target regions and background regions in the original visible light image and the original infrared image;

[0022] Generate matrices with the same width and height as the original visible light image and the original infrared image respectively, and with 1 channel. Set the values at the corresponding positions of the target regions segmented on the matrices to 1, and the values at the corresponding positions of the background regions to 0, to obtain the first mask m1 and the second mask m2;

[0023] According to the formula Execute the adversarial sample of the visible light target object And perform the merging operation with the original visible light image, where, represents the visible light adversarial image, and the symbol ⊙ represents the matrix dot multiplication operation, I tv represents the original visible light image;

[0024] According to the formula Execute the adversarial sample of the infrared target object And perform the merging operation with the original infrared image, where, represents the infrared adversarial image, I ir represents the original infrared image.

[0025] Step S5 specifically includes;

[0026] Perform EOT data augmentation on the visible light adversarial image, that is:

[0027]

[0028] where p is a weight factor with a value range of [0, 1], is the image transformation operation;

[0029] Perform EOT data augmentation on the infrared adversarial image That is:

[0030]

[0031] In the step S5, adopt the Gaussian probability distribution, dynamically and randomly adjust the color and gray scale ranges of the visible light adversarial image and the mid-infrared adversarial image, and then perform the EOT data augmentation operation respectively.

[0032] In step S6, the multi-loss function L is:

[0033] L = αL cls + βL score + γL dist

[0034] where α, β, γ are weight parameters, L cls is the classification adversarial loss, Lscore is the classification confidence adversarial loss, L distThe grayscale loss is calculated as follows: According to the color limitations of the visible light physical implementation material, a color value set P is set, and then the grayscale loss is calculated: r(i) represents the pixel value at the i-th position of the target adversarial image, and p j is the j-th value in the color value set P, and n is the total number of pixels of the adversarial texture.

[0035] The second aspect of the present invention discloses a system for generating and physically implementing a target full-coverage adversarial texture for multi-band image collaborative optimization, and the system includes:

[0036] A first processing module, configured to randomly initialize a visible light adversarial texture and an infrared adversarial texture to obtain an initial visible light adversarial texture T tv and an initial infrared adversarial texture T ir ;

[0037] A second processing module, configured to divide the initial infrared adversarial texture T ir into an infrared high emissivity adversarial texture and an infrared low emissivity adversarial texture according to a determination threshold θ, and set the part of the initial visible light adversarial texture T tv corresponding to the position of the infrared low emissivity adversarial texture to a specific color;

[0038] A third processing module, configured to use a 3D differentiable renderer to render the initial visible light adversarial texture T tv and the initial infrared adversarial texture T ir onto the surface of the target object model respectively to obtain a visible light target object adversarial sample and an infrared target object adversarial sample

[0039] A fourth processing module, configured to merge the visible light target object adversarial sample with the visible light original image to obtain a visible light adversarial image, and merge the infrared target object adversarial sample with the infrared original image to obtain an infrared adversarial image;

[0040] A fifth processing module, configured to perform EOT data enhancement on the visible light adversarial image and the infrared adversarial image respectively, and input the data-enhanced visible light adversarial image and infrared adversarial image into a multi-band detector to obtain a detection result;

[0041] The sixth processing module is configured to calculate a multi-loss function based on the detection results, perform gradient backpropagation, optimize and update the visible light adversarial texture and the infrared adversarial texture. When the function value of the multi-loss function no longer decreases, based on the generated infrared high-emissivity adversarial texture and infrared low-emissivity adversarial texture, an infrared low-emissivity material is integrally coated on the surface of the target object, and based on the generated visible light adversarial texture, color blocks corresponding to the positions of the visible light adversarial texture are coated at the positions of the infrared high-emissivity adversarial texture, thereby achieving a physical implementation.

[0042] In a third aspect of the present invention, an electronic device is disclosed. The electronic device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps in any one of the above-mentioned methods for generating a target full-coverage adversarial texture with multi-band image collaborative optimization and its physical implementation are implemented.

[0043] In a fourth aspect of the present invention, a computer-readable storage medium is disclosed. A computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, the steps in any one of the above-mentioned methods for generating a target full-coverage adversarial texture with multi-band image collaborative optimization and its physical implementation are implemented.

[0044] In summary, the solution proposed by the present invention has the following technical effects: The proposed end-to-end digital domain multi-band adversarial sample generation framework in this method uses differentiable 3D rendering, fully considers the interaction between different bands, realizes the collaborative optimization of multi-band adversarial samples, and the generated adversarial texture covers the object surface. The adversarial property is robust to the observation angle, meeting the actual application requirements. At the same time, the present invention uses the EOT data augmentation technology to increase the diversity of training data to enhance the robustness of the generated adversarial samples in different environments, jointly uses the classification loss, confidence loss and grayscale loss to guide the network for balanced optimization, and uses infrared low-emissivity materials and infrared high-emissivity materials of different colors to physically implement the generated multi-band adversarial texture. Description of the Drawings

[0045] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0046] Figure 1 FIG. is a flowchart of end-to-end multi-band target full-coverage adversarial texture generation proposed according to an embodiment of the present invention;

[0047] Figure 2An end-to-end multi-band target full-coverage adversarial texture generation network architecture proposed according to an embodiment of the present invention;

[0048] Figure 3 A schematic diagram of dynamic data augmentation based on Gaussian probability proposed according to an embodiment of the present invention;

[0049] Figure 4 A structural diagram of an electronic device according to an embodiment of the present invention. Detailed implementation manners

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0051] It can be understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of this application, the first image may be referred to as the second image, and similarly, the second image may be referred to as the first image. Both the first image and the second image are images, but they are not the same image.

[0052] The method of the present invention is based on digital image processing technology and artificial intelligence technology, and combines deep learning technology and 3D rendering technology. The present invention applies a data augmentation method to enhance the diversity of training samples to obtain more robust adversarial textures, proposes an end-to-end digital domain multi-band adversarial texture generation algorithm, and through a differentiable 3D rendering method combined with a multi-loss function design, guides the network to perform multi-band adversarial texture collaborative optimization, realizes the generation of robust adversarial textures with full target coverage and multi-band applicability, and uses infrared low-emissivity materials and infrared high-emissivity materials of different colors to achieve the physical implementation of multi-band adversarial textures.

[0053] According to an embodiment of the present invention, in the first aspect, the present invention discloses a method for generating and physically implementing target full-coverage adversarial textures with multi-band image collaborative optimization. Please refer to Figure 1 and Figure 2 , the method includes:

[0054] Step S1, randomly initialize the visible light adversarial texture and the infrared adversarial texture to obtain the initial visible light adversarial texture T tv and the initial infrared adversarial texture T ir ;

[0055] In this step, the initial visible light countermeasure texture T is randomly generated tv , and the pixel values are randomly taken in the range of 0 - 1; the binary initial infrared countermeasure texture T ir is randomly generated, and the pixel values are randomly taken in the range of 0 - 1.

[0056] Step S2: Divide the initial infrared countermeasure texture T ir into an infrared high emissivity countermeasure texture and an infrared low emissivity countermeasure texture according to the determination threshold θ, and set the part of the initial visible light countermeasure texture T tv corresponding to the position of the infrared low emissivity countermeasure texture to a specific color;

[0057] In the said step S2, dividing the initial infrared countermeasure texture T ir into an infrared high emissivity countermeasure texture and an infrared low emissivity countermeasure texture specifically includes:

[0058] Set the determination threshold θ, such as 0.5. If the pixel value at a certain position of the initial infrared countermeasure texture is greater than this threshold θ, it is considered the corresponding position of the infrared high emissivity. If it is less than this threshold θ, it is considered the corresponding position of the infrared low emissivity;

[0059] Traverse all positions of the initial infrared countermeasure texture to obtain the infrared high emissivity countermeasure texture and the infrared low emissivity countermeasure texture.

[0060] Step S3: Use a 3D differentiable renderer to render the initial visible light countermeasure texture T tv and the initial infrared countermeasure texture T ir onto the surface of the target object model respectively to obtain a visible light target object countermeasure sample and an infrared target object countermeasure sample

[0061] The said step S3 specifically includes:

[0062] Given the target object model and rendering parameters, create a visible light 3D differentiable renderer R tv and an infrared 3D differentiable renderer R ir (executed only in the first iteration);

[0063] Use the visible light 3D differentiable renderer R tv to combine with the visible light image shooting parameters to render the initial visible light countermeasure texture T tv onto the surface of the target object model to obtain a full - coverage visible light target object countermeasure sample Use the infrared 3D differentiable renderer R ir to combine with the infrared image shooting parameters to render the initial infrared countermeasure texture T ir onto the surface of the target object model to obtain an infrared target object countermeasure sample

[0064] Step S4. Combine the visible light target object adversarial sample with the original visible light image to obtain a visible light adversarial image, and combine the infrared target object adversarial sample with the original infrared image to obtain an infrared adversarial image;

[0065] The specific steps of step S4 include:

[0066] Train and apply the image segmentation network U-Net to segment the target regions and background regions in the original visible light image and the original infrared image;

[0067] Generate matrices with the same width and height as the original visible light image and the original infrared image respectively, and with the number of channels being 1. Set the values at the corresponding positions of the target regions segmented on the matrices to 1, and set the values at the corresponding positions of the background regions to 0, to obtain the first mask m1 and the second mask m2;

[0068] According to the formula Perform the operation of combining the visible light target object adversarial sample with the original visible light image, where represents the visible light adversarial image, the symbol ⊙ represents the matrix dot product operation, and I tv represents the original visible light image;

[0069] According to the formula Perform the operation of combining the infrared target object adversarial sample with the original infrared image, where represents the infrared adversarial image, and I ir represents the original infrared image.

[0070] Step S5. Perform EOT data augmentation on the visible light adversarial image and the infrared adversarial image respectively, and input the data-augmented visible light adversarial image and infrared adversarial image into a multi-band detector to obtain detection results;

[0071] The specific steps of step S5 include:

[0072] Perform EOT data augmentation on the visible light adversarial image, that is:

[0073]

[0074] where p is a weight factor with a value range of [0, 1], is an image transformation operation;

[0075] Perform EOT data augmentation on the infrared adversarial image That is:

[0076]

[0077] In step S5, in order to further improve the robustness of adversarial samples, the Gaussian probability dynamic data augmentation technique is used. That is, on the basis of traditional EOT, the Gaussian probability distribution is adopted to dynamically and randomly adjust the color and grayscale ranges of visible light and mid-infrared digital domain adversarial textures, so as to ensure that the adversarial samples can adapt to the impacts of various environmental and device changes after being transferred to the physical domain. For details, please refer to Figure 3 , a visible light parameter and an infrared parameter are dynamically generated according to the Gaussian probability distribution, and after dynamically and randomly adjusting the color and grayscale ranges of the visible light adversarial image and the mid-infrared adversarial image by using the visible light parameter and the infrared parameter respectively, the EOT data augmentation operation is performed respectively.

[0078] Step S6: Calculate the multi-loss function based on the detection results, and perform gradient backpropagation to optimize and update the visible light adversarial texture and the infrared adversarial texture;

[0079] In step S6, the multi-loss function L is:

[0080] L = αL cls + βL score + γL dist

[0081] where α, β, and γ are weight parameters, L cls is the classification adversarial loss, Lscore is the classification confidence adversarial loss, L dist is the grayscale loss, and the calculation method is: according to the color limit of the visible light physical implementation material, set the color value set P, and then calculate the grayscale loss: r(i) represents the pixel value at the i-th position of the target adversarial image, p j is the j-th value in the color value set P, and n is the total number of pixels of the adversarial texture.

[0082] Among them, minimizing the classification adversarial loss is to reduce the probability that the sample is detected as the correct class, minimizing the confidence adversarial loss is to reduce the confidence score of the positive sample, and minimizing the grayscale loss is to constrain the color of the visible light adversarial texture to be close to the color that the actual physical implementation material can present; through the gradient descent algorithm, the network is guided to jointly optimize the multi-band adversarial texture.

[0083] Step S7: Repeat step S2 to step S6 until the function value of the multi-loss function no longer decreases, and then execute step S6;

[0084] Step S8: Based on the generated infrared high-emissivity adversarial texture and infrared low-emissivity adversarial texture, a layer of infrared low-emissivity material is integrally covered on the surface of the target object, and based on the generated visible light adversarial texture, the color blocks at the corresponding positions of the visible light adversarial texture are covered at the positions of the infrared high-emissivity adversarial texture to achieve physical implementation.

[0085] Step S8 specifically includes:

[0086] Overall cover a layer of infrared low-emissivity material on the surface of the target object;

[0087] Cover the color blocks at the corresponding positions of the visible light countermeasure texture on the high-emissivity positions of the infrared countermeasure texture.

[0088] The second aspect of the present invention discloses a multi-band image collaborative optimization target full-coverage countermeasure texture generation and its physical implementation system, and the system includes:

[0089] The first processing module is configured to randomly initialize the visible light countermeasure texture and the infrared countermeasure texture to obtain the initial visible light countermeasure texture T tv and the initial infrared countermeasure texture T ir ;

[0090] The second processing module is configured to divide the initial infrared countermeasure texture T ir into an infrared high-emissivity countermeasure texture and an infrared low-emissivity countermeasure texture according to the determination threshold θ, and set the part of the initial visible light countermeasure texture T tv corresponding to the position of the infrared low-emissivity countermeasure texture to a specific color;

[0091] The third processing module is configured to use a 3D differentiable renderer to render the initial visible light countermeasure texture T tv and the initial infrared countermeasure texture T ir onto the surface of the target object model respectively to obtain a visible light target object countermeasure sample and an infrared target object countermeasure sample

[0092] The fourth processing module is configured to merge the visible light target object countermeasure sample with the visible light original image to obtain a visible light countermeasure image, and merge the infrared target object countermeasure sample with the infrared original image to obtain an infrared countermeasure image;

[0093] The fifth processing module is configured to perform EOT data enhancement on the visible light countermeasure image and the infrared countermeasure image respectively, and input the data-enhanced visible light countermeasure image and infrared countermeasure image into a multi-band detector to obtain a detection result;

[0094] The sixth processing module is configured to calculate a multi-loss function based on the detection results, perform gradient backpropagation, optimize and update the visible light adversarial texture and the infrared adversarial texture. Until the function value of the multi-loss function no longer decreases, based on the generated infrared high-emissivity adversarial texture and infrared low-emissivity adversarial texture, an infrared low-emissivity material is integrally covered on the surface of the target object, and based on the generated visible light adversarial texture, color blocks corresponding to the positions of the visible light adversarial texture are covered at the positions of the infrared high-emissivity adversarial texture, thereby realizing a physical implementation.

[0095] The third aspect of the present invention discloses an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps in a method for generating a target full-coverage adversarial texture with multi-band image collaborative optimization and its physical implementation according to any one of the first aspects of the present disclosure are implemented.

[0096] Figure 4 FIG. is a structural diagram of an electronic device according to an embodiment of the present invention, as Figure 3 shown, the electronic device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner. The wireless manner can be achieved through WIFI, a carrier network, near field communication (NFC), or other technologies. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen. The input device of the electronic device can be a touch layer covered on the display screen, or a button, a trackball, or a touchpad provided on the housing of the electronic device, or an external keyboard, a touchpad, or a mouse, etc.

[0097] Those skilled in the art can understand that Figure 4 the structure shown in

[0098] is only a structural diagram of a part related to the technical solution of the present disclosure, and does not constitute a limitation on the electronic device to which the solution of the present application is applied. A specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have a different component layout.

[0099] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that it is still possible to modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features, and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating full-coverage adversarial texture for multi-band image collaborative optimization and its physical implementation, characterized in that: The method comprises: Step S1: Randomly initialize the visible light adversarial texture and infrared adversarial texture to obtain the initial visible light adversarial texture T tv and the initial infrared adversarial texture T ir ; Step S2: According to the judgment threshold θ, the initial infrared countermeasure texture T ir It is divided into infrared high emissivity counter-texture and infrared low emissivity counter-texture, and the initial visible light counter-texture T tv The portion corresponding to the position of the infrared low emissivity counter-texture is set to a specific color; Step S3: Use a 3D differentiable renderer to transform the initial visible light adversarial texture T tv and the initial infrared adversarial texture T ir Render them to the surface of the target object model respectively to obtain the visible light target object adversarial sample Adversarial samples with infrared target objects Step S4: Use the visible light target object against the sample The visible light original image is merged with the visible light original image to obtain the visible light adversarial image, and the infrared target object adversarial sample is Merge with the original infrared image to obtain the infrared adversarial image; Step S5, performing EOT data enhancement on the visible light confrontation image and the infrared confrontation image respectively, and inputting the data-enhanced visible light confrontation image and the infrared confrontation image into a multi-band detector to obtain a detection result; Step S6: Calculate multiple loss functions based on the detection results, perform gradient back propagation, and optimize and update the visible light adversarial texture and infrared adversarial texture; Step S7, repeating steps S2 to S6 until the function value of the multi-loss function no longer decreases, based on the generated infrared high emissivity adversarial texture and infrared low emissivity adversarial texture, covering the entire surface of the target object with a layer of infrared low emissivity material, and based on the generated visible light adversarial texture, covering the position of the infrared high emissivity adversarial texture with the color block at the corresponding position of the visible light adversarial texture.

2. The method according to claim 1, characterized in that In step S2, the initial infrared countermeasure texture T is set according to the judgment threshold θ. ir It is divided into infrared high emissivity counter texture and infrared low emissivity counter texture, including: Set a judgment threshold θ. If the pixel value at a certain position of the initial infrared adversarial texture is greater than this threshold θ, it is considered to be a position corresponding to infrared high emissivity. If it is less than this threshold θ, it is considered to be a position corresponding to infrared low emissivity. Traverse all positions of the initial infrared adversarial texture to obtain infrared high emissivity adversarial texture and infrared low emissivity adversarial texture.

3. The method according to claim 1, characterized in that The step S3 specifically includes: Given the target object model and rendering parameters, create a visible light 3D differentiable renderer R tv and infrared 3D differentiable renderer R ir ; Using the Visible Light 3D Differentiable Renderer R tv Combined with the visible light image shooting parameters, the initial visible light adversarial texture T tv Rendering to the surface of the target object model to obtain a fully covered visible light target object adversarial sample Using the Infrared 3D Differentiable Renderer R ir Combined with the infrared image shooting parameters, the initial infrared adversarial texture T ir Rendering to the surface of the target object model to obtain the infrared target object adversarial sample 4. The method according to claim 3, characterized in that The step S4 specifically includes: Train and apply the image segmentation network U-Net to segment the target area and background area in the visible light original image and infrared original image; Generate matrices with the same width and height as the visible light original image and the infrared original image, and the number of channels is 1, set the corresponding position values ​​of the target area segmented on the matrix to 1, and set the corresponding position values ​​of the background area to 0, to obtain the first mask m1 and the second mask m2; By formula Execute visible light object adversarial examples Merge operation with the visible light original image, where represents the visible light adversarial image, the symbol ⊙ represents the matrix dot multiplication operation, I tv Represents the original visible light image; By formula Execute infrared target object adversarial samples Merge operation with the original infrared image, where: represents the infrared adversarial image, I ir Indicates the original infrared image.

5. The method according to claim 4, characterized in that Step S5 specifically includes: Perform EOT data enhancement on the visible light adversarial image, namely: Among them, p is a weight factor with a value interval of [0, 1], is the image transformation operation; Infrared countermeasure image Perform EOT data augmentation, namely:

6. The method according to claim 5, characterized in that In step S5, Gaussian probability distribution is used to dynamically and randomly adjust the color and grayscale range of the visible light adversarial image and the mid-infrared adversarial image, and then EOT data enhancement operations are performed respectively.

7. The method according to claim 5, characterized in that In step S6, the multi-loss function L is: L=αL cls +βL score +γL dist Among them, α, β, γ are weight parameters, L cls is the classification adversarial loss, Lscore is the classification confidence adversarial loss; L dist is the grayscale loss, which is calculated as follows: according to the color limit of the material implemented by visible light physics, the color value set P is set, and then the grayscale loss is calculated: r(i) represents the pixel value at the i-th position of the target adversarial image, p j is the jth value in the color value set P, and n is the total number of pixels of the adversarial texture.

8. A target full coverage adversarial texture generation and physical implementation system for multi-band image collaborative optimization, characterized in that: The system comprises: The first processing module is configured to randomly initialize the visible light adversarial texture and the infrared adversarial texture to obtain an initial visible light adversarial texture T tv and the initial infrared adversarial texture T ir ; The second processing module is configured to convert the initial infrared countermeasure texture T into ir It is divided into infrared high emissivity counter-texture and infrared low emissivity counter-texture, and the initial visible light counter-texture T tv The portion corresponding to the position of the infrared low emissivity counter-texture is set to a specific color; The third processing module is configured to use a 3D differentiable renderer to transform the initial visible light into a texture T tv and the initial infrared adversarial texture T ir Render them to the surface of the target object model respectively to obtain the visible light target object adversarial sample Adversarial samples with infrared target objects The fourth processing module is configured to transform the visible light target object into an adversarial sample The visible light original image is merged with the visible light original image to obtain the visible light adversarial image, and the infrared target object adversarial sample is Merge with the original infrared image to obtain the infrared adversarial image; A fifth processing module is configured to perform EOT data enhancement on the visible light confrontation image and the infrared confrontation image respectively, and input the data-enhanced visible light confrontation image and the infrared confrontation image into the multi-band detector to obtain a detection result; The sixth processing module is configured to calculate multiple loss functions based on the detection results, and perform gradient back propagation to optimize and update the visible light adversarial texture and the infrared adversarial texture until the function value of the multiple loss function no longer decreases. Based on the generated infrared high emissivity adversarial texture and infrared low emissivity adversarial texture, a layer of infrared low emissivity material is covered on the entire surface of the target object, and based on the generated visible light adversarial texture, the color block at the corresponding position of the visible light adversarial texture is covered at the position of the infrared high emissivity adversarial texture.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the steps of the target full-coverage adversarial texture generation and physical implementation method of multi-band image collaborative optimization described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the method for generating target full-coverage adversarial texture for collaborative optimization of multi-band images and its physical implementation are implemented as described in any one of claims 1 to 7.

Images