Multi-band image collaborative optimization target full coverage counter texture generation and physical implementation method thereof

Abstract

The application provides a multi-band image cooperative optimization target full-coverage adversarial texture generation and physical implementation method, and relates to the technical field of image processing.The end-to-end digital domain multi-band adversarial sample generation framework is provided, a differentiable 3D rendering is adopted, the interaction of different bands is fully considered, the cooperative optimization of multiple band adversarial samples is realized, the generated adversarial texture covers the surface of an object, the adversarial nature is robust to the observation angle, and the actual application requirements are met; meanwhile, the EOT data enhancement technology is used, the diversity of training data is increased, the robustness of the generated adversarial sample in different environments is enhanced, the network is guided to balanced optimization by combining the classification loss, the confidence loss and the gray scale loss, and the generated multi-band adversarial texture is physically implemented by using infrared low-emissivity materials and infrared high-emissivity materials with different colors.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for generating adversarial textures with full target coverage through multi-band image collaborative optimization and its physical implementation. Background Technology

[0002] Current adversarial attack methods mainly target single-band images, but modern optoelectronic intelligent sensing systems, such as target detection and target tracking systems, often use multi-band optoelectronic acquisition devices (such as visible light, near-infrared, mid-infrared, and far-infrared sensors).

[0003] Research on multi-band intelligent models primarily focuses on the interaction and fusion of information from different bands. For data from multi-band photoelectric sensors, fusion methods can be broadly categorized into early fusion, mid-fusion, and late fusion. In image processing, early fusion, mid-fusion, and late fusion are also known as pixel-level fusion, feature-level fusion, and decision-level fusion, respectively. Early fusion fuses the original data directly before feature extraction; mid-fusion fuses information after extracting certain features; and late fusion integrates results only after each band's output. Early fusion is suitable for tasks where data from different bands exhibits temporal and spatial consistency, such as when the returned data from different bands represent image information from the same field of view. Mid-fusion addresses the spatial inconsistency of the original data across different bands by extracting features from each band separately before fusion. Late fusion only fuses the outputs from different bands, requiring the least data consistency. When there are significant differences in the data across different bands, such as differences in dimensionality and sampling rate, late fusion is more suitable. Each of the three fusion methods has its advantages and disadvantages. Early and mid-fusion can better capture the relationship between data and features, but they are prone to overfitting the training data. Late fusion can better handle overfitting, but it is difficult to capture relevant information from different dimensions. With the successive emergence and development of various band acquisition devices, intelligent detection systems have tended to achieve comprehensive results of single-band, multi-band composite, and multi-band fusion. How to achieve an intelligent perception model that uses the same adversarial example to counter multi-band acquisition devices is an unsolved problem in the field of intelligent perception deception, attracting a number of researchers to conduct exploratory experiments.

[0004] With the rapid development and widespread application of artificial intelligence and deep network technologies, computer security issues have become increasingly prominent, giving rise to adversarial attacks. Adversarial attacks refer to adding imperceptible micro-perturbations to the original input to cause the network to output incorrect results. Based on whether the network's structural parameters are visible, adversarial attacks can be divided into white-box attacks and black-box attacks. In white-box attacks, attackers can access the network model parameters and use gradient-based adversarial attack algorithms; in black-box attacks, the network model parameters are not visible, and attackers can only obtain the network's output, creating alternative networks by continuously querying the input and output. Unlike single-band adversarial attacks, multi-band adversarial attacks combine information from multiple bands, possessing stronger robustness, making multi-band adversarial attacks more challenging. Therefore, how to generate multi-band adversarial examples has become an urgent technical problem to be solved. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention proposes a method for generating adversarial textures with full target coverage through multi-band image collaborative optimization and its physical implementation.

[0006] The first aspect of this invention discloses a method for generating adversarial textures covering the entire target area through multi-band image collaborative optimization and its physical implementation, the method comprising:

[0007] Step S1: Randomly initialize the visible light anti-countermeasure texture and the infrared anti-countermeasure texture to obtain the initial visible light anti-countermeasure texture T. tv and the initial infrared adversarial texture T ir ;

[0008] Step S2: Based on the judgment threshold θ, the initial infrared adversarial texture T ir It is divided into infrared high emissivity adversarial textures and infrared low emissivity adversarial textures, and the initial visible light adversarial texture T is also divided into infrared high emissivity adversarial textures and infrared low emissivity adversarial textures. tv The portion corresponding to the low infrared emissivity counter-texture is set to a specific color;

[0009] Step S3: Use a 3D differentiable renderer to render the initial visible light adversarial texture T. tv and the initial infrared adversarial texture T ir Render each sample onto the surface of the target object model to obtain adversarial examples of the target object in visible light. Adversarial examples of infrared target objects

[0010] Step S4: Present the visible light target object as an adversarial sample. The visible light image is merged with the original visible light image to obtain a visible light adversarial image, and the infrared target object adversarial sample is then used. The infrared countermeasures image is obtained by merging the original infrared image with the original infrared image.

[0011] Step S5: Perform EOT data augmentation on the visible light countermeasures image and the infrared countermeasures image respectively, and input the data-augmented visible light countermeasures image and infrared countermeasures image into the multi-band detector to obtain the detection results;

[0012] Step S6: Calculate multiple loss functions based on the detection results, perform gradient backpropagation, and optimize and update the visible light adversarial texture and infrared adversarial texture;

[0013] Step S7: Repeat steps S2 to S6 until the function value of the multiple loss function no longer decreases. Based on the generated infrared high emissivity adversarial texture and infrared low emissivity adversarial texture, cover the entire surface of the target object with a layer of infrared low emissivity material. Based on the generated visible light adversarial texture, cover the corresponding color block of the visible light adversarial texture at the position of the infrared high emissivity adversarial texture.

[0014] In step S2, the initial infrared adversarial texture T is determined according to the judgment threshold θ. ir Divided into infrared high emissivity adversarial textures and infrared low emissivity adversarial textures, specifically including:

[0015] Set a judgment threshold θ. If the pixel value at a certain position of the initial infrared adversarial texture is greater than this threshold θ, it is considered to be the position corresponding to high infrared emissivity. If it is less than this threshold θ, it is considered to be the position corresponding to low infrared emissivity.

[0016] Traverse all positions of the initial infrared adversarial texture to obtain infrared high emissivity adversarial textures and infrared low emissivity adversarial textures.

[0017] Step S3 specifically includes:

[0018] Given a target object model and rendering parameters, create a visible light 3D differentiable renderer R. tv and infrared 3D differentiable renderer R ir ;

[0019] Using the visible light 3D differentiable renderer R tv Combining visible light image capture parameters to create the initial visible light adversarial texture T tv Visible light adversarial examples rendered onto the surface of the target object model to achieve full coverage of the target object's surface. Using the infrared 3D differentiable renderer R ir Combining infrared image capture parameters to create the initial infrared adversarial texture T ir Infrared adversarial examples of target objects are obtained by rendering them onto the surface of the target object model.

[0020] Step S4 specifically includes:

[0021] Train and apply the image segmentation network U-Net to segment the target region and background region in the original visible light image and the original infrared image;

[0022] Generate matrices with the same width and height as the original visible light image and the original infrared image, and with 1 channel. Set the corresponding position value of the target area segmented on the matrix to 1 and the corresponding position value of the background area to 0 to obtain the first mask m1 and the second mask m2.

[0023] According to the formula Execute visible light target object adversarial samples The operation of merging with the original visible light image, where, Represents a visible light countermeasure image, with the symbol ⊙ indicating matrix dot product operation, and I tv Represents the original image of visible light;

[0024] According to the formula Performing infrared target adversarial examples The operation of merging with the original infrared image, in which, Indicates infrared countermeasures image, I ir This represents the original infrared image.

[0025] Step S5 specifically includes:

[0026] EOT data augmentation is performed on visible light adversarial images, namely:

[0027]

[0028] Where p is a weighting factor with a value range of [0, 1]. It is an image transformation operation;

[0029] Infrared countermeasures images Perform EOT data augmentation, namely:

[0030]

[0031] In step S5, a Gaussian probability distribution is used to dynamically and randomly adjust the color and grayscale range of the visible light countermeasure image and the mid-infrared countermeasure image before EOT data enhancement operations are performed respectively.

[0032] In step S6, the multiple loss function L is:

[0033] L=αL cls +βL score +γL dist

[0034] Where α, β, γ are weight parameters, L cls For classification adversarial loss, Lscore is the classification confidence adversarial loss, L distThe grayscale loss is calculated as follows: based on the color limitations of materials in visible light physical reality, a color value set P is set, and then the grayscale loss is calculated. r(i) represents the pixel value at the i-th position in the target adversarial image, p j Let be the j-th value in the color value set P, and n be the total number of pixels in the adversarial texture.

[0035] A second aspect of this invention discloses a system for multi-band image collaborative optimization of target full-coverage adversarial texture generation and its physical implementation, the system comprising:

[0036] The first processing module is configured to randomly initialize visible light anti-countermeasure textures and infrared anti-countermeasure textures to obtain an initial visible light anti-countermeasure texture T. tv and the initial infrared adversarial texture T ir ;

[0037] The second processing module is configured to process the initial infrared adversarial texture T according to the judgment threshold θ. ir It is divided into infrared high emissivity adversarial textures and infrared low emissivity adversarial textures, and the initial visible light adversarial texture T is also divided into infrared high emissivity adversarial textures and infrared low emissivity adversarial textures. tv The portion corresponding to the low infrared emissivity counter-texture is set to a specific color;

[0038] The third processing module is configured to use a 3D differentiable renderer to process the initial visible light adversarial texture T. tv and the initial infrared adversarial texture T ir Render each sample onto the surface of the target object model to obtain adversarial examples of the target object in visible light. Adversarial examples of infrared target objects

[0039] The fourth processing module is configured to process adversarial examples of visible light target objects. The visible light image is merged with the original visible light image to obtain a visible light adversarial image, and the infrared target object adversarial sample is then used. The infrared countermeasures image is obtained by merging the original infrared image with the original infrared image.

[0040] The fifth processing module is configured to perform EOT data augmentation on the visible light countermeasures image and the infrared countermeasures image respectively, and input the data-augmented visible light countermeasures image and infrared countermeasures image into the multi-band detector to obtain the detection results;

[0041] The sixth processing module is configured to calculate multiple loss functions based on the detection results, perform gradient backpropagation, optimize and update the visible light adversarial texture and infrared adversarial texture until the function value of the multiple loss functions no longer decreases. Based on the generated infrared high emissivity adversarial texture and infrared low emissivity adversarial texture, a layer of infrared low emissivity material is applied to the entire surface of the target object. Based on the generated visible light adversarial texture, color blocks at the corresponding positions of the infrared high emissivity adversarial texture are applied to the positions of the visible light adversarial texture, thereby achieving physical realization.

[0042] A third aspect of the present invention discloses an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps in the method for target full-coverage adversarial texture generation and its physical implementation as described in any of the preceding claims.

[0043] The fourth aspect of the present invention discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in the method for generating and physically implementing target full-coverage adversarial textures through multi-band image collaborative optimization as described in any of the preceding claims.

[0044] In summary, the proposed solution of this invention has the following technical effects: The end-to-end digital domain multi-band adversarial sample generation framework proposed by this method adopts differentiable 3D rendering, fully considers the interaction of different bands, realizes the collaborative optimization of multiple band adversarial samples, the generated adversarial texture covers the object surface, and the adversarial nature is robust to the observation angle, meeting the needs of practical applications; at the same time, this invention uses EOT data augmentation technology to increase the diversity of training data to enhance the robustness of the generated adversarial samples in different environments, and uses the joint classification loss, confidence loss and grayscale loss to guide the network to perform equalization optimization, and uses infrared low emissivity materials and infrared high emissivity materials of different colors to physically realize the generated multi-band adversarial texture. Attached Figure Description

[0045] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0046] Figure 1 This is a flowchart of the end-to-end multi-band target full-coverage adversarial texture generation process proposed in an embodiment of the present invention;

[0047] Figure 2This is an end-to-end multi-band target full-coverage adversarial texture generation network architecture proposed in an embodiment of the present invention;

[0048] Figure 3 A schematic diagram of dynamic data augmentation based on Gaussian probability proposed according to an embodiment of the present invention;

[0049] Figure 4 This is a structural diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] It is understood that the terms "first," "second," etc., used herein may be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of this application, a first image may be referred to as a second image, and similarly, a second image may be referred to as a first image. Both the first image and the second image are images, but they are not the same image.

[0052] This invention utilizes digital image processing and artificial intelligence technologies, integrating deep learning and 3D rendering techniques. It employs data augmentation to enhance the diversity of training samples, resulting in more robust adversarial textures. An end-to-end digital domain multi-band adversarial texture generation algorithm is proposed. Through a differentiable 3D rendering method combined with multiple loss function design, the network is guided to perform collaborative optimization of multi-band adversarial textures, achieving robust adversarial texture generation with full target coverage and applicability to multiple bands. Furthermore, the physical realization of multi-band adversarial textures is achieved using low-emissivity infrared materials and high-emissivity infrared materials of different colors.

[0053] According to an embodiment of the present invention, in a first aspect, a method for generating target full-coverage adversarial textures and its physical implementation based on multi-band image collaborative optimization is provided. Please refer to [link to relevant documentation]. Figure 1 and Figure 2 The method includes:

[0054] Step S1: Randomly initialize the visible light anti-countermeasure texture and the infrared anti-countermeasure texture to obtain the initial visible light anti-countermeasure texture T. tv and the initial infrared adversarial texture T ir ;

[0055] In this step, an initial visible light adversarial texture T is randomly generated. tv The pixel values ​​are randomly selected from 0 to 1; a random binary initial infrared adversarial texture T is generated. ir The pixel value is randomly selected from 0 to 1.

[0056] Step S2: Based on the judgment threshold θ, the initial infrared adversarial texture T ir It is divided into infrared high emissivity adversarial textures and infrared low emissivity adversarial textures, and the initial visible light adversarial texture T is also divided into infrared high emissivity adversarial textures and infrared low emissivity adversarial textures. tv The portion corresponding to the low infrared emissivity counter-texture is set to a specific color;

[0057] In step S2, the initial infrared adversarial texture T is determined according to the judgment threshold θ. ir Divided into infrared high emissivity adversarial textures and infrared low emissivity adversarial textures, specifically including:

[0058] Set a judgment threshold θ, such as 0.5. If the pixel value at a certain position of the initial infrared adversarial texture is greater than this threshold θ, it is considered to be the position corresponding to high infrared emissivity. If it is less than this threshold θ, it is considered to be the position corresponding to low infrared emissivity.

[0059] Traverse all positions of the initial infrared adversarial texture to obtain infrared high emissivity adversarial textures and infrared low emissivity adversarial textures.

[0060] Step S3: Use a 3D differentiable renderer to render the initial visible light adversarial texture T. tv and the initial infrared adversarial texture T ir Render each sample onto the surface of the target object model to obtain adversarial examples of the target object in visible light. Adversarial examples of infrared target objects

[0061] Step S3 specifically includes:

[0062] Given a target object model and rendering parameters, create a visible light 3D differentiable renderer R. tv and infrared 3D differentiable renderer R ir (Executed only during the first iteration);

[0063] Using the visible light 3D differentiable renderer R tv Combining visible light image capture parameters to create the initial visible light adversarial texture T tv Visible light adversarial examples rendered onto the surface of the target object model to achieve full coverage of the target object's surface. Using the infrared 3D differentiable renderer R ir Combining infrared image capture parameters to create the initial infrared adversarial texture T ir Infrared adversarial examples of target objects are obtained by rendering them onto the surface of the target object model.

[0064] Step S4: Present the visible light target object as an adversarial sample. The visible light image is merged with the original visible light image to obtain a visible light adversarial image, and the infrared target object adversarial sample is then used. The infrared countermeasures image is obtained by merging the original infrared image with the original infrared image.

[0065] Step S4 specifically includes:

[0066] Train and apply the image segmentation network U-Net to segment the target region and background region in the original visible light image and the original infrared image;

[0067] Generate matrices with the same width and height as the original visible light image and the original infrared image, and with 1 channel. Set the corresponding position value of the target area segmented on the matrix to 1 and the corresponding position value of the background area to 0 to obtain the first mask m1 and the second mask m2.

[0068] According to the formula Execute visible light target object adversarial samples The operation of merging with the original visible light image, where, Represents a visible light countermeasure image, with the symbol ⊙ indicating matrix dot product operation, and I tv Represents the original image of visible light;

[0069] According to the formula Performing infrared target adversarial examples The operation of merging with the original infrared image, in which, Indicates infrared countermeasures image, I ir This represents the original infrared image.

[0070] Step S5: Perform EOT data augmentation on the visible light countermeasures image and the infrared countermeasures image respectively, and input the data-augmented visible light countermeasures image and infrared countermeasures image into the multi-band detector to obtain the detection results;

[0071] Step S5 specifically includes:

[0072] EOT data augmentation is performed on visible light adversarial images, namely:

[0073]

[0074] Where p is a weighting factor with a value range of [0, 1]. It is an image transformation operation;

[0075] Infrared countermeasures images Perform EOT data augmentation, namely:

[0076]

[0077] In step S5, to further enhance the robustness of adversarial examples, Gaussian probabilistic dynamic data augmentation technology is employed. Specifically, based on traditional EOT, a Gaussian probability distribution is used to dynamically and randomly adjust the color and grayscale range of the adversarial texture in the visible light and mid-infrared digital domains. This ensures that the adversarial examples, after being transferred to the physical domain, can adapt to various environmental and device changes. For details, please refer to [link to relevant documentation]. Figure 3 Based on the Gaussian probability distribution, a visible light parameter and an infrared parameter are dynamically generated. The color and grayscale range of the visible light countermeasure image and the mid-infrared countermeasure image are then dynamically and randomly adjusted using the visible light parameter and the infrared parameter, and then EOT data augmentation operation is performed on them respectively.

[0078] Step S6: Calculate multiple loss functions based on the detection results, perform gradient backpropagation, and optimize and update the visible light adversarial texture and infrared adversarial texture;

[0079] In step S6, the multiple loss function L is:

[0080] L=αL cls +βL score +γL dist

[0081] Where α, β, γ are weight parameters, L cls For classification adversarial loss, Lscore is the classification confidence adversarial loss, L dist The grayscale loss is calculated as follows: based on the color limitations of materials in visible light physical reality, a color value set P is set, and then the grayscale loss is calculated. r(i) represents the pixel value at the i-th position in the target adversarial image, p j Let be the j-th value in the color value set P, and n be the total number of pixels in the adversarial texture.

[0082] Among them, the classification adversarial loss is minimized to reduce the probability of a sample being detected as the correct category, the confidence adversarial loss is minimized to reduce the confidence score of positive samples, and the grayscale loss is minimized to constrain the visible light adversarial texture color to be close to the color that the actual physical realization material can present. The gradient descent algorithm guides the network to jointly optimize the multi-band adversarial texture.

[0083] Step S7: Repeat steps S2 to S6 until the function value of the multiple loss function no longer decreases, then execute step S6.

[0084] Step S8: Based on the generated infrared high emissivity countermeasure texture and infrared low emissivity countermeasure texture, cover the entire surface of the target object with a layer of infrared low emissivity material, and based on the generated visible light countermeasure texture, cover the corresponding color block of the visible light countermeasure texture at the position of the infrared high emissivity countermeasure texture to achieve physical realization.

[0085] Step S8 specifically includes:

[0086] The entire surface of the target object is covered with a layer of low infrared emissivity material;

[0087] Color blocks from the corresponding positions of the visible light anti-texture are overlaid on the high emissivity positions of the infrared anti-texture.

[0088] A second aspect of this invention discloses a system for multi-band image collaborative optimization of target full-coverage adversarial texture generation and its physical implementation, the system comprising:

[0089] The first processing module is configured to randomly initialize visible light anti-countermeasure textures and infrared anti-countermeasure textures to obtain an initial visible light anti-countermeasure texture T. tv and the initial infrared adversarial texture T ir ;

[0090] The second processing module is configured to process the initial infrared adversarial texture T according to the judgment threshold θ. ir It is divided into infrared high emissivity adversarial textures and infrared low emissivity adversarial textures, and the initial visible light adversarial texture T is also divided into infrared high emissivity adversarial textures and infrared low emissivity adversarial textures. tv The portion corresponding to the low infrared emissivity counter-texture is set to a specific color;

[0091] The third processing module is configured to use a 3D differentiable renderer to process the initial visible light adversarial texture T. tv and the initial infrared adversarial texture T ir Render each sample onto the surface of the target object model to obtain adversarial examples of the target object in visible light. Adversarial examples of infrared target objects

[0092] The fourth processing module is configured to process adversarial examples of visible light target objects. The visible light image is merged with the original visible light image to obtain a visible light adversarial image, and the infrared target object adversarial sample is then used. The infrared countermeasures image is obtained by merging the original infrared image with the original infrared image.

[0093] The fifth processing module is configured to perform EOT data augmentation on the visible light countermeasures image and the infrared countermeasures image respectively, and input the data-augmented visible light countermeasures image and infrared countermeasures image into the multi-band detector to obtain the detection results;

[0094] The sixth processing module is configured to calculate multiple loss functions based on the detection results, perform gradient backpropagation, optimize and update the visible light adversarial texture and infrared adversarial texture until the function value of the multiple loss functions no longer decreases. Based on the generated infrared high emissivity adversarial texture and infrared low emissivity adversarial texture, a layer of infrared low emissivity material is applied to the entire surface of the target object. Based on the generated visible light adversarial texture, color blocks at the corresponding positions of the infrared high emissivity adversarial texture are applied to the positions of the visible light adversarial texture, thereby achieving physical realization.

[0095] A third aspect of this invention discloses an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the method for target full-coverage adversarial texture generation and its physical implementation in the multi-band image collaborative optimization method of any one of the first aspects of this disclosure.

[0096] Figure 4 This is a structural diagram of an electronic device according to an embodiment of the present invention, such as... Figure 3 As shown, the electronic device includes a processor, memory, communication interface, display screen, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, Near Field Communication (NFC), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input device can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the device's casing, or an external keyboard, touchpad, or mouse.

[0097] Those skilled in the art will understand that Figure 4 The structure shown is merely a structural diagram of the part related to the technical solution of this disclosure and does not constitute a limitation on the electronic device to which the solution of this application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0098] A fourth aspect of this invention discloses a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a method for multi-band image collaborative optimization of target full-coverage adversarial texture generation and its physical implementation, as described in any of the first aspects of this disclosure.

[0099] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein, and such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating adversarial textures covering full target coverage using multi-band image collaborative optimization and its physical implementation, characterized in that, The method includes: Step S1: Randomly initialize the visible light anti-counterfeiting texture and the infrared anti-counterfeiting texture to obtain the initial visible light anti-counterfeiting texture. and initial infrared adversarial texture ; Step S2: Based on the judgment threshold θ, the initial infrared adversarial texture is... It is divided into infrared high emissivity adversarial textures and infrared low emissivity adversarial textures, and the initial visible light adversarial texture is also included. The portion corresponding to the low infrared emissivity counter-texture is set to a specific color; Step S3: Use a 3D differentiable renderer to render the initial visible light adversarial texture. and initial infrared adversarial texture Render each sample onto the surface of the target object model to obtain adversarial examples of the target object in visible light. Adversarial examples of infrared target objects ; Step S3 specifically includes: Given a target object model and rendering parameters, create a visible light 3D differentiable renderer. and infrared 3D differentiable renderer ; Using a visible light 3D differentiable renderer Combining visible light image capture parameters to create initial visible light anti-texture Visible light adversarial examples rendered onto the surface of the target object model to achieve full coverage of the target object's surface. Using an infrared 3D differentiable renderer Combine infrared image capture parameters to create initial infrared countermeasure texture Infrared adversarial examples of target objects are obtained by rendering them onto the surface of the target object model. ; Step S4: Present the visible light target object as an adversarial sample. The visible light image is merged with the original visible light image to obtain a visible light adversarial image, and the infrared target object adversarial sample is then used. The infrared countermeasures image is obtained by merging the original infrared image with the original infrared image. Step S5: Perform EOT data augmentation on the visible light countermeasures image and the infrared countermeasures image respectively, and input the data-augmented visible light countermeasures image and infrared countermeasures image into the multi-band detector to obtain the detection results; Step S6: Calculate multiple loss functions based on the detection results, perform gradient backpropagation, and optimize and update the visible light adversarial texture and infrared adversarial texture; Step S7: Repeat steps S2 to S6 until the function value of the multiple loss function no longer decreases. Based on the generated infrared high emissivity adversarial texture and infrared low emissivity adversarial texture, cover the entire surface of the target object with a layer of infrared low emissivity material. Based on the generated visible light adversarial texture, cover the corresponding color block of the visible light adversarial texture at the position of the infrared high emissivity adversarial texture.

2. The method according to claim 1, characterized in that, In step S2, the initial infrared adversarial texture is determined according to the judgment threshold θ. Divided into infrared high emissivity adversarial textures and infrared low emissivity adversarial textures, specifically including: Set a judgment threshold θ. If the pixel value at a certain position of the initial infrared adversarial texture is greater than this threshold θ, it is considered to be the position corresponding to high infrared emissivity. If it is less than this threshold θ, it is considered to be the position corresponding to low infrared emissivity. Traverse all positions of the initial infrared adversarial texture to obtain infrared high emissivity adversarial textures and infrared low emissivity adversarial textures.

3. The method according to claim 1, characterized in that, Step S4 specifically includes: Train and apply the image segmentation network U-Net to segment the target region and background region in the original visible light image and the original infrared image; Generate matrices with the same dimensions as the original visible light image and the original infrared image, each with 1 channel. Set the values ​​at the corresponding positions of the target regions segmented in the matrices to 1, and the values ​​at the corresponding positions of the background regions to 0, thus obtaining the first mask. Second mask ; According to the formula ʘ ʘ Execute visible light target object adversarial samples The operation of merging with the original visible light image, where, This represents a visible light adversarial image, and the symbol ʘ represents the matrix dot product operation. Represents the original image of visible light; According to the formula ʘ ʘ Performing infrared target adversarial examples The operation of merging with the original infrared image, in which, Represents infrared countermeasures images. This represents the original infrared image.

4. The method according to claim 3, characterized in that, Step S5 specifically includes: EOT data augmentation is performed on visible light adversarial images, namely: in, It is a weighting factor with a value range of [0, 1]. It is an image transformation operation; Infrared countermeasures images Perform EOT data augmentation, namely: 。 5. The method according to claim 4, characterized in that, In step S5, a Gaussian probability distribution is used to dynamically and randomly adjust the color and grayscale range of the visible light countermeasure image and the mid-infrared countermeasure image before EOT data enhancement operations are performed respectively.

6. The method according to claim 4, characterized in that, In step S6, the multiple loss function L is: in, For weight parameters, To classify and combat losses, To mitigate loss through classification confidence; The grayscale loss is calculated as follows: based on the color limitations of materials in visible light physical reality, a color value set P is set, and then the grayscale loss is calculated. , This represents the pixel value at position i in the target adversarial image. Let be the j-th value in the color value set P, and n be the total number of pixels in the adversarial texture.

7. A system for multi-band image collaborative optimization of target full-coverage adversarial texture generation and its physical implementation, characterized in that, The system includes: The first processing module is configured to randomly initialize visible light anti-countermeasure textures and infrared anti-countermeasure textures to obtain an initial visible light anti-countermeasure texture. and initial infrared adversarial texture ; The second processing module is configured to process the initial infrared adversarial texture according to a judgment threshold θ. It is divided into infrared high emissivity adversarial textures and infrared low emissivity adversarial textures, and the initial visible light adversarial texture is also included. The portion corresponding to the low infrared emissivity counter-texture is set to a specific color; The third processing module is configured to use a 3D differentiable renderer to process the initial visible light adversarial texture. and initial infrared adversarial texture Render each sample onto the surface of the target object model to obtain adversarial examples of the target object in visible light. Adversarial examples of infrared target objects The third processing module is specifically configured as follows: Given a target object model and rendering parameters, create a visible light 3D differentiable renderer. and infrared 3D differentiable renderer ; Using a visible light 3D differentiable renderer Combining visible light image capture parameters to create initial visible light anti-texture Visible light adversarial examples rendered onto the surface of the target object model to achieve full coverage of the target object's surface. Using an infrared 3D differentiable renderer Combine infrared image capture parameters to create initial infrared countermeasure texture Infrared adversarial examples of target objects are obtained by rendering them onto the surface of the target object model. ; The fourth processing module is configured to process adversarial examples of visible light target objects. The visible light image is merged with the original visible light image to obtain a visible light adversarial image, and the infrared target object adversarial sample is then used. The infrared countermeasures image is obtained by merging the original infrared image with the original infrared image. The fifth processing module is configured to perform EOT data augmentation on the visible light countermeasures image and the infrared countermeasures image respectively, and input the data-augmented visible light countermeasures image and infrared countermeasures image into the multi-band detector to obtain the detection results; The sixth processing module is configured to calculate multiple loss functions based on the detection results, perform gradient backpropagation, optimize and update visible light adversarial textures and infrared adversarial textures until the function value of the multiple loss functions no longer decreases. Based on the generated infrared high emissivity adversarial textures and infrared low emissivity adversarial textures, a layer of infrared low emissivity material is applied to the entire surface of the target object. Based on the generated visible light adversarial textures, color blocks at the corresponding positions of the visible light adversarial textures are applied to the positions of the infrared high emissivity adversarial textures.

8. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, it implements the steps in the method for generating and physically implementing a multi-band image collaborative optimization target full-coverage adversarial texture as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the method for generating and physically implementing target full-coverage adversarial textures for multi-band image collaborative optimization as described in any one of claims 1 to 6.

Images