Infrared stealth method and device based on fusion of model features and target texture information

Abstract

This invention discloses an infrared stealth method and device that fuses model features with target texture information. It also designs an adaptive patch initialization method based on target texture information. Specifically, by utilizing the target image's texture information to guide the design of the infrared adversarial patch's shape, size, and position, the conflict between the adversarial patch's complexity and its optimization difficulty is resolved. This allows for faster and more efficient patch design optimization while maintaining stealth effectiveness and completing the transition from the digital domain to the physical domain.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning security, and specifically relates to an infrared stealth method and device that fuses model features and target texture information. Background Art

[0002] Object detection is a crucial task in computer vision, aiming to detect the location and category of objects from images or videos. It holds significant research significance and application value in areas such as video surveillance, autonomous driving, and human-computer interaction. With the rapid development of deep learning algorithms, object detection technology is also evolving. Currently, mainstream object detection algorithms include Faster R-CNN, YOLO, and SSD. These methods all utilize convolutional neural networks (CNNs) as feature extraction modules and apply optimization methods such as sliding windows and pooling during the detection phase. However, as complex software systems, deep learning models are susceptible to various hacker attacks. Attacking deep learning systems can easily threaten property safety, personal privacy, traffic safety, and public safety.

[0003] The widespread application of deep learning and the vulnerability of object detection systems to adversarial attacks in both the digital and physical domains have led to growing concerns about their security. In recent years, the security of deep neural networks in visible light has garnered significant attention. However, in conditions of insufficient illumination or visual obstruction, such as foggy, rainy, and dark nights, the recognition performance of visible light object recognition models can be significantly reduced. Compared to visible light, all objects with temperatures above absolute zero emit thermal infrared radiation, allowing for observation without natural light supplementation, both during the day and at night. However, the security of infrared target detection remains a significant area for research. Consequently, many researchers have begun designing stealth algorithms for infrared target detection models to explore potential security vulnerabilities.

[0004] Some researchers have previously conducted related research, such as using infrared cloaks made of small light bulbs and aerogel to attack infrared detectors. This represents the first physical attack against thermal infrared intelligent pedestrian detectors. Other attack methods include using aerosols, a thermally insulating material, and heat-permeable fabrics to reproduce QR code-like adversarial textures in digital scenes, and using hot and cold stickers to reproduce black and white checkered adversarial textures in digital scenes. However, most existing methods only consider the attack effectiveness of adversarial examples. The designed shapes and patterns of adversarial examples often lack the deceptive power they possess in the physical domain to the human eye. Furthermore, the optimization process for adversarial examples often requires a significant computational overhead to traverse the entire target. Existing infrared cloaking methods often focus solely on the adversarial examples themselves, lacking a comprehensive consideration of target characteristics and adversarial examples.

[0005] In summary, we need to study an infrared stealth method that fuses model features and target texture information. This method can combine the characteristic information of pedestrian targets to reduce the optimization overhead of adversarial samples; design a master-slave multi-patch joint attack method to enhance the adversarial effect; and use conditional generative adversarial networks to optimize the patch shape in the master patch design to enhance the deceptiveness of the adversarial patch in the real world. Summary of the Invention

[0006] The purpose of the present invention is to address the deficiencies of the existing technology and provide an infrared stealth method and device that fuses model features and target texture information.

[0007] The object of the present invention is achieved through the following technical solution: an infrared stealth method for fusing model features and target texture information, comprising the following steps:

[0008] (1) Obtain the original image dataset X: X = {x1, x2, ..., x u ,…,x K}, where x u For any original image, u=1,2,…,u,…,K; each original image x u Contains one or more pedestrian target instances;

[0009] Then each original image x in the original image dataset X u Input into the target detection model F to obtain the detection result set X f :X f ={x 1,f ,x 2,f ,…,x u,f ,…,x K,f}, where x u,f Represents the original image x u The corresponding test results are calculated as follows:

[0010] x u,f =F(x u ,θ);

[0011] Among them, θ is the parameter of the target detection model F;

[0012] Detection result set X f Each test result x u,f for in, The test result x u,f Any pedestrian target result, v=1,…,v,…,V u , V u Indicates the test result x u,fThe total contains V u Pedestrian target results;

[0013] Each pedestrian target result Include pedestrian targets Pedestrian detection box position The probability of a pedestrian being identified as a pedestrian and the categories of pedestrian targets

[0014] (2) For the test result x u,f Any pedestrian target result Generate the corresponding initial master patch set

[0015] (3) For the initial master patch set Any initial main patch Generate 3 slave patches and And the initial main patch From the patch and Optimize and get the optimized main patch And the optimized patch and

[0016] (4) For the optimized main patch Perform smoothing to obtain the smoothed main patch The smoothed main patch is then And a random noise z is input into the generator of the trained conditional generative adversarial network to obtain a confusing adversarial main patch

[0017] (5) For the initial master patch set Repeat steps (3)-(4) for each initial main patch in to obtain the pedestrian target result The corresponding set of deceptive adversarial main patches

[0018] (6) For the test result x u,f Repeat steps (2) to (5) for each pedestrian target result in the image to obtain the original image x u The corresponding set of deceptive adversarial main patches {(Y u )2};

[0019] (7) In the digital domain, the obtained set of deceptive adversarial master patches {(Y u )2} Each deceptive adversarial main patch is sequentially pasted to the original image xu , get the patched image (x u ) * ;

[0020] (8) Repeat steps (2) to (7) for each original image in the original image dataset X to obtain the patched image dataset X * :X * ={(x1) * ,(x2) * ,…,(x u ) * ,…,(x k ) *}, realizing infrared stealth of the original image.

[0021] Furthermore, the step (2) specifically includes the following sub-steps:

[0022] (2.1) Results for any pedestrian target According to the obtained detection frame of the pedestrian target, each detected pedestrian target is cropped and stretched. Crop and stretch the image to a size of 64 pixels * 64 pixels Perform 8-bit quantization operation to make the grayscale range from 0 to 255;

[0023] Then the image Cropped into 64 images of size 8*8: And for each image The gray level is compressed to 0~8;

[0024] (2.2) For any image Define a 2×2 sliding window and select the horizontal direction as the direction for calculating the co-occurrence matrix. Calculate the co-occurrence relationship between each pixel in the image and its right neighbor pixel to obtain a 9×9 grayscale co-occurrence matrix. The gray-level co-occurrence matrix obtained Each element in represents the horizontal co-occurrence frequency of pixels with gray levels i and j, where i = 0, 1, 2, 3, 4, 5, 6, 7, 8 and j = 0, 1, 2, 3, 4, 5, 6, 7, 8;

[0025] And calculate and calculate any image Homogeneity of the gray-level co-occurrence matrix entropy and correlation The calculation formula is as follows:

[0026]

[0027]

[0028] Among them, Mean represents mean; Variance represents variance;

[0029] (2.3) and homogeneity entropy and correlation Perform weighted average calculation to obtain image Texture information score The calculation formula is as follows:

[0030]

[0031] Among them, w h 、w e and w c are weight vectors respectively;

[0032] (2.4) According to the image Texture information score Adaptively adjust the patch initialization position to generate an image Initial master patch for The initial master patch Size and image Same, position The position with the highest score is initialized with a random black or white block;

[0033] (2.5) For each image Repeat steps (2.2) to (2.4) to get the image Initial master patch set for

[0034] Furthermore, the step (3) specifically includes the following sub-steps:

[0035] (3.1) For the initial master patch set Any initial main patch Corresponding image Added 3 random from the patch and The patch and In the image The position of the patch is arbitrary; and The areas of the initial main patch are 1 / 10 of the patch and Divide into 3×3 grids, each grid is a black block or a white block; and from the patch and The area of ​​each black or white patch and the initial main patch The area of ​​each black block or white block is the same;

[0036] (3.2) and define the optimization parameter set in, Representing an image Corresponding initial master patch From the patch and The total number of black blocks, Indicates the initial master patch From the patch and The location set of

[0037] (3.3) Based on the cuckoo algorithm, the initial main patch From the patch and To optimize:

[0038] Use the following formula for optimization update:

[0039]

[0040] in, That is, image At the position of the tth generation, that is, the image Corresponding patch area parameter and positional parameters At the position of generation t; α represents the step length control amount; Levy(λ) is the Levy random search path, which belongs to random walk and adopts the Levy flight mechanism. The step length of its walk satisfies a heavy-tailed stable distribution, and the Levy distribution of the random step length Levy(λ) is:

[0041] Levy(λ)=t λ ,1≤λ≤3;

[0042] Update the next generation according to formula (1); after the position is updated, a randomly generated value R that obeys a uniform distribution from 0 to 1 is used to find the nest owner. The probability (P a ) t+1 In comparison, if R>(P a ) t+1 then Make random changes, otherwise it is wrong Make changes;

[0043] Continuously optimize and update until the number of optimization updates reaches the set maximum number of iterations N, and the best optimization update is obtained As the global optimal solution;

[0044] (3.4) According to the global optimal solution Satisfy location set And the total number of black blocks The main patch of the optimized main patch Optimized slave patch and

[0045] Furthermore, the step (4) specifically includes the following sub-steps:

[0046] (4.1) For the optimized main patch The line is reduced in size and then median filtered. After median filtering, the vertices are restored to their original proportions. Then, the Bezier third-order curve formula is used for smoothing: first, the midpoint of each edge is found, the midpoints on both sides of the vertex are connected, and then the line segment is vertically translated until it passes through the vertex. At this time, the two end points of the line segment are control points. Finally, the control points and the vertex are used to calculate the points on the curve to generate the smoothed main patch.

[0047] (4.2) Then the smoothed main patch And a random noise z is input into the generator of the trained conditional generative adversarial network to obtain a confusing adversarial main patch

[0048] Furthermore, the step (4.2) specifically includes the following sub-steps:

[0049] (4.2.1) Make a dataset: Make an image containing patch contours: r1, r2, ..., r d ,…,r D And the real image corresponding to the desired fitting output: s1,s2,…,s d ,…,s D The dataset consists of s d is the patch contour image r d The real image corresponding to the desired fitting output;

[0050] (4.2.2) Construct a conditional generative adversarial network consisting of a generator and a discriminator; the generator is responsible for converting the input image into a generated output image, and the discriminator is used to evaluate the authenticity of the generated image;

[0051] (4.2.3) The generator adopts an encoder-decoder structure, and the encoder converts the patch contour image rd Encoded as a latent representation r d ′, and then the encoder transforms the latent representation r d ′ is decoded into the generated image r d ″;

[0052] (4.2.4) The discriminator is a binary classifier; the discriminator receives the generated image r d ″ and as the real image s d As input, and output the generated image r d ″ is the real image s d probability;

[0053] (4.2.5) Repeat steps (4.2.3) to (4.2.4) to perform adversarial training on the conditional generative adversarial network based on the dataset obtained in step (4.2.1), and iteratively train the generator and discriminator in the conditional generative adversarial network to obtain a trained conditional generative adversarial network;

[0054] (4.2.6) The smoothed main patch And a random noise z is input into the generator of the trained conditional generative adversarial network to obtain a confusing adversarial main patch

[0055] The present invention also includes an infrared stealth device that fuses model features and target texture information, including a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, it is used for the above-mentioned infrared stealth method that fuses model features and target texture information.

[0056] The present invention also includes a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the infrared stealth method of fusing the above-mentioned model features and target texture information is implemented.

[0057] The beneficial effects of the present invention are as follows: the infrared stealth method and device can contribute to research in the field of infrared vision model security. The present invention can successfully identify visual model vulnerabilities that prevent the infrared vision model from properly recognizing pedestrians or reduce the model's recognition accuracy. The present invention can quickly obtain a dataset of incorrectly recognized pedestrians, thereby facilitating retraining of the pedestrian recognition model and enhancing its robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is a flow chart of an infrared stealth method that fuses model features and target texture information;

[0059] Figure 2Schematic diagram of the results of an infrared stealth device that fuses model features and target texture information. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to illustrate the present invention, rather than to represent all embodiments. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments of the present invention without creative work are within the scope of protection of the present invention.

[0061] The technical concept of this invention is to develop an infrared stealth method that integrates model features with target texture information. Specifically for pedestrian stealth in infrared scenes, integrating the target's texture information enhances the patch's countermeasures. Furthermore, during the patch initialization phase, an adaptive patch initialization function designed based on the target's texture information significantly reduces the cost of patch optimization, accelerating the optimization process. Furthermore, to improve stealth in infrared scenes, we propose a master-slave multi-patch joint attack method. Adaptive patch initialization primarily initializes the position of the master patch, which resembles a large print on a pedestrian's clothing. Besides providing the primary attack effect, the master patch can also serve as a contour input after passing through a conditional generative adversarial network (CGN), generating a more deceptive, less perceptible print pattern in the physical domain. Furthermore, to compensate for the loss of countermeasures after the master patch passes through the CGN, we incorporate random slave patches into the optimization process alongside the master patch. Slave patches are smaller and simpler in structure. This master-slave multi-patch joint attack significantly enhances the overall stealth effectiveness of the method.

[0062] Example 1

[0063] This paper provides an infrared stealth method that integrates model features and target texture information. It combines the texture information of pedestrian targets with the generation and optimization of adversarial patches. On this basis, a master-slave multi-patch joint attack strategy and master patch graphics optimization based on conditional generative adversarial networks are designed. Figure 1 , an infrared stealth method for fusing model features and target texture information, comprising the following steps:

[0064] (1) Obtain the original image dataset X: X = {x1, x2, ..., x u ,…,x K}, where x u For any original image, u=1,2,…,u,…,K; each original image x u Contains one or more pedestrian target instances.

[0065] Then each original image x in the original image dataset X u Input into the target detection model F to obtain the detection result set X f :X f ={x 1,f ,x 2,f ,…,x u,f ,…,x K,f}, where x u,f Represents the original image x u The corresponding test results are calculated as follows:

[0066] x u,f =F(x u ,θ);

[0067] Among them, θ is the parameter of the target detection model F.

[0068] Detection result set X f Each test result x u,f for in, The test result x u,f Any pedestrian target result, v=1,…,v,…,V u , V u Indicates the test result x u,f The total contains V u Pedestrian target results.

[0069] Each pedestrian target result Include pedestrian targets Pedestrian detection box position The probability of a pedestrian being identified as a pedestrian and the categories of pedestrian targets

[0070] (2) For the test result x u,f Any pedestrian target result Generate the corresponding initial master patch set

[0071] The step (2) specifically includes the following sub-steps:

[0072] (2.1) Results for any pedestrian target According to the obtained detection frame of the pedestrian target, each detected pedestrian target is cropped and stretched. Crop and stretch the image to a size of 64 pixels * 64 pixels Perform 8-bit quantization operation to make the grayscale level range from 0 to 255.

[0073] Then the image Cropped into 64 images of size 8*8: And for each image The grayscale level is compressed to 0~8.

[0074] (2.2) For any image Define a 2×2 sliding window and select the horizontal direction as the direction for calculating the co-occurrence matrix. Calculate the co-occurrence relationship between each pixel in the image and its right neighbor pixel to obtain a 9×9 grayscale co-occurrence matrix. The gray-level co-occurrence matrix obtained Each element in Represents the horizontal co-occurrence frequency of pixels with gray levels i and j, where i = 0, 1, 2, 3, 4, 5, 6, 7, 8 and j = 0, 1, 2, 3, 4, 5, 6, 7, 8.

[0075] And calculate and calculate any image Homogeneity of the gray-level co-occurrence matrix entropy and correlation The calculation formula is as follows:

[0076]

[0077] Among them, Mean represents mean; Variance represents variance.

[0078] (2.3) and homogeneity entropy and correlation Perform weighted average calculation to obtain image Texture information score The calculation formula is as follows:

[0079]

[0080] Among them, w h 、w e and w c are weight vectors respectively.

[0081] (2.4) According to the image Texture information score Adaptively adjust the patch initialization position to generate an image Initial master patch for The initial master patch Size and image Same, position The position with the highest score initializes the shape to a random black or white block.

[0082] (2.5) For each image Repeat steps (2.2) to (2.4) to get the image Initial master patch set for

[0083] (3) For the initial master patch set Any initial main patch Generate 3 slave patches and And the initial main patch From the patch and Optimize and get the optimized main patch And the optimized patch and

[0084] The step (3) specifically includes the following sub-steps:

[0085] (3.1) For the initial master patch set Any initial main patch Corresponding image Added 3 random from the patch and The patch and In the image The position of the patch is arbitrary; and The areas of the initial main patch are 1 / 10 of the patch and Divide into 3×3 grids, each grid is a black block or a white block; and from the patch and The area of ​​each black or white patch and the initial main patch The area of ​​each black block or white block is the same.

[0086] (3.2) and define the optimization parameter set in, Representing an image Corresponding initial master patch From the patch and The total number of black blocks, Indicates the initial master patch From the patch and A collection of locations.

[0087] (3.3) Based on the cuckoo algorithm, the initial main patch From the patch and To optimize:

[0088] Use the following formula for optimization update:

[0089]

[0090] in, That is, image At the position of the tth generation, that is, the image Corresponding patch area parameter and positional parameters At the position of generation t; α represents the step length control amount; Levy(λ) is the Levy random search path, which belongs to random walk and adopts the Levy flight mechanism. The step length of its walk satisfies a heavy-tailed stable distribution, and the Levy distribution of the random step length Levy(λ) is:

[0091] Levy(λ)=t λ ,1≤λ≤3;

[0092] Update the next generation according to formula (1); after the position is updated, a randomly generated value R that obeys a uniform distribution from 0 to 1 is used to find the nest owner. The probability (P a ) t+1 In comparison, if R>(P a ) t+1 then Make random changes, otherwise it is wrong Make changes.

[0093] Continuously optimize and update until the number of optimization updates reaches the set maximum number of iterations N, and the best optimization update is obtained as the global optimal solution.

[0094] (3.4) According to the global optimal solution Satisfy location set And the total number of black blocks The main patch of the optimized main patch Optimized slave patch and

[0095] (4) For the optimized main patch Perform smoothing to obtain the smoothed main patch Then the smoothed main patch And a random noise z is input into the generator of the trained conditional generative adversarial network to obtain a confusing adversarial main patch

[0096] The step (4) specifically includes the following sub-steps:

[0097] (4.1) For the optimized main patch The line is reduced in size and then median filtered. After median filtering, the vertices are restored to their original proportions. Then, the Bezier third-order curve formula is used for smoothing: first, the midpoint of each edge is found, the midpoints on both sides of the vertex are connected, and then the line segment is vertically translated until it passes through the vertex. At this time, the two end points of the line segment are control points. Finally, the control points and the vertex are used to calculate the points on the curve to generate the smoothed main patch.

[0098] (4.2) Then the smoothed main patch And a random noise z is input into the generator of the trained conditional generative adversarial network to obtain a confusing adversarial main patch

[0099] The step (4.2) specifically includes the following sub-steps:

[0100] (4.2.1) Make a dataset: Make an image containing patch contours: r1, r2, ..., r d ,…,r D And the real image corresponding to the desired fitting output: s1,s2,…,s d ,…,s D The dataset consists of s d is the patch contour image r d The real image corresponding to the desired fitting output.

[0101] (4.2.2) Construct a conditional generative adversarial network consisting of a generator and a discriminator; the generator is responsible for converting the input image into a generated output image, and the discriminator is used to evaluate the authenticity of the generated image.

[0102] (4.2.3) The generator adopts an encoder-decoder structure, and the encoder converts the patch contour image r d Encoded as a latent representation r d ′, and then the encoder transforms the latent representation r d ′ is decoded into the generated image r d ″.

[0103] (4.2.4) The discriminator is a binary classifier; the discriminator receives the generated image r d ″ and as the real image sd As input, and output the generated image r d ″ is the real image s d probability.

[0104] (4.2.5) Repeat steps (4.2.3) to (4.2.4) based on the data set obtained in step (4.2.1) to perform adversarial training on the conditional generative adversarial network, and repeatedly iteratively train the generator and discriminator in the conditional generative adversarial network to obtain a trained conditional generative adversarial network.

[0105] (4.2.6) The smoothed main patch And a random noise z is input into the generator of the trained conditional generative adversarial network to obtain a confusing adversarial main patch

[0106] (5) For the initial master patch set Repeat steps (3)-(4) for each initial main patch in to obtain the pedestrian target result The corresponding set of deceptive adversarial main patches

[0107] (6) For the test result x u,f Repeat steps (2) to (5) for each pedestrian target result in the image to obtain the original image x u The corresponding set of deceptive adversarial main patches {(Y u )2}.

[0108] (7) In the digital domain, the obtained set of deceptive adversarial master patches {(Y u )2} Each deceptive adversarial main patch is sequentially pasted to the original image x u , get the patched image (x u ) * .

[0109] (8) Repeat steps (2) to (7) for each original image in the original image dataset X to obtain the patched image dataset X * :X * ={(x1) * ,(x2) * ,…,(x u ) * ,…,(x k ) *}, realizing infrared stealth of the original image.

[0110] Example 2

[0111] This embodiment relates to an infrared stealth device that fuses model features and target texture information, including a memory and one or more processors. The memory stores executable code, and when the one or more processors execute the executable code, it is used for the infrared stealth method that fuses model features and target texture information in the above-mentioned embodiment 1; the device embodiment can be applied to any device with data processing capabilities, and the any device with data processing capabilities can be a device or apparatus such as a computer.

[0112] like Figure 2 At the hardware level, the knowledge distillation device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory. Of course, it may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 Of course, in addition to software implementation, the present invention does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0113] Improvements to a technology can be clearly distinguished as either hardware improvements (for example, improvements to circuit structures such as diodes, transistors, and switches) or software improvements (improvements to method flows). However, with the advancement of technology, many current method flow improvements can now be considered direct improvements to hardware circuit structures. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that a method flow improvement cannot be implemented using a hardware physical module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is an integrated circuit whose logical function is determined by user programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to hire a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly done using "logic compiler" software. This is similar to the software compiler used when developing programs. Before compilation, the original code must also be written in a specific programming language, called a hardware description language (HDL). There is not just one HDL, but many, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art will also understand that by simply programming the method flow in one of these hardware description languages ​​and then programming it into an integrated circuit, a hardware circuit that implements the logic method flow can be easily obtained.

[0114] The controller can be implemented in any suitable manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that in addition to implementing the controller in a purely computer-readable program code format, the controller can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software modules that implement the method and structures within the hardware component.

[0115] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0116] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0117] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0118] The present invention may be described in the general context of computer-executable instructions, such as program modules, executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media, including storage devices.

[0119] Example 3

[0120] An embodiment of the present invention further provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, the infrared stealth method of fusing model features and target texture information of the above-mentioned embodiment 1 is implemented.

[0121] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An infrared stealth method that fuses model features and target texture information, characterized in that: The following steps are involved: (1) Obtain the original image dataset X: X = {x1, x2, ..., x u ,…,x K }, where x u For any original image, u=1,2,…,u,…,K; each original image x u Contains one or more pedestrian target instances; Then each original image x in the original image dataset X u Input into the target detection model F to obtain the detection result set Xf: X f ={x 1,f ,x 2,f ,…,x u,f ,…,x K,f }, where xu,f represents the original image x u The corresponding test results are calculated as follows: x u,f =F(x u ,i); Among them, θ is the parameter of the target detection model F; Detection result set X f Each test result x u,f for in, The test result x u,f Any pedestrian target result, v=1,…,v,…,V u , V u Indicates the test result x u,f The total contains V u Pedestrian target results; Each pedestrian target result Include pedestrian targets Pedestrian detection box position The probability of a pedestrian being identified as a pedestrian and the categories of pedestrian targets (2) For the test result x u,f Any pedestrian target result Generate the corresponding initial master patch set (3) For the initial master patch set Any initial main patch Generate 3 slave patches and And the initial main patch From the patch and Optimize and get the optimized main patch And the optimized patch and (4) For the optimized main patch Perform smoothing to obtain the smoothed main patch The smoothed main patch is then And a random noise z is input into the generator of the trained conditional generative adversarial network to obtain a confusing adversarial main patch (5) For the initial master patch set Repeat steps (3)-(4) for each initial main patch in to obtain the pedestrian target result The corresponding set of deceptive adversarial main patches (6) For the test result x u,f Repeat steps (2) to (5) for each pedestrian target result in the image to obtain the original image x u The corresponding set of deceptive adversarial main patches {(Y u )2}; (7) In the digital domain, the obtained set of deceptive adversarial master patches {(Y u )2} Each deceptive adversarial main patch is sequentially pasted to the original image x u , get the patched image (x u ) * ; (8) Repeat steps (2) to (7) for each original image in the original image dataset X to obtain the patched image dataset X * :X * ={(x1) * ,(x2) * ,…,(x u ) * ,…,(x k ) * }, realizing infrared stealth of the original image.

2. The infrared stealth method of fusing model features and target texture information according to claim 1 is characterized in that: The step (2) specifically includes the following sub-steps: (2.1) Results for any pedestrian target According to the obtained detection frame of the pedestrian target, each detected pedestrian target is cropped and stretched. Crop and stretch the image to a size of 64 pixels * 64 pixels Perform 8-bit quantization operation to make the grayscale range from 0 to 255; Then the image Cropped into 64 images of size 8*8: And for each image The gray level is compressed to 0~8; (2.2) For any image Define a 2×2 sliding window and select the horizontal direction as the direction for calculating the co-occurrence matrix. Calculate the co-occurrence relationship between each pixel in the image and its right neighbor pixel to obtain a 9×9 grayscale co-occurrence matrix. The gray-level co-occurrence matrix obtained Each element in represents the horizontal co-occurrence frequency of pixels with gray levels i and j, where i = 0, 1, 2, 3, 4, 5, 6, 7, 8 and j = 0, 1, 2, 3, 4, 5, 6, 7, 8; And calculate and calculate any image Homogeneity of the gray-level co-occurrence matrix entropy and correlation The calculation formula is as follows: Among them, Mean represents mean; Variance represents variance; (2.3) and homogeneity entropy and correlation Perform weighted average calculation to obtain image Texture information score The calculation formula is as follows: Among them, w h 、w e and w c are weight vectors respectively; (2.4) According to the image Texture information score Adaptively adjust the patch initialization position to generate an image Initial master patch for The initial master patch Size and image Same, position The position with the highest score is initialized with a random black or white block; (2.5) For each image Repeat steps (2.2) to (2.4) to get the image Initial master patch set for 3. The infrared stealth method of fusing model features and target texture information according to claim 1 is characterized in that: The step (3) specifically includes the following sub-steps: (3.1) For the initial master patch set Any initial main patch Corresponding image Added 3 random from the patch and The patch and In the image The position of the patch is arbitrary; and The areas of the initial main patch are 1 / 10 of the patch and Divide into 3×3 grids, each grid is a black block or a white block; and from the patch and The area of ​​each black or white patch and the initial main patch The area of ​​each black block or white block is the same; (3.2) and define the optimization parameter set in, Representing an image Corresponding initial master patch From the patch and The total number of black blocks, Indicates the initial master patch From the patch and The location set of (3.3) Based on the cuckoo algorithm, the initial main patch From the patch and To optimize: Use the following formula for optimization update: in, That is, image At the position of the tth generation, that is, the image Corresponding patch area parameter and positional parameters At the position of generation t; α represents the step length control amount; Levy(λ) is the Levy random search path, which belongs to random walk and adopts the Levy flight mechanism. The step length of its walk satisfies a heavy-tailed stable distribution, and the Levy distribution of the random step length Levy(λ) is: Levy(λ)=t λ ,1≤λ≤3; Update the next generation according to formula (1); after the position is updated, a randomly generated value R that obeys a uniform distribution from 0 to 1 is used to find the nest owner. The probability (P a ) t+1 In comparison, if R>(P a ) t+1 Then Make random changes, otherwise it is wrong Make changes; Continuously optimize and update until the number of optimization updates reaches the set maximum number of iterations N, and the best optimization update is obtained As the global optimal solution; (3.4) According to the global optimal solution Satisfy location set And the total number of black blocks The main patch of the optimized main patch Optimized slave patch and 4. The infrared stealth method of fusing model features and target texture information according to claim 1, characterized in that: The step (4) specifically includes the following sub-steps: (4.1) For the optimized main patch The line is reduced in size and then median filtered. After median filtering, the vertices are restored to their original proportions. Then, the Bezier third-order curve formula is used for smoothing: first, the midpoint of each edge is found, the midpoints on both sides of the vertex are connected, and then the line segment is vertically translated until it passes through the vertex. At this time, the two end points of the line segment are control points. Finally, the control points and the vertex are used to calculate the points on the curve to generate the smoothed main patch. (4.2) Then the smoothed main patch And a random noise z is input into the generator of the trained conditional generative adversarial network to obtain a confusing adversarial main patch 5. The infrared stealth method of fusing model features and target texture information according to claim 4 is characterized in that: The step (4.2) specifically includes the following sub-steps: (4.2.1) Make a dataset: Make a patch contour image: r1, r2, ..., r d ,…,r D And the real image corresponding to the desired fitting output: s1,s2,…,s d ,…,s D The dataset consists of s d is the patch contour image r d The real image corresponding to the desired fitting output; (4.2.2) Construct a conditional generative adversarial network consisting of a generator and a discriminator; the generator is responsible for converting the input image into a generated output image, and the discriminator is used to evaluate the authenticity of the generated image; (4.2.3) The generator adopts an encoder-decoder structure, and the encoder converts the patch contour image r d Encoded as a latent representation r d ′, and then the encoder transforms the latent representation r d ′ is decoded into the generated image r d ″; (4.2.4) The discriminator is a binary classifier; the discriminator receives the generated image r d ″ and as the real image s d As input, and output the generated image r d ″ is the real image s d probability; (4.2.5) Repeat steps (4.2.3) to (4.2.4) to perform adversarial training on the conditional generative adversarial network based on the dataset obtained in step (4.2.1), and iteratively train the generator and discriminator in the conditional generative adversarial network to obtain a trained conditional generative adversarial network; (4.2.6) The smoothed main patch And a random noise z is input into the generator of the trained conditional generative adversarial network to obtain a confusing adversarial main patch 6. An infrared stealth device that fuses model features and target texture information, characterized in that: The invention comprises a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the infrared stealth method of fusing model features and target texture information as described in any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that A program is stored thereon, and when the program is executed by a processor, an infrared stealth method for fusing model features and target texture information as described in any one of claims 1 to 5 is implemented.

Images