Model feature and target texture information fused infrared stealth method and device

Abstract

The invention discloses a model feature and target texture information fused infrared stealth method and device, and designs an adaptive patch initialization method based on target texture information. The texture information of the target image is utilized to guide the shape, size and position design of the infrared adversarial patch, so that the conflict between the complexity of the adversarial patch and the optimization difficulty of the adversarial patch is solved; therefore, on the premise that the stealth effect is guaranteed and migration from the digital domain to the physical domain is completed, the patch design optimization process can be completed more efficiently and quickly.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning security, and specifically relates to an infrared stealth method and device that fuses model features and target texture information. Background Art

[0002] Object detection is an important task in the field of computer vision. It aims to detect the location and category of objects from images or videos. It has important research significance and application value in video surveillance, autonomous driving, human-computer interaction, etc. With the rapid development of deep learning algorithms, object detection technology is also evolving. At present, some mainstream object detection algorithms include Faster R-CNN, YOLO, SSD, etc. These methods all use convolutional neural networks (CNN) as feature extraction modules, and apply optimization methods such as sliding windows and pooling operations in the detection stage. However, as a very complex software system, the deep learning model will face various hacker attacks. By attacking the deep learning system, it is often easy to threaten property safety, personal privacy, traffic safety and public safety.

[0003] The extensive application of deep learning and the vulnerability of target detection systems exposed by adversarial attacks in the digital and physical domains have led to increasing concerns about their security. In recent years, the security of deep neural networks under visible light has attracted great attention. However, in foggy days, rainy days, nights, and when the light is insufficient or the vision is obstructed, the recognition ability of target recognition models under visible light will be greatly reduced. Compared with visible light, all objects with a temperature higher than absolute zero have thermal infrared radiation to the outside, and objects can be observed without natural light supplementation at night or during the day. However, there is still a lot of room for research on the security of infrared target detection. Therefore, many researchers have begun to design stealth algorithms for infrared target detection models to explore the security vulnerabilities therein.

[0004] Some researchers have done related research before, such as using infrared stealth cloaks made of small light bulbs and aerogels to attack infrared detectors. This is the first physical attack on thermal infrared intelligent pedestrian detectors. In addition, there are also attack methods such as using aerosols, thermal insulation materials, and heat-permeable fabrics to reproduce QR code-like adversarial textures in digital scenes, and using hot and cold stickers to reproduce black and white checkered adversarial textures in digital domain scenes. However, most of the current methods only consider the attack effect of adversarial samples unilaterally, and the design shape and pattern of adversarial samples often lack the confusion to the human eye in the physical domain. In addition, a large amount of computational overhead is often required to traverse the entire target during the optimization process of adversarial samples. Existing infrared stealth methods often only focus on the adversarial samples themselves and lack a comprehensive consideration of target features and adversarial samples.

[0005] In summary, we need to study an infrared stealth method that fuses model features and target texture information. This method can combine the feature information of pedestrian targets to reduce the optimization overhead of adversarial samples; design a master-slave multi-patch joint attack method to enhance the adversarial effect; and use conditional generative adversarial networks to optimize the patch shape in the master patch design to enhance the deceptiveness of the adversarial patch in the real world. Summary of the invention

[0006] The purpose of the present invention is to address the deficiencies of the prior art and to provide an infrared stealth method and device that fuses model features and target texture information.

[0007] The objective of the present invention is achieved through the following technical scheme: an infrared stealth method for fusing model features and target texture information, comprising the following steps:

[0008] (1) Obtain the original image dataset X: X = {x 1 ,x 2 ,…,x u ,…,x K}, where x u is any original image, u=1,2,…,u,…,K; each original image x u Contains one or more pedestrian target instances;

[0009] Then each original image x in the original image dataset X u Input into the target detection model F to obtain the detection result set X f :X f ={x 1,f ,x 2,f ,…,x u,f ,…,x K,f}, where x u,f Represents the original image x u The corresponding test results are calculated as follows:

[0010] x u,f =F(x u ,θ);

[0011] Among them, θ is the parameter of the target detection model F;

[0012] Detection result set X f Each test result x u,f for in, The test result x u,f Any pedestrian target result in , v=1,…,v,…,V u , V uIndicates the test result x u,f The total number of u Pedestrian target results;

[0013] Each pedestrian target result Include pedestrian targets Pedestrian target detection box position The probability of a pedestrian being identified as a pedestrian and the categories of pedestrian targets

[0014] (2) For the test result x u,f Any pedestrian target result Generate the corresponding initial master patch set

[0015] (3) For the initial main patch set Any of the initial main patches Generate 3 slave patches and And the initial main patch From the patch and Optimize and get the optimized main patch And the optimized slave patch and

[0016] (4) For the optimized main patch Perform smoothing to obtain the smoothed main patch The smoothed main patch is then And a random noise z is input into the generator of the trained conditional generative adversarial network to obtain a deceptive adversarial main patch

[0017] (5) For the initial main patch set Repeat steps (3)-(4) for each initial main patch in to obtain the pedestrian target result The corresponding set of deceptive adversarial main patches

[0018] (6) For the test result x u,f Repeat steps (2) to (5) for each pedestrian target result in the image to obtain the original image x u The corresponding set of deceptive adversarial main patches {(Y u ) 2};

[0019] (7) In the digital domain, the obtained set of deceptive adversarial main patches {(Y u )2 Each deceptive adversarial main patch in} is sequentially pasted to the original image x u , get the patched image (x u ) * ;

[0020] (8) Repeat steps (2) to (7) for each original image in the original image dataset X to obtain the patched image dataset X * :X * ={(x 1 ) * ,(x 2 ) * ,…,(x u ) * ,…,(x k ) *}, realizing infrared invisibility of the original image.

[0021] Furthermore, the step (2) specifically includes the following sub-steps:

[0022] (2.1) Results for any pedestrian target According to the obtained pedestrian target detection frame, each detected pedestrian target is cropped and stretched. Crop and stretch to an image size of 64 pixels * 64 pixels Perform 8-bit quantization operation to make the grayscale level 0 to 255;

[0023] Then the image Cropped into 64 images of size 8*8: And for each image The gray level is compressed to 0-8;

[0024] (2.2) For any image Define a 2×2 sliding window and select the horizontal direction as the direction for calculating the co-occurrence matrix. Calculate the co-occurrence relationship between each pixel in the image and its right neighbor pixel to obtain a 9×9 grayscale co-occurrence matrix. The gray-level co-occurrence matrix obtained Each element in represents the co-occurrence frequency of pixels with gray levels i and j in the horizontal direction, where i = 0, 1, 2, 3, 4, 5, 6, 7, 8 and j = 0, 1, 2, 3, 4, 5, 6, 7, 8;

[0025] And calculate and calculate any image Homogeneity of the gray-level co-occurrence matrix entropy and relevance The calculation formula is as follows:

[0026]

[0027]

[0028] Among them, Mean represents mean; Variance represents variance;

[0029] (2.3) and homogeneity entropy and relevance Perform weighted average calculation to get the image Texture information score The calculation formula is as follows:

[0030]

[0031] Among them, w h 、w e and w c are weight vectors respectively;

[0032] (2.4) According to the image Texture information score Adaptively adjust the patch initialization position to generate an image Initial master patch for The initial master patch Size and image Same, location The position with the highest score is initialized with a random black or white block;

[0033] (2.5) For each image Repeat steps (2.2) to (2.4) to get the image Initial master patch set for

[0034] Furthermore, the step (3) specifically includes the following sub-steps:

[0035] (3.1) For the initial main patch set Any of the initial main patches Corresponding image Added 3 random from the patch and The patch and In the image The position of and The areas of the initial main patch are 1 / 10 of the and Divide into 3×3 grids, each grid is a black block or a white block; and from the patch and The area of ​​each black or white patch in the initial main patch The area of ​​each black block or white block is the same;

[0036] (3.2) and define the optimization parameter set in, Representing images Corresponding initial master patch From the patch and The total number of black blocks, Indicates the initial master patch From the patch and The location set of

[0037] (3.3) Based on the cuckoo algorithm, the initial main patch From the patch and To optimize:

[0038] Use the following formula for optimization update:

[0039]

[0041] in, Image At the position of the tth generation, that is, the image Corresponding patch area parameter And positional parameters At the position of generation t; α represents the step length control amount; Levy(λ) is the Levy random search path, which belongs to random walk and adopts the Levy flight mechanism. The step length of its walk satisfies a heavy-tailed stable distribution, and the Levy distribution of the random step length Levy(λ) is:

[0042] Levy(λ)=t λ ,1≤λ≤3;

[0043] According to formula (1), the next generation is updated; after the position is updated, a randomly generated value R that follows a uniform distribution from 0 to 1 is used to find the nest owner. The probability (P a ) t+1 In comparison, if R>(P a ) t+1 then Make random changes, otherwise it is wrong Make changes;

[0044] Continuously optimize and update until the number of optimization updates reaches the set maximum number of iterations N, and the best optimization update is obtained As the global optimal solution;

[0045] (3.4) According to the global optimal solution Satisfy location set And the total number of black blocks The main patch of the optimized main patch Optimized slave patch and

[0046] Furthermore, the step (4) specifically includes the following sub-steps:

[0047] (4.1) For the optimized main patch The line is reduced in size and then median filtered. After median filtering, the vertices are restored in proportion. Then the Bezier third-order curve formula is used for smoothing: first find the midpoint of each edge, connect the midpoints on both sides of the vertex, and then translate the line segment vertically until it passes through the vertex. At this time, the two end points of the line segment are control points. Finally, the control points and vertices are used to calculate the points on the curve to finally generate the smoothed main patch.

[0048] (4.2) Then the smoothed main patch And a random noise z is input into the generator of the trained conditional generative adversarial network to obtain a deceptive adversarial main patch

[0049] Furthermore, the step (4.2) specifically includes the following sub-steps:

[0050] (4.2.1) Make a dataset: Make an image containing patch contours: 1 ,r 2 ,……,r d ,…,r D And the real image corresponding to the desired fitting output: s 1 ,s 2 ,…,s d ,…,s D The dataset consists of s d is the patch contour image r d The real image corresponding to the desired fitting output;

[0051] (4.2.2) constructing a conditional generative adversarial network consisting of a generator and a discriminator; the generator is responsible for converting an input image into a generated output image, and the discriminator is used to evaluate the authenticity of the generated image;

[0052] (4.2.3) The generator adopts an encoder-decoder structure, and the encoder converts the patch contour image r d Encoded as a latent representation r d ′, and then the encoder transforms the latent representation r d ′ is decoded into the generated image r d ″;

[0053] (4.2.4) The discriminator is a binary classifier; the discriminator receives the generated image r d ″ and as the real image s d As input, and output the generated image r d ″ is the real image s d probability;

[0054] (4.2.5) Repeat steps (4.2.3) to (4.2.4) according to the data set obtained in step (4.2.1) to perform adversarial training on the conditional generative adversarial network, and repeatedly iteratively train the generator and discriminator in the conditional generative adversarial network to obtain a trained conditional generative adversarial network;

[0055] (4.2.6) The smoothed main patch And a random noise z is input into the generator of the trained conditional generative adversarial network to obtain a deceptive adversarial main patch

[0056] The present invention also includes an infrared stealth device that fuses model features and target texture information, including a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, it is used for the above-mentioned infrared stealth method that fuses model features and target texture information.

[0057] The present invention also includes a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the infrared stealth method of fusing the above-mentioned model feature and target texture information is implemented.

[0058] The beneficial effects of the present invention are as follows: the infrared stealth method and device of the present invention can contribute to the research in the field of infrared visual model security. The present invention can successfully find visual model loopholes, so that the infrared visual model cannot normally identify pedestrians or reduce the recognition accuracy of the visual model. The present invention can quickly obtain a dataset of pedestrians that are incorrectly identified, which is beneficial to retraining the pedestrian recognition model and enhancing the robustness of the pedestrian recognition model. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 It is a flow chart of an infrared stealth method that fuses model features and target texture information;

[0060] Figure 2 This is a schematic diagram of the results of an infrared stealth device that fuses model features and target texture information. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical scheme and advantages of the present invention more clear, the present invention is further described in detail in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0062] The technical concept of the present invention is: an infrared stealth method that integrates model features and target texture information. For pedestrian target stealth in infrared scenes, combining the texture information of pedestrian targets can enhance the adversarial effect of patches on the one hand, and on the other hand, in the initialization stage of patches, a patch initialization function with adaptive capabilities designed according to the texture information of the target can greatly reduce the consumption of the patch optimization process and speed up the patch optimization process. In addition, in order to improve the stealth effect in infrared scenes, we propose a master-slave multi-patch joint attack method. The adaptive initialization of patches is mainly used for the position initialization of the main patch. The main patch is similar to a larger print on the clothes of pedestrians. In addition to providing the main attack effect, the main patch can also be used as a contour map condition input to generate a more confusing and difficult to be detected by the naked eye in the physical domain. Printing patterns. In addition, in order to make up for the loss of the adversarial effect of the main patch after passing through the conditional generative adversarial network, we added random slave patches to optimize the process together with the main patch. The slave patch is a patch with a smaller size and a simpler structure. The master-slave multi-patch joint attack method greatly improves the overall stealth effect of the method.

[0063] Example 1

[0064] The present invention provides an infrared stealth method that integrates model features and target texture information, combines the texture information of pedestrian targets with the generation and optimization of adversarial patches, and designs a master-slave multi-patch joint attack strategy and master patch graphics optimization based on conditional generative adversarial networks. Figure 1 , an infrared stealth method for fusing model features and target texture information, comprising the following steps:

[0065] (1) Obtain the original image dataset X: X = {x1 ,x 2 ,…,x u ,…,x K}, where x u is any original image, u=1,2,…,u,…,K; each original image x u Contains one or more pedestrian target instances.

[0066] Then each original image x in the original image dataset X u Input into the target detection model F to obtain the detection result set X f :X f ={x 1,f ,x 2,f ,…,x u,f ,…,x K,f}, where x u,f Represents the original image x u The corresponding test results are calculated as follows:

[0067] x u,f =F(x u ,θ);

[0068] Among them, θ is the parameter of the target detection model F.

[0069] Detection result set X f Each test result x u,f for in, The test result x u,f Any pedestrian target result in , v=1,…,v,…,V u , V u Indicates the test result x u,f The total number of u Pedestrian target results.

[0070] Each pedestrian target result Include pedestrian targets Pedestrian target detection box position The probability of a pedestrian being identified as a pedestrian and the categories of pedestrian targets

[0071] (2) For the test result x u,f Any pedestrian target result Generate the corresponding initial master patch set

[0072] The step (2) specifically includes the following sub-steps:

[0073] (2.1) Results for any pedestrian target According to the obtained pedestrian target detection frame, each detected pedestrian target is cropped and stretched. Crop and stretch to an image size of 64 pixels * 64 pixels Perform 8-bit quantization operation to make the grayscale level range from 0 to 255.

[0074] Then the image Cropped into 64 images of size 8*8: And for each image The grayscale level is compressed to 0~8.

[0075] (2.2) For any image Define a 2×2 sliding window and select the horizontal direction as the direction for calculating the co-occurrence matrix. Calculate the co-occurrence relationship between each pixel in the image and its right neighbor pixel to obtain a 9×9 grayscale co-occurrence matrix. The gray-level co-occurrence matrix obtained Each element in Represents the co-occurrence frequency of pixels with gray levels i and j in the horizontal direction, where i = 0, 1, 2, 3, 4, 5, 6, 7, 8 and j = 0, 1, 2, 3, 4, 5, 6, 7, 8.

[0076] And calculate and calculate any image Homogeneity of the gray-level co-occurrence matrix entropy and relevance The calculation formula is as follows:

[0077]

[0078] Among them, Mean represents mean; Variance represents variance.

[0079] (2.3) and homogeneity entropy and relevance Perform weighted average calculation to get the image Texture information score The calculation formula is as follows:

[0080]

[0081] Among them, w h 、w e and w c are weight vectors respectively.

[0082] (2.4) According to the image Texture information score Adaptively adjust the patch initialization position to generate an image Initial master patch for The initial master patch Size and image Same, location The position with the highest score initializes the shape to a random black or white block.

[0083] (2.5) For each image Repeat steps (2.2) to (2.4) to get the image Initial master patch set for

[0084] (3) For the initial main patch set Any of the initial main patches Generate 3 slave patches and And the initial main patch From the patch and Optimize and get the optimized main patch And the optimized slave patch and

[0085] The step (3) specifically includes the following sub-steps:

[0086] (3.1) For the initial main patch set Any of the initial main patches Corresponding image Added 3 random from the patch and The patch and In the image The position of and The areas of the initial main patch are 1 / 10 of the and Divide into 3×3 grids, each grid is a black block or a white block; and from the patch and The area of ​​each black or white patch in the initial main patch The area of ​​each black block or white block is the same.

[0087] (3.2) and define the optimization parameter set in, Representing images Corresponding initial master patch From the patch and The total number of black blocks, Indicates the initial master patch From the patch and A collection of locations.

[0088] (3.3) Based on the cuckoo algorithm, the initial main patch From the patch and To optimize:

[0089] Use the following formula for optimization update:

[0090]

[0091] in, Image At the position of the tth generation, that is, the image Corresponding patch area parameter And positional parameters At the position of generation t; α represents the step length control amount; Levy(λ) is the Levy random search path, which belongs to random walk and adopts the Levy flight mechanism. The step length of its walk satisfies a heavy-tailed stable distribution, and the Levy distribution of the random step length Levy(λ) is:

[0092] Levy(λ)=t λ ,1≤λ≤3;

[0093] According to formula (1), the next generation is updated; after the position is updated, a randomly generated value R that follows a uniform distribution from 0 to 1 is used to find the nest owner. The probability (P a ) t+1 In comparison, if R>(P a ) t+1 then Make random changes, otherwise it is wrong Make changes.

[0094] Continuously optimize and update until the number of optimization updates reaches the set maximum number of iterations N, and the best optimization update is obtained as the global optimal solution.

[0095] (3.4) According to the global optimal solution Satisfy location set And the total number of black blocks The main patch of the optimized main patch Optimized slave patch and

[0096] (4) For the optimized main patch Perform smoothing to obtain the smoothed main patch The smoothed main patch is then And a random noise z is input into the generator of the trained conditional generative adversarial network to obtain a deceptive adversarial main patch

[0097] The step (4) specifically includes the following sub-steps:

[0098] (4.1) For the optimized main patch The line is reduced in size and then median filtered. After median filtering, the vertices are restored in proportion. Then the Bezier third-order curve formula is used for smoothing: first find the midpoint of each edge, connect the midpoints on both sides of the vertex, and then translate the line segment vertically until it passes through the vertex. At this time, the two end points of the line segment are control points. Finally, the control points and vertices are used to calculate the points on the curve to finally generate the smoothed main patch.

[0099] (4.2) Then the smoothed main patch And a random noise z is input into the generator of the trained conditional generative adversarial network to obtain a deceptive adversarial main patch

[0100] The step (4.2) specifically includes the following sub-steps:

[0101] (4.2.1) Make a dataset: Make an image containing patch contours: 1 ,r 2 ,…,r d ,…,r D And the real image corresponding to the desired fitting output: s 1 ,s 2 ,…,s d ,…,s D The dataset consists of s d is the patch contour image r d The real image corresponding to the desired fitting output.

[0102] (4.2.2) Construct a conditional generative adversarial network consisting of a generator and a discriminator; the generator is responsible for converting the input image into a generated output image, and the discriminator is used to evaluate the authenticity of the generated image.

[0103] (4.2.3) The generator adopts an encoder-decoder structure, and the encoder converts the patch contour image r d Encoded as a latent representation r d ′, and then the encoder transforms the latent representation r d ′ is decoded into the generated image r d ″.

[0104] (4.2.4) The discriminator is a binary classifier; the discriminator receives the generated image r d ″ and as the real image s d As input, and output the generated image r d ″ is the real image s d probability.

[0105] (4.2.5) Repeat steps (4.2.3) to (4.2.4) according to the data set obtained in step (4.2.1) to perform adversarial training on the conditional generative adversarial network, and repeatedly iteratively train the generator and discriminator in the conditional generative adversarial network to obtain a trained conditional generative adversarial network.

[0106] (4.2.6) The smoothed main patch And a random noise z is input into the generator of the trained conditional generative adversarial network to obtain a deceptive adversarial main patch

[0107] (5) For the initial main patch set Repeat steps (3)-(4) for each initial main patch in to obtain the pedestrian target result The corresponding set of deceptive adversarial main patches

[0108] (6) For the test result x u,f Repeat steps (2) to (5) for each pedestrian target result in the image to obtain the original image x u The corresponding set of deceptive adversarial main patches {(Y u ) 2}.

[0109] (7) In the digital domain, the obtained set of deceptive adversarial main patches {(Y u ) 2 Each deceptive adversarial main patch in} is sequentially pasted to the original image x u , get the patched image (x u ) * .

[0110] (8) Repeat steps (2) to (7) for each original image in the original image dataset X to obtain the patched image dataset X * :X * ={(x 1 ) * ,(x 2 ) * ,…,(x u ) * ,…,(x k ) *}, realizing infrared invisibility of the original image.

[0111] Example 2

[0112] The present embodiment relates to an infrared stealth device that fuses model features and target texture information, including a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, an infrared stealth method that fuses model features and target texture information in the above-mentioned embodiment 1 is used; the device embodiment can be applied to any device with data processing capabilities, and the any device with data processing capabilities can be a device or apparatus such as a computer.

[0113] like Figure 2 At the hardware level, the knowledge distillation device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 Of course, in addition to the software implementation, the present invention does not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0114] For the improvement of a technology, it can be clearly distinguished whether it is a hardware improvement (for example, improvement of the circuit structure of diodes, transistors, switches, etc.) or a software improvement (improvement of the method flow). However, with the development of technology, many improvements of the method flow today can be regarded as direct improvements of the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented with a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming themselves, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0115] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0116] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0117] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0118] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0119] The present invention may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0120] Example 3

[0121] An embodiment of the present invention further provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, the infrared stealth method of fusing a model feature and target texture information of the above-mentioned embodiment 1 is implemented.

[0122] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An infrared stealth method that fuses model features and target texture information, characterized in that: The following steps are involved: (1) Obtain the original image dataset X: X = {x1, x2, …, x u ,…,x K }, where x u is any original image, u=1,2,…,u,…,K; each original image x u Contains one or more pedestrian target instances; Then each original image x in the original image dataset X u Input into the target detection model F to obtain the detection result set Xf: X f ={x 1,f ,x 2,f ,…,x u,f ,…,x K,f }, where xu,f represents the original image x u The corresponding test results are calculated as follows: x u,f =F(x u ,i); Among them, θ is the parameter of the target detection model F; Detection result set X f Each test result x u,f for in, The test result x u,f Any pedestrian target result in , v=1,…,v,…,V u , V u Indicates the test result x u,f The total number of u Pedestrian target results; Each pedestrian target result Include pedestrian targets Pedestrian target detection box position The probability of a pedestrian being identified as a pedestrian and the categories of pedestrian targets (2) For the test result x u,f Any pedestrian target result Generate the corresponding initial master patch set (3) For the initial main patch set Any of the initial main patches Generate 3 slave patches and And the initial main patch From the patch and Optimize and get the optimized main patch And the optimized slave patch and (4) For the optimized main patch Perform smoothing to obtain the smoothed main patch The smoothed main patch is then And a random noise z is input into the generator of the trained conditional generative adversarial network to obtain a deceptive adversarial main patch (5) For the initial main patch set Repeat steps (3)-(4) for each initial main patch in to obtain the pedestrian target result The corresponding set of deceptive adversarial main patches (6) For the test result x u,f Repeat steps (2) to (5) for each pedestrian target result in the image to obtain the original image x u The corresponding set of deceptive adversarial main patches {(Y u )2}; (7) In the digital domain, the obtained set of deceptive adversarial main patches {(Y u Each deceptive adversarial main patch in )2} is sequentially pasted to the original image x u , get the patched image (x u ) * ; (8) Repeat steps (2) to (7) for each original image in the original image dataset X to obtain the patched image dataset X * :X * ={(x1) * ,(x2) * ,…,(x u ) * ,…,(x k ) * }, realizing infrared invisibility of the original image.

2. The infrared stealth method of fusing model features and target texture information according to claim 1 is characterized in that: The step (2) specifically includes the following sub-steps: (2.1) Results for any pedestrian target According to the obtained pedestrian target detection frame, each detected pedestrian target is cropped and stretched. Crop and stretch to an image size of 64 pixels * 64 pixels Perform 8-bit quantization operation to make the grayscale level 0 to 255; Then the image Cropped into 64 images of size 8*8: And for each image The gray level is compressed to 0-8; (2.2) For any image Define a 2×2 sliding window and select the horizontal direction as the direction for calculating the co-occurrence matrix. Calculate the co-occurrence relationship between each pixel in the image and its right neighbor pixel to obtain a 9×9 grayscale co-occurrence matrix. The gray-level co-occurrence matrix obtained Each element in represents the co-occurrence frequency of pixels with gray levels i and j in the horizontal direction, where i = 0, 1, 2, 3, 4, 5, 6, 7, 8 and j = 0, 1, 2, 3, 4, 5, 6, 7, 8; And calculate and calculate any image Homogeneity of the gray-level co-occurrence matrix entropy and relevance The calculation formula is as follows: Among them, Mean represents mean; Variance represents variance; (2.3) and homogeneity entropy and relevance Perform weighted average calculation to get the image Texture information score The calculation formula is as follows: Among them, w h 、w e and w c are weight vectors respectively; (2.4) According to the image Texture information score Adaptively adjust the patch initialization position to generate an image Initial master patch for The initial master patch Size and image Same, location The position with the highest score is initialized with a random black or white block; (2.5) For each image Repeat steps (2.2) to (2.4) to get the image Initial master patch set for 3. The infrared stealth method of fusing model features and target texture information according to claim 1 is characterized in that: The step (3) specifically includes the following sub-steps: (3.1) For the initial main patch set Any of the initial main patches Corresponding image Added 3 random slaves to the patch and The patch and In the image The position of and The areas of the initial main patch are 1 / 10 of the and Divide into 3×3 grids, each grid is a black block or a white block; and from the patch and The area of ​​each black or white patch in the initial main patch The area of ​​each black block or white block is the same; (3.2) and define the optimization parameter set in, Representing images Corresponding initial master patch From the patch and The total number of black blocks, Indicates the initial master patch From the patch and The location set of (3.3) Based on the cuckoo algorithm, the initial main patch From the patch and To optimize: Use the following formula for optimization update: in, Image At the position of the tth generation, that is, the image Corresponding patch area parameter And positional parameters At the position of generation t; α represents the step length control amount; Levy(λ) is the Levy random search path, which belongs to random walk and adopts the Levy flight mechanism. The step length of its walk satisfies a heavy-tailed stable distribution, and the Levy distribution of the random step length Levy(λ) is: Levy(λ)=t λ ,1≤λ≤3; According to formula (1), the next generation is updated; after the position is updated, a randomly generated value R that follows a uniform distribution from 0 to 1 is used to find the nest owner. The probability (P a ) t+1 In comparison, if R>(P a ) t+1 then Make random changes, otherwise it is wrong Make changes; Continuously optimize and update until the number of optimization updates reaches the set maximum number of iterations N, and the best optimization update is obtained As the global optimal solution; (3.4) According to the global optimal solution Satisfy location set And the total number of black blocks The main patch of the optimized main patch Optimized slave patch and 4. The infrared stealth method of fusing model features and target texture information according to claim 1 is characterized in that: The step (4) specifically includes the following sub-steps: (4.1) For the optimized main patch The line is reduced in size and then median filtered. After median filtering, the vertices are restored in proportion. Then the Bezier third-order curve formula is used for smoothing: first find the midpoint of each edge, connect the midpoints on both sides of the vertex, and then translate the line segment vertically until it passes through the vertex. At this time, the two end points of the line segment are control points. Finally, the control points and vertices are used to calculate the points on the curve to finally generate the smoothed main patch. (4.2) Then the smoothed main patch And a random noise z is input into the generator of the trained conditional generative adversarial network to obtain a deceptive adversarial main patch 5. The infrared stealth method of fusing model features and target texture information according to claim 4 is characterized in that: The step (4.2) specifically includes the following sub-steps: (4.2.1) Make a dataset: Make a patch contour image: r1, r2, …, r d ,…,r D And the corresponding real image of the desired fitting output: s1,s2,…,s d ,…,s D The dataset consists of d is the patch contour image r d The real image corresponding to the desired fitting output; (4.2.2) constructing a conditional generative adversarial network consisting of a generator and a discriminator; the generator is responsible for converting an input image into a generated output image, and the discriminator is used to evaluate the authenticity of the generated image; (4.2.3) The generator adopts an encoder-decoder structure, and the encoder converts the patch contour image r d Encoded as a latent representation r d ′, and then the encoder transforms the latent representation r d ′ is decoded into the generated image r d ″; (4.2.4) The discriminator is a binary classifier; the discriminator receives the generated image r d ″ and as the real image s d As input, and output the generated image r d ″ is the real image s d probability; (4.2.5) Repeat steps (4.2.3) to (4.2.4) according to the data set obtained in step (4.2.1) to perform adversarial training on the conditional generative adversarial network, and repeatedly iteratively train the generator and discriminator in the conditional generative adversarial network to obtain a trained conditional generative adversarial network; (4.2.6) The smoothed main patch And a random noise z is input into the generator of the trained conditional generative adversarial network to obtain a deceptive adversarial main patch 6. An infrared stealth device that fuses model features and target texture information, characterized in that: It comprises a memory and one or more processors, wherein the memory stores executable codes, and when the one or more processors execute the executable codes, they are used to implement the infrared stealth method of fusing model features and target texture information as described in any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that: A program is stored thereon, and when the program is executed by a processor, an infrared stealth method for fusing model features and target texture information as described in any one of claims 1 to 5 is implemented.

Images