Method and apparatus for generating camouflage pattern, electronic device, and storage medium
By generating natural camouflage patterns, using Tyson polygon segmentation and color sampling technology, combined with deep learning detector optimization, the existing stealth methods are solved, and the more efficient stealth effect is achieved.
Patent Information
- Application Number
- PCT/CN2024/072733
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-16
- Filing Date
- 2024-01-17
- Publication Date
- 2024-11-21
AI Technical Summary
The existing pedestrian object detection algorithm is easily interfered by the anti-counterfeiting samples, resulting in misjudgment or misjudgment. The patch patterns or tiled patterns of the existing invisibility methods appear visually unnatural, reducing the invisibility effect.
By generating natural camouflage patterns, using Tyson polygons to segment two-dimensional images and calculate color probability based on the distance between pixel points and control points, color sampling method is used to color pixel points, generate camouflage patterns that are close to the real ones, combining three-dimensional mannequin and Deep learning detectors optimize camouflage patterns for improved stealth.
The generated camouflage pattern is more visually natural, improving the stealth effect, and effectively evading the detection of pedestrian detectors at multiple angles, which is better than the stealth effect of existing methods.
Smart Images

Figure CN2024072733_21112024_PF_FP_ABST
Abstract
Description
Camouflage pattern generation method, device, electronic device and storage medium
[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on May 16, 2023, with application number 202310552626.2 and invention name “A camouflage pattern generation method, device, electronic device and storage medium”, the entire contents of which are incorporated by reference into this application. Technical Field
[0002] The present application relates to the field of computer vision, and in particular to a camouflage pattern generation method, device, electronic device and storage medium. Background Art
[0003] Pedestrian detection uses computers to determine the location of pedestrians in a scene, typically using a rectangular box to represent the area within each pedestrian. As an automatic image localization algorithm, pedestrian detection has important applications in areas such as intelligent camera surveillance, autonomous driving, and intelligent robotics. Currently, pedestrian detection algorithms typically use deep neural networks (DNNs) to determine the position and size of the rectangular box corresponding to each pedestrian. Pedestrian detection can be performed using a pedestrian detector, which leverages computer vision techniques to determine the presence and location of pedestrians in an image or video sequence. Pedestrian detectors can be deep learning detectors based on DNNs. However, DNNs have been found to be susceptible to certain perturbations, resulting in false or missed detections. These perturbations, known as adversarial examples, exist not only in the digital world but also in the physical world. Such adversarial examples can help researchers evade pedestrian detectors. Specifically, when a pedestrian is within the detector's detection range, such adversarial examples can prevent the detector from detecting the presence of a pedestrian. Adversarial examples can also be used to test the robustness of pedestrian detectors. Currently, camouflage methods used to evade pedestrian detectors in the physical world often use patch patterns or textures composed of repeating patterns to create adversarial examples. Specifically, one or more square patterns are affixed to clothing, causing them to interfere with the DNN's output after being received by a camera. However, patch patterns only work when facing the camera directly, and these patterns often appear unnatural, especially when covering the entire body. They are visually conspicuous and thus reduce the camouflage effect.
[0004] Summary of the Invention
[0005] In view of this, the present application proposes a camouflage pattern generation method, device, electronic device and storage medium, which can generate natural camouflage patterns, thereby improving the stealth effect.
[0006] According to one aspect of the present application, a camouflage pattern generation method is provided, comprising: obtaining a two-dimensional image, N colors, and M×N control points; wherein each of the N colors corresponds to M control points, and different colors correspond to different control points; the M×N control points are used to segment the two-dimensional image into M×N polygons; N and M are integers not less than 1; obtaining a first probability that the first pixel takes the first color based on the coordinates of a first control point corresponding to a first color, the coordinates of a first pixel, and a first parameter; wherein the first probability is positively correlated with a first distance, the first distance represents the distance between the first pixel and the first control point, the first parameter represents a smoothing radius of the polygon, the first pixel is any pixel in the two-dimensional image, the first color is any color of the N colors, and the first control point is any control point of the M×N control points; determining the color of the first pixel based on the probability that the first pixel takes each of the N colors, and generating a camouflage pattern based on the colors of each pixel in the two-dimensional image.
[0007] In one possible implementation, obtaining a first probability that the first pixel point takes the first color based on the coordinates of the first control point corresponding to the first color, the coordinates of the first pixel point, and the first parameter includes: solving the first probability using a first function based on the coordinates of the first control point corresponding to the first color, the coordinates of the first pixel point, and the first parameter; wherein the first function represents the probability that the target pixel point takes the target color, and is positively correlated with the distance between the target pixel point and the control point corresponding to the target color.
[0008] In one possible implementation, determining the color of the first pixel based on the probability that the first pixel takes each of the N colors, and generating a camouflage pattern based on the color of each pixel in the two-dimensional image includes: solving the color of the first pixel using a second function based on the N colors, the first probability, a first variable, and a temperature parameter; wherein the second function represents the relationship between the color of the target pixel and the probability that the target pixel takes each of the N colors; the first variable includes a random variable that satisfies a standard Gumbel distribution or a random variable that satisfies a (0,1) uniform distribution; and coloring each pixel according to the color of each pixel to generate the camouflage pattern.
[0009] In a possible implementation manner, the first function and the second function are differentiable functions.
[0010] In one possible implementation, the method further includes: generating an image of a pedestrian wearing clothing with the camouflage pattern based on the camouflage pattern and the three-dimensional human body model; inputting the pedestrian image into a deep learning detector for target detection to obtain a first loss function; the first loss function is used to measure the ability of the pedestrian image to be invisible to the deep learning detector; updating the camouflage pattern based on the first loss function and the second loss function; the second loss function is used to measure the concentration of the M×N control points.
[0011] In one possible implementation, the first variable includes a random variable that satisfies a standard Gumbel distribution; updating the camouflage pattern according to the first loss function and the second loss function includes: updating the coordinates of the M×N control points according to the first loss function and the second loss function; and performing the above-mentioned operations of obtaining the first probability that the first pixel point takes the first color and subsequent operations based on the updated coordinates of the M×N control points to obtain a newly generated camouflage pattern.
[0012] In one possible implementation, the first variable includes a random variable that satisfies a (0,1) uniform distribution; updating the camouflage pattern according to the first loss function and the second loss function includes: updating the coordinates of the M×N control points and the first variable according to the first loss function and the second loss function; performing the above-mentioned operations of obtaining the first probability of the first pixel point taking the first color and subsequent operations according to the updated coordinates of the M×N control points and the updated first variable to obtain a newly generated camouflage pattern.
[0013] According to another aspect of the present application, a camouflage pattern generating device is provided, comprising: an acquisition module for acquiring a two-dimensional image, N colors, and M×N control points; wherein each of the N colors corresponds to M control points, and different colors correspond to different control points; the M×N control points are used to segment the two-dimensional image into M×N polygons; N and M are integers not less than 1; a probability module for obtaining a first probability that the first pixel takes the first color based on the coordinates of a first control point corresponding to a first color, the coordinates of a first pixel, and a first parameter; wherein the first probability is positively correlated with a first distance, the first distance representing the distance between the first pixel and the first control point, the first parameter representing a smoothing radius of the polygon, the first pixel being any pixel in the two-dimensional image, the first color being any color of the N colors, and the first control point being any control point of the M×N control points; and a generation module for determining the color of the first pixel based on the probability that the first pixel takes each of the N colors, and generating a camouflage pattern based on the colors of each pixel in the two-dimensional image.
[0014] According to another aspect of the present application, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above-mentioned camouflage pattern generation method when executing the instructions stored in the memory.
[0015] According to another aspect of the present application, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions implement the above-mentioned camouflage pattern generation method when executed by a processor.
[0016] According to another aspect of the present application, a computer program product is provided, comprising a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above-mentioned camouflage pattern generation method.
[0017] The camouflage pattern generation method provided by the present application divides a two-dimensional image into multiple polygons according to N colors and control points corresponding to each color. The control points correspond to the polygons one-to-one, and the control points can control the shape of the polygons. These polygons can be regarded as approximate Thiessen polygons; the probability of each pixel taking each color is calculated according to the distance between each pixel point and the control point corresponding to each color and the smoothing radius of the polygon, and the color of each pixel point is determined according to the probability, and each pixel point is colored to generate a camouflage pattern. This coloring method can be called color sampling; in this way, the present application generates polygons corresponding to camouflage colors by using approximate Thiessen polygons, and colors the pixels by color sampling, so that the edges of the polygons can produce a fragmented effect, thereby being closer to real digital camouflage patterns; in some examples, the shape of the polygons and the color of the pixels can be adjusted by adjusting the coordinates of the control points to avoid unnatural shapes and overly bright colors, thereby generating a more natural and easy-to-optimize camouflage pattern, producing a more natural visual effect, and thus improving the stealth effect.
[0018] Other features and aspects of the present application will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the application and, together with the description, serve to explain the principles of the application.
[0020] FIG1 shows a flow chart of a camouflage pattern generating method according to an embodiment of the present application.
[0021] FIG2 is a schematic diagram showing a camouflage pattern generated by a camouflage pattern generating method according to an embodiment of the present application.
[0022] FIG3 is a schematic diagram showing a camouflage pattern generation process according to an embodiment of the present application.
[0023] FIG4 shows a schematic structural diagram of a camouflage pattern generating device according to an embodiment of the present application.
[0024] FIG5 shows a block diagram of an electronic device 1900 according to an embodiment of the present application. DETAILED DESCRIPTION
[0025] Various exemplary embodiments, features, and aspects of the present application will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.
[0026] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.
[0027] In addition, numerous specific details are provided in the detailed description below to better illustrate the present application. Those skilled in the art will appreciate that the present application can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art are not described in detail in order to highlight the main purpose of the present application.
[0028] Currently, adversarial examples (i.e., adversarial patterns) are typically constructed using patch or tile patterns to disguise pedestrian detectors in the physical world. These patterns are generated by minimizing an objective function: the average objectness score of each bounding box in the DNN output. There are several ways to generate adversarial patterns in related technologies: 1. On an existing dataset, a patch image is pasted at a certain position of the annotation box at a certain ratio according to the position of the annotation box to simulate the situation when a pedestrian wears the corresponding pattern and stands in front of the camera with the adversarial pattern facing the camera; 2. The tester wears a piece of clothing with checkerboard points marked at specific positions in advance, and shoots a video of the front of the tester facing the camera to calibrate the position of the plane grid points corresponding to the checkerboard in the digital world in the real world, and uses the Thin Plate Spline (TPS) method to simulate the deformation of clothes in the real world; 3. Optimize a pattern that can be densely laid to generate adversarial textures, and produce adversarial clothing that is effectively invisible at multiple angles by cutting and printing fabrics with this adversarial texture; 4. Use a pre-trained generative network to optimize an adversarial pattern that looks more natural; 5. In a 3D simulation scene, paste the pattern on a 3D human model and render to simulate the real-world situation.
[0029] Current methods for cloaking pedestrian detectors in the physical world rely on optimizing adversarial patterns, generally without restricting the color and shape spaces of the generated patterns. Consequently, the adversarial patterns generated by various methods in the aforementioned related art often exhibit overly bright colors and unnatural shapes, making them visually overly noticeable to humans. Among the methods for generating adversarial patterns in the aforementioned related art, only method 4 considers the naturalness of the adversarial pattern. However, the adversarial pattern generated by this method is only suitable for placement in a localized location. For example, the adversarial pattern generated by this method is a dog's head, which is only suitable for placement on the chest, thus failing to achieve cloaking from different angles. Covering the entire body with this pattern would make the entire garment appear unnatural. For example, a garment covered with dog's head would appear unnatural. Exploring more natural adversarial patterns has significant practical significance and application value for the safety and reliability of pedestrian detectors.
[0030] For example, a camouflage pattern can be used as a countermeasure pattern. For example, a camouflage pattern can be generated using Thiessen polygons, which are composed of several camouflage colors. Thiessen polygons can be used to divide a two-dimensional plane into multiple polygons using multiple control points. Each polygon contains one of the control points, making the distance from any position within the polygon to this control point closer than the distance to other control points. By optimizing the coordinates of these control points, the shape of the Thiessen polygons can be optimized. However, the shapes of these Thiessen polygons are not differentiable functions of the control points, which makes it difficult to quickly optimize the Thiessen polygon shapes, and thus the generated camouflage polygon patterns.
[0031] In view of this, an embodiment of the present application provides a camouflage pattern generation method that can generate a natural camouflage pattern; in some examples, by adjusting the coordinates of the control points, a more natural and easy-to-optimize camouflage pattern can be generated, thereby improving the stealth effect.
[0032] FIG1 shows a flow chart of a method for generating a camouflage pattern according to an embodiment of the present application. The method may be executed by a device with processing capabilities, such as a processor or a server. As shown in FIG1 , the method may include:
[0033] S101. Obtain a two-dimensional image, N colors, and M×N control points; wherein each of the N colors corresponds to M control points, and different colors correspond to different control points; the M×N control points are used to divide the two-dimensional image into M×N polygons; N and M are integers not less than 1.
[0034] For example, a two-dimensional image may be obtained, which may be a blank image and may be denoted as R 2 As an example, the two-dimensional image may be a two-dimensional image with a length of H and a width of W.
[0035] For example, N colors can be obtained as camouflage colors, and these camouflage colors can be recorded as C = {c i =(R i ,G i ,B i )|i=1,…,N}, where C represents the camouflage color set (i.e., a set of N colors), c i represents the i-th camouflage color in C, (R i ,G i ,B i ) indicates camouflage color c i The red, green, and blue (RGB) color components, R i Represents the red component, G i Represents the green component, B iThe value of N can be set by those skilled in the art according to actual needs.
[0036] For example, for each of N camouflage colors, M control points can be selected within the 2D image to correspond to it. For each of the N camouflage colors, M×N control points can be selected within the 2D image to correspond to it, with different control points corresponding to different colors. These M×N control points can be used to divide the 2D image into M×N polygons, each containing one of the control points. That is, each control point corresponds to a polygon, and there is a one-to-one correspondence between M×N control points and M×N polygons. For any polygon, the distance to the corresponding control point is closer than to any other control point within the polygon. The value of M can be set by those skilled in the art based on practical needs.
[0037] As an example, M×N control points can be randomly selected on a two-dimensional image, and then M control points can be randomly assigned to each of the N camouflage colors. As another example, M×N control points can be evenly selected at a preset interval on a two-dimensional image, and then M control points can be randomly assigned to each of the N camouflage colors.
[0038] In one embodiment, a blank two-dimensional image with a length of H and a width of W can be obtained, and three colors c1, c2, and c3 can be selected as camouflage colors (i.e., N=3). M can be set to 10, that is, for each camouflage color, 10 control points can be selected in the two-dimensional image to correspond to it. 30 control points can be randomly selected in the two-dimensional image, denoted as a1 to a 30 . The control points a1~a 10 Assigned to camouflage color c1 as the control point corresponding to camouflage color c1; control point a 11 ~a 20 Assigned to camouflage color c2 as the control point corresponding to camouflage color c2; control point a 21 ~a 10 Assigned to camouflage color c3 as the control point corresponding to camouflage color c3.
[0039] S102. Obtain a first probability that the first pixel takes the first color based on the coordinates of the first control point corresponding to the first color, the coordinates of the first pixel, and a first parameter; wherein the first probability is positively correlated with a first distance, the first distance represents the distance between the first pixel and the first control point, the first parameter represents a smoothing radius of the polygon, the first pixel is any pixel in the two-dimensional image, the first color is any color among the N colors, and the first control point is any control point among the M×N control points.
[0040] In one possible implementation, obtaining the first probability that the first pixel point takes the first color based on the coordinates of the first control point corresponding to the first color, the coordinates of the first pixel point, and the first parameter may include: solving the first probability using a first function based on the coordinates of the first control point corresponding to the first color, the coordinates of the first pixel point, and the first parameter; wherein the first function represents the probability that the target pixel point takes the target color, and is positively correlated with the distance between the target pixel point and the control point corresponding to the target color.
[0041] For example, after selecting M×N control points in the two-dimensional image, the coordinates of the M×N control points in the two-dimensional image can be expressed as {b ij ∈R 2 ,i=1,2,…,N,j=1,2,…,M}, where b ij Represents the coordinates of the j-th control point of the i-th camouflage color (that is, the coordinates of the first control point corresponding to the first color).
[0042] For example, the first function may be a differentiable function, and the first function may be used to approximate the Thiessen polygon. The first function may be expressed by the following formula:
[0043] Wherein, x represents the pixel point in the two-dimensional image (i.e., the first pixel point), Indicates that the color of pixel x is c k (i.e. the kth camouflage color) probability (i.e. the first probability), Indicates that pixel x corresponds to camouflage color c i (i.e. the weight of the i-th camouflage color), Indicates that pixel x corresponds to camouflage color c k The weight of . It can be calculated by the following formula:
[0044] Among them, xb ijrepresents the distance between pixel x and the jth control point of the i-th camouflage color (i.e., the first distance), ‖·‖2 represents the 2-norm, and α represents the first parameter, i.e., the smoothing radius of the M×N polygons obtained by segmenting the two-dimensional image according to the M×N control points. α>0. As α increases, the edges of these polygons become blurrier; as α decreases, the edges of these polygons become clearer. The value of α can be set by those skilled in the art according to actual needs; for example, α can be 1. The probability of pixel x taking each of the N camouflage colors can be calculated using a first function based on the coordinates of the M×N control points, the coordinates of pixel x, and the value of α. When α approaches 0, the probability of pixel x taking one of the N camouflage colors, calculated according to the first function, approaches 1, and the probability of taking other camouflage colors approaches 0. At this point, the probability distribution calculated according to the first function approaches the probability distribution of true Thiessen polygons, and the M×N polygons obtained by segmenting the two-dimensional image according to the M×N control points approach Thiessen polygons. From the above formula, we can see that for a target pixel point x′, the target color c i The probability of the target pixel x′ and the target color c i The corresponding control point b ij The distance between them is positively correlated. When the target pixel x′ is close to the control point b ij When the target pixel x′ takes the target color c i The probability of .
[0045] S103 : Determine the color of the first pixel according to the probability of the first pixel taking each color of the N colors, and generate a camouflage pattern according to the color of each pixel in the two-dimensional image.
[0046] In one possible implementation, determining the color of the first pixel based on the probability of the first pixel taking each of the N colors, and generating a camouflage pattern based on the colors of each pixel in the two-dimensional image may include the following steps:
[0047] (1) Based on the N colors, the first probability, the first variable, and the temperature parameter, a second function is used to solve the color of the first pixel; wherein the second function represents the relationship between the color of the target pixel and the probability of the target pixel taking each of the N colors; the first variable includes a random variable that satisfies a standard Gumbel distribution or a random variable that satisfies a (0,1) uniform distribution.
[0048] For example, for a two-dimensional image with a length of H and a width of W, a probability distribution graph P with a size of N×H×W can be calculated according to the first function. The probability distribution graph P can be recorded as in, Indicates that the pixel point x in the two-dimensional plane takes the camouflage color c i The probability distribution graph P can be used to color each pixel in the two-dimensional plane. Coloring the pixel x according to the probability distribution graph P is equivalent to:
[0049] Among them, c (x) Indicates the color of pixel x, c k Indicates the kth camouflage color among N camouflage colors, k represents The value of parameter i when the maximum value is obtained, g i ~Gumbel(0,1) is a random variable that satisfies the standard Gumbel distribution. Since the argmax function in the above formula is not differentiable, it can be approximated by the Softmax function to obtain a second function. The second function can be a differentiable function and can be expressed as follows:
[0050] Among them, c i Indicates the i-th camouflage color among N camouflage colors, g i represents the first variable (i.e., a random variable satisfying the standard Gumbel distribution), τ represents the temperature parameter, τ>0, and the value of τ can be set by those skilled in the art according to actual needs; for example, τ can be set to 0.3. It can be expressed as follows:
[0051] When τ approaches 0, the approximation approaches accuracy, that is, lim τ→0 c (x) =c k As can be seen from the above formula, the second function can express the relationship between the color of the target pixel point x′ and the probability of the target pixel point x′ taking each of the N colors.
[0052] For example, the color of the pixel point x can be calculated using the second function based on N camouflage colors, the probability of the pixel point x taking each of the N camouflage colors, the values of the first variable and the temperature parameter.
[0053] As an example, the probability of the target pixel x′ taking each of N camouflage colors is The color of the target pixel x′ calculated by the second function is the superposition of the RGB values of N camouflage colors; if the target pixel x′ takes the camouflage color c k Probability Maximum, then when τ approaches 0, the color c of the target pixel x′ calculated according to the second function is (x′)Close to camouflage color c k .
[0054] For example, an optimizable variable g′ can be used i Instead of the random variable g i As the first variable. Gumbel random variable g i Equivalent to g i = -log(-logu), where u~Uniform(0,1) is a uniform distribution between (0,1).
[0055] in, can remain unchanged during the optimization process, It can be an optimizable variable, and λ is a hyperparameter that controls the ratio of optimizable variables. According to the above formula, the random variable g′ that satisfies the uniform distribution based on (0,1) can be obtained. i , you can update Update the value of g′ i , g′ i Can replace g in the second function i As the first variable, the second function can be expressed as follows:
[0056] In this way, the optimizable random variable g′ is obtained based on the (0,1) uniform distribution i , the optimizable parameter space can be expanded, thereby improving the stealth effect of the generated camouflage pattern.
[0057] (2) Coloring each pixel according to the color of each pixel to generate the camouflage pattern.
[0058] For example, each pixel in the two-dimensional image obtained in step (1) can be colored to generate a camouflage pattern. This method of coloring each pixel can be called color sampling.
[0059] As an example, the first control point b of the camouflage color c1 11 In the two-dimensional image, the second control point b of the camouflage color c2 corresponds to a polygon l1. 22 In the two-dimensional image, corresponding to a polygon l2, for the pixel point x in l1, if the pixel point x is closer to the control point b 11, the closer the color of pixel x is to c1; for pixel y at the boundary of l1 and l2, its color may be closer to c1 or closer to c2. Figures 2(a)-(c) show schematic diagrams of generating a camouflage pattern according to a camouflage pattern generation method according to an embodiment of the present application. Figure 2(a) shows the polygons corresponding to the control points in a two-dimensional image. Each polygon contains a control point, and the control points correspond one-to-one with the polygons. For example, the control point S1 in Figure 2(a) corresponds to the polygon L1 corresponding to the control point S1; Figure 2(b) shows the camouflage color corresponding to each control point. The color of each polygon in Figure 2(b) represents the color corresponding to the control point in the polygon; Figure 2(c) shows the camouflage pattern generated by the camouflage pattern generation method according to an embodiment of the present application. As shown in Figures 2(a)-(c), when the pixel x is close to the camouflage color c i The control point b ij When the color of pixel x is closer to c i The color differences between pixels near the edges of polygons are large, creating a fragmented effect. By coloring the pixels through color sampling, the edges of the polygons corresponding to the control points can be fragmented, mimicking a realistic digital camouflage pattern. This makes the generated camouflage pattern more natural and improves stealth.
[0060] In one embodiment, if the first variable is a random variable g that satisfies the standard Gumbel distribution i , the first function can be updated by updating the coordinates of the M×N control points, thereby updating the generated camouflage pattern. In another embodiment, if the first variable is a variable g′ obtained based on a (0,1) uniform distribution i , by simultaneously updating the coordinates of the M×N control points and the variable g′ i To update the first and second functions, the generated camouflage pattern can be updated. Since the first and second functions are differentiable functions, the gradient backpropagation method can be used to update the coordinates of the M×N control points and the variable g′ i , thereby increasing the update speed and enabling rapid optimization of the generated camouflage pattern. By adjusting the coordinates of the control points and the first variable, the polygon shape and pixel color can be adjusted, avoiding unnatural shapes and overly bright colors, thereby generating a more natural camouflage pattern that is easier to optimize.
[0061] The embodiment of the present application uses a first function to approximate the Thiessen polygon, calculates the probability of each pixel in the two-dimensional image taking each color of N colors, and determines the color of each pixel according to the second function, so as to generate a more natural camouflage pattern; and the first function and the second function are differentiable functions, and the entire camouflage pattern generation process is represented by a differentiable function, which is convenient for optimization using the gradient backpropagation method, so that the generated camouflage pattern can be quickly optimized, thereby achieving a better stealth effect.
[0062] In a possible implementation, the method may further include the following steps:
[0063] (1) Based on the camouflage pattern and the three-dimensional human body model, an image of a pedestrian wearing the camouflage pattern clothing is generated.
[0064] For example, the generated camouflage pattern can be used as a texture map for a 3D clothing model to obtain a 3D camouflage clothing model. The 3D camouflage clothing model can be combined with a 3D human model to obtain a pedestrian model wearing the camouflage clothing. As an example, the 3D camouflage clothing model can be combined with a 3D human model using PyTorch3D software to obtain a pedestrian model wearing the camouflage clothing.
[0065] For example, a renderer can be used to render a pedestrian model wearing camouflage clothing to generate an image of the pedestrian wearing the camouflage clothing. For example, the renderer can be a differentiable renderer, which can calculate derivatives of the rendering process. Using a differentiable renderer facilitates the use of gradient backpropagation to optimize the camouflage pattern, thereby increasing optimization speed.
[0066] For example, by rotating a pedestrian model wearing camouflage clothing and rendering it at different angles using a renderer, images of the pedestrian wearing camouflage clothing at different angles can be obtained, thereby simulating pedestrian images at different angles detected by a pedestrian detector. This allows the camouflage clothing to achieve a stealth effect at different angles for the pedestrian detector.
[0067] For example, a background dataset can be pre-acquired, which may include background images. The obtained image of a pedestrian wearing camouflage clothing can be synthesized with the background images in the background dataset to obtain images of pedestrians in different backgrounds. This can simulate images of pedestrians detected by a pedestrian detector in different scenarios. This can improve the stealth effect of camouflage clothing in different scenarios.
[0068] (2) Inputting the pedestrian image into a deep learning detector for target detection to obtain a first loss function; the first loss function is used to measure the ability of the pedestrian image to be invisible to the deep learning detector.
[0069] For example, an image of a pedestrian wearing camouflage clothing can be input into a deep learning detector (i.e., a pedestrian detector), and the deep learning detector is used to perform target detection to obtain a first loss function. It can be expressed as the following formula:
[0070] Where I represents the set of all prediction boxes output by the deep learning detector, X represents the set of all pedestrian images input to the deep learning detector, x represents the pedestrian image in X, and gt (x) represents the ground-truth detection box in the pedestrian image x. Represents the set of detection boxes output by the deep learning detector for the pedestrian image x, Indicates that the real detection box gt (x) The detection box with the largest Intersection Over Union (IoU) ratio, Represents the detection box The confidence score of . The first loss function It can measure the ability of pedestrian images to be invisible to deep learning detectors. The smaller the value of , the stronger the ability of the pedestrian image to hide from the deep learning detector, that is, the better the stealth effect of the corresponding camouflage pattern in the pedestrian image. In order to improve the ability of the pedestrian image to hide from the deep learning detector, the first loss function can be minimized. The value of .
[0071] (3) updating the camouflage pattern according to the first loss function and the second loss function; the second loss function is used to measure the concentration degree of the M×N control points.
[0072] For example, the second loss function It can be expressed as the following formula:
[0073] in, Represents the coordinates of the k1th control point of the jth camouflage color, represents the coordinates of the k2th control point of the jth camouflage color, ‖·‖2 represents the 2-norm, and σ represents the parameter that controls the strength of the second loss function. The value of σ can be set by those skilled in the art according to actual needs; for example, σ can be set to 1. Second loss function It can measure the concentration of M×N control points. The smaller the value of , the lower the concentration of the M×N control points. The second loss function can ensure the stability of camouflage pattern generation and prevent the control points from being overly concentrated and resulting in too small camouflage polygons.
[0074] In one embodiment, the first variable includes a random variable g that satisfies a standard Gumbel distribution. i Updating the camouflage pattern according to the first loss function and the second loss function may include: updating the coordinates of the M×N control points according to the first loss function and the second loss function; and performing the above-mentioned operations of obtaining the first probability of the first pixel point taking the first color and subsequent operations according to the updated coordinates of the M×N control points to obtain a newly generated camouflage pattern.
[0075] In another embodiment, the first variable includes a random variable g′ satisfying the (0,1) Uniform distribution. i Updating the camouflage pattern according to the first loss function and the second loss function may include: updating the coordinates of the M×N control points and the first variable according to the first loss function and the second loss function; performing the above-mentioned operations of obtaining the first probability that the first pixel point takes the first color and subsequent operations according to the updated coordinates of the M×N control points and the updated first variable to obtain a newly generated camouflage pattern.
[0076] Exemplarily, a total loss function may be determined based on the first loss function and the second loss function; and the camouflage pattern may be updated based on the total loss function.
[0077] As an example, the first loss function can be and the second loss function Perform weighted summation to obtain the total loss function It can be expressed as follows:
[0078] Among them, the parameter α con is the weight parameter used to measure the first loss function and the second loss function The proportion of. In the total loss function When the value of is lower than the preset threshold, the camouflage pattern is stopped from being updated to obtain the final camouflage pattern.
[0079] In an embodiment of the present application, the generated camouflage pattern is used as a texture map of a three-dimensional clothing model and merged with a three-dimensional human body model. A renderer is used to generate an image of a pedestrian wearing clothing with the camouflage pattern. A deep learning detector is used to perform target detection on the pedestrian image. The camouflage pattern is updated according to a first loss function and a second loss function. The camouflage pattern can be optimized through three-dimensional modeling to improve the stealth effect of the camouflage pattern, so that clothing using the final generated camouflage pattern can be invisible to pedestrian detectors based on deep neural networks at multiple angles.
[0080] FIG3 is a schematic diagram showing a camouflage pattern generation process according to an embodiment of the present application. As shown in FIG3 , a two-dimensional plane, N camouflage colors, and M×N control points can be selected. The probability of each pixel in the two-dimensional plane taking each of the N camouflage colors can be calculated according to the first function, thereby obtaining a probability distribution diagram. According to the probability distribution diagram and N kinds of camouflage colors, use the second function to calculate the color of each pixel, and color each pixel by color sampling to generate a camouflage pattern The camouflage pattern can be The texture map of the three-dimensional clothing model is used, and the three-dimensional clothing model is then combined with the three-dimensional human body model to obtain a pedestrian model. By rotating the pedestrian model and rendering it with a differentiable renderer, the pedestrian image wearing camouflage pattern clothing detected by the pedestrian detector at multiple angles can be simulated and generated; the pedestrian image can be synthesized with the background image in the background dataset to obtain pedestrian images in different scenes; the synthesized image can be input into the deep learning detector for target detection, and the first loss function obtained is used. and the second loss function Perform weighted summation to obtain the total loss function According to the total loss function Updated camouflage pattern exist When the value of falls below a preset threshold, the camouflage pattern is updated, and the final camouflage pattern is obtained. In this way, the embodiments of the present application can generate more natural camouflage patterns with faster optimization speeds and better stealth effects. Experiments have shown that the camouflage patterns generated by the embodiments of the present application are more natural than the adversarial patterns generated by existing methods. Camouflage clothing using the camouflage patterns generated by the embodiments of the present application can evade pedestrian detectors at different angles relative to the camera in the physical world, and the stealth effects at multiple angles relative to the camera exceed those of existing methods.
[0081] FIG4 shows a schematic structural diagram of a camouflage pattern generating device according to an embodiment of the present application. As shown in FIG4 , the device may include: an acquisition module 401, configured to acquire a two-dimensional image, N colors, and M×N control points; wherein each of the N colors corresponds to M control points, and different colors correspond to different control points; the M×N control points are used to divide the two-dimensional image into M×N polygons; N and M are integers not less than 1; a probability module 402, configured to obtain, based on the coordinates of the first control point corresponding to the first color, the coordinates of the first pixel point, and the first parameter, the probability of the first pixel point taking the first color. a first probability of a color; wherein the first probability is positively correlated with a first distance, the first distance represents the distance between the first pixel point and the first control point, the first parameter represents the smoothing radius of the polygon, the first pixel point is any pixel point in the two-dimensional image, the first color is any color of the N colors, and the first control point is any control point of the M×N control points; a generation module 403 is used to determine the color of the first pixel point based on the probability of the first pixel point taking each color of the N colors, and generate a camouflage pattern based on the colors of each pixel point in the two-dimensional image.
[0082] In one possible implementation, the probability module 402 is further used to: solve the first probability using a first function based on the coordinates of the first control point corresponding to the first color, the coordinates of the first pixel point and the first parameter; wherein the first function represents the probability of the target pixel point taking the target color, and is positively correlated with the distance between the target pixel point and the control point corresponding to the target color.
[0083] In one possible implementation, the generation module 403 is further used to: solve the color of the first pixel point using a second function based on the N colors, the first probability, the first variable and the temperature parameter; wherein the second function represents the relationship between the color of the target pixel point and the probability of the target pixel point taking each of the N colors; the first variable includes a random variable that satisfies the standard Gumbel distribution or a random variable that satisfies the (0,1) uniform distribution; and color each pixel point according to the color of each pixel point to generate the camouflage pattern.
[0084] In a possible implementation manner, the first function and the second function are differentiable functions.
[0085] In one possible implementation, the device also includes: a pedestrian image generation module, used to generate an image of a pedestrian wearing clothes with the camouflage pattern based on the camouflage pattern and the three-dimensional human body model; a detection module, used to input the pedestrian image into a deep learning detector for target detection to obtain a first loss function; the first loss function is used to measure the ability of the pedestrian image to be invisible to the deep learning detector; an update module, used to update the camouflage pattern according to the first loss function and the second loss function; the second loss function is used to measure the concentration of the M×N control points.
[0086] In one possible implementation, the first variable includes a random variable that satisfies a standard Gumbel distribution; the update module is further used to: update the coordinates of the M×N control points according to the first loss function and the second loss function; and perform the above-mentioned operations of obtaining the first probability of the first pixel point taking the first color and subsequent operations according to the updated coordinates of the M×N control points to obtain a newly generated camouflage pattern.
[0087] In one possible implementation, the first variable includes a random variable that satisfies a (0,1) uniform distribution; the update module is further used to: update the coordinates of the M×N control points and the first variable according to the first loss function and the second loss function; and perform the above-mentioned operations of obtaining the first probability of the first pixel point taking the first color and subsequent operations according to the updated coordinates of the M×N control points and the updated first variable to obtain a newly generated camouflage pattern.
[0088] In some embodiments, the functions or modules included in the device provided in the embodiments of the present application can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.
[0089] The camouflage pattern generating device provided in the implementation of the present application divides a two-dimensional image into multiple polygons according to N colors and control points corresponding to each color. The control points correspond to the polygons one-to-one, and the control points can control the shape of the polygons. These polygons can be regarded as approximate Thiessen polygons; the probability of each pixel taking each color is calculated according to the distance between each pixel point and the control point corresponding to each color and the smoothing radius of the polygon, and the color of each pixel point is determined according to the probability, and each pixel point is colored to generate a camouflage pattern. This coloring method can be called color sampling; in this way, the present application generates polygons corresponding to camouflage colors by using approximate Thiessen polygons, and colors the pixels by color sampling, so that the edges of the polygons can produce a fragmented effect, thereby being closer to real digital camouflage patterns; in some examples, the shape of the polygons and the color of the pixels can be adjusted by adjusting the coordinates of the control points to avoid unnatural shapes and overly bright colors, thereby generating a more natural and easy-to-optimize camouflage pattern, producing a more natural visual effect, and thus improving the stealth effect. In other examples, by using a first function to approximate the Thiessen polygons, calculating the probability of each pixel in the two-dimensional image taking each of N colors, and determining the color of each pixel according to a second function, a more natural camouflage pattern can be generated; and the first function and the second function are differentiable functions, and the entire camouflage pattern generation process is represented by a differentiable function, which facilitates optimization using the gradient backpropagation method, so that the generated camouflage pattern can be quickly optimized, thereby achieving a better stealth effect.
[0090] An embodiment of the present application also proposes an electronic device, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to implement the camouflage pattern generation method of Figure 1 above when executing the instructions stored in the memory.
[0091] Figure 5 shows a block diagram of an electronic device 1900 according to an embodiment of the present application. For example, device 1900 can be provided as a server or terminal device. Referring to Figure 5 , device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions executable by processing component 1922, such as applications. The applications stored in memory 1932 can include one or more modules, each corresponding to a set of instructions. Furthermore, processing component 1922 is configured to execute instructions to perform the camouflage pattern generation method described in Figure 1 .
[0092] The device 1900 may also include a power supply component 1926 configured to perform power management of the device 1900, a wired or wireless network interface 1950 configured to connect the device 1900 to a network, and an input / output (I / O) interface 1958. The device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or the like.
[0093] The present application also provides a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implement the camouflage pattern generation method of FIG1 . The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.
[0094] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the device 1900 to implement the camouflage pattern generation method of Figure 1 above.
[0095] An embodiment of the present application also provides a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the camouflage pattern generation method of Figure 1 above.
[0096] The present application may be a system, method and / or computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present application.
[0097] A computer-readable storage medium can be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination thereof. As used herein, a computer-readable storage medium is not to be construed as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through an electrical wire.
[0098] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.
[0099] The computer program instructions for performing the operation of the present application can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data or source code or object code written in any combination of one or more programming languages, wherein the programming language includes object-oriented programming languages such as Smalltalk, C++, and conventional procedural programming languages such as "C" language or similar programming languages. Computer-readable program instructions can be executed completely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or executed completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer by any type of network including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (such as by using an Internet service provider to connect to the Internet). In certain embodiments, by utilizing the state information of computer-readable program instructions to personalize electronic circuits, such as programmable logic circuits, field programmable gate arrays (FPGAs) or programmable logic arrays (PLAs), the electronic circuits can execute computer-readable program instructions, thereby realizing various aspects of the present application.
[0100] Various aspects of the present application are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.
[0101] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.
[0102] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.
[0103] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the system, method and computer program product according to multiple embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a special hardware-based system that performs the function or action of the specification, or can be implemented by a combination of special hardware and computer instructions.
[0104] While various embodiments of the present application have been described above, the foregoing description is intended to be illustrative, non-exhaustive, and not limiting to the disclosed embodiments. Numerous modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or technological improvements in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein.
Claims
1. A camouflage pattern generation method, characterized in that: include: Obtain a two-dimensional image, N colors, and M×N control points; wherein each of the N colors corresponds to M control points, and different colors correspond to different control points; the M×N control points are used to divide the two-dimensional image into M×N polygons; N and M are integers not less than 1; According to the coordinates of the first control point corresponding to the first color, the coordinates of the first pixel point and the first parameter, a first probability that the first pixel point takes the first color is obtained; wherein the first probability is positively correlated with a first distance, the first distance represents the distance between the first pixel point and the first control point, the first parameter represents the smoothing radius of the polygon, the first pixel point is any pixel point in the two-dimensional image, the first color is any color among the N colors, and the first control point is any control point among the M×N control points; The color of the first pixel is determined according to the probability of the first pixel taking each color of the N colors, and a camouflage pattern is generated according to the color of each pixel in the two-dimensional image.
2. The method according to claim 1, characterized in that The obtaining, according to the coordinates of the first control point corresponding to the first color, the coordinates of the first pixel point and the first parameter, a first probability of the first pixel point taking the first color comprises: The first probability is solved using a first function according to the coordinates of the first control point corresponding to the first color, the coordinates of the first pixel point and the first parameter; wherein the first function represents the probability of the target pixel point taking the target color, and is positively correlated with the distance between the target pixel point and the control point corresponding to the target color.
3. The method according to claim 2, characterized in that The method of determining the color of the first pixel point according to the probability of the first pixel point taking each color of the N colors, and generating a camouflage pattern according to the color of each pixel point in the two-dimensional image includes: The color of the first pixel is solved by using a second function according to the N colors, the first probability, the first variable and the temperature parameter; wherein the second function represents the relationship between the color of the target pixel and the probability that the target pixel takes each of the N colors; the first variable includes a random variable that satisfies a standard Gumbel distribution or a random variable that satisfies a (0,1) uniform distribution; The camouflage pattern is generated by coloring each pixel according to the color of each pixel.
4. The method according to claim 3, characterized in that The first function and the second function are differentiable functions.
5. The method according to claim 3, characterized in that: The method further comprises: Generating an image of a pedestrian wearing the camouflage pattern clothing according to the camouflage pattern and the three-dimensional human body model; Inputting the pedestrian image into a deep learning detector for target detection to obtain a first loss function; the first loss function is used to measure the ability of the pedestrian image to be invisible to the deep learning detector; The camouflage pattern is updated according to the first loss function and the second loss function; the second loss function is used to measure the concentration degree of the M×N control points.
6. The method according to claim 5, characterized in that The first variable includes a random variable that satisfies a standard Gumbel distribution; The updating of the camouflage pattern according to the first loss function and the second loss function comprises: According to the first loss function and the second loss function, updating the coordinates of the M×N control points; According to the updated coordinates of the M×N control points, the above-mentioned first probability of the first pixel point taking the first color and subsequent operations are performed to obtain a newly generated camouflage pattern.
7. The method according to claim 5, characterized in that The first variable includes a random variable that satisfies a (0,1) Uniform distribution; The updating of the camouflage pattern according to the first loss function and the second loss function comprises: According to the first loss function and the second loss function, updating the coordinates of the M×N control points and the first variable; According to the updated coordinates of the M×N control points and the updated first variable, the above-mentioned first probability of the first pixel point taking the first color and subsequent operations are performed to obtain a newly generated camouflage pattern.
8. A camouflage pattern generating device, characterized in that: include: An acquisition module, used for acquiring a two-dimensional image, N colors, and M×N control points; wherein each of the N colors corresponds to M control points, and different colors correspond to different control points; the M×N control points are used to divide the two-dimensional image into M×N polygons; N and M are integers not less than 1; a probability module, configured to obtain a first probability that the first pixel takes the first color according to the coordinates of a first control point corresponding to the first color, the coordinates of a first pixel, and a first parameter; wherein the first probability is positively correlated with a first distance, the first distance represents the distance between the first pixel and the first control point, the first parameter represents the smoothing radius of the polygon, the first pixel is any pixel in the two-dimensional image, the first color is any color among the N colors, and the first control point is any control point among the M×N control points; A generating module is used to determine the color of the first pixel point according to the probability of the first pixel point taking each color of the N colors, and to generate a camouflage pattern according to the color of each pixel point in the two-dimensional image.
9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method described in any one of claims 1 to 7 when executing the instructions stored in the memory.
10. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.
Citation Information
Patent Citations
Camouflage patch generation method for confronting intelligent reconnaissance identification
CN113469873A
Adversarial sample generation method for unmanned aerial vehicle image target detection
CN113643278A
Infrared omnidirectional stealth method based on gray parameter reforming
CN116091670A
Camouflage pattern generation method and device, electronic equipment and storage medium
CN116580119A
Detecting hybdrid-distance adversarial patches
US20230005111A1
Cited By
Deformed camouflage spot generation method based on superpixel segmentation and fuzzy C-means clustering
CN119991839A
