Environment texture data processing method and system and camouflage pattern generation method and system

Abstract

The invention relates to the technical field of image processing, and provides an environment texture data processing method and system and a camouflage pattern generation method and system.The environment texture data processing method comprises the following steps that S11, an environment image is obtained; s12, segmenting a foreground object and a background object of the environment image to form a single-channel image; and S13, extracting background features according to the environment image and the single-channel image, and reconstructing an environment texture map based on the background features. According to the scheme, interference of foreground objects is eliminated, background patterns are better fused, and the generation quality of camouflage patterns is improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a data processing method and system for environmental textures, and a method and system for generating camouflage patterns. Background Technology

[0002] Camouflage patterns play an indispensable role in protecting equipment and personnel. By applying appropriate camouflage patterns, equipment and personnel can blend seamlessly into the environment, effectively deceiving the enemy. Camouflage pattern generation is a technology that creates patterns with concealment functions through manual design or automated algorithms. It mainly uses visual deception to make the target object blend into the environmental background, thereby hiding specific visible light information from detection.

[0003] The development of camouflage pattern generation technology has evolved from manual drawing to algorithm-driven methods, and from static templates to dynamic adaptation. Early camouflage pattern generation relied on natural observation and artistic design, using hand-drawn irregular color blocks to imitate the features of forests, deserts, cities, and other terrains. With breakthroughs in digital image processing technology, algorithm-based intelligent generation methods have gradually become mainstream. Their core lies in simulating the perceptual characteristics of the human visual system through mathematical modeling, achieving a non-linear fusion of target surface texture and environmental background.

[0004] Existing methods for generating camouflage patterns do not take into account the interference of foreground objects in the background image. The generated camouflage patterns will contain both the color and texture of the foreground objects, resulting in a significant reduction in the final effect of blending with the background.

[0005] The first paper, "Adaptive Camouflage Pattern Generation to Different Environments via Content-Aware Style Transfer," proposes a content-adaptive camouflage pattern generation method. The framework of this method is as follows: Figure 1 As shown, the core idea is to use a standard camouflage pattern as the content image and the environment image as the style image, fusing them through the feature space of a convolutional neural network to generate a camouflage pattern adapted to the environment. This method innovatively introduces panoramic segmentation technology to identify and eliminate objects in the environment image that are unfavorable for camouflage (such as the sky, trains, etc.). When calculating the style loss, a weighted masking process is applied to the interference region, thereby reducing the influence of foreground object textures on the generated result. However, this method cannot completely eliminate interference objects, inevitably leading to artifacts in the generated image (i.e., residual colors and textures of interference objects). This is because the method simply performs a scaling transformation on the mask image and then performs a masking operation with the feature map. Since the scaling operation of the mask image cannot strictly match the convolution operation of the feature map, this mismatch directly leads to artifacts, which is particularly noticeable when the style image has inconsistent dimensions.

[0006] The second paper, "A small-spot deformation camouflage design algorithm based on background texture matching," proposes a small-spot camouflage generation algorithm based on adversarial autoencoders, such as... Figure 2 As shown, this method extracts background texture features through adversarial training, generates irregular small spots using simple linear iterative clustering, and then uses k-means clustering to extract the dominant environmental color for color replacement of the irregular small spots. Although the generated digital camouflage blends well against static backgrounds, the spot shapes are still limited to square templates, making them easily identifiable at close range, and the method does not effectively extract environmental features.

[0007] The third paper, "An Environment-Adaptive Generator of Adversarial Patches," proposes a different approach, exploring rapid generation strategies by decoupling patterns and colors, such as... Figure 3 As shown in the figure. Research has found that the effectiveness of adversarial patch attacks primarily depends on the pattern (i.e., the relative intensity relationship between pixels) rather than specific colors. Based on this, the method first optimizes the generation of patterns with high attack performance, and then uses k-means clustering to extract the dominant color of the new environment for color replacement. While this strategy significantly improves generation efficiency, the color replacement relies on a fixed number of clusters with only three dominant colors. This limited color set leads to significant differences in color gradation between the patch and the environment. When the patch size decreases, high-frequency texture information is severely lost, resulting in a decline in the camouflage effect on small-scale targets.

[0008] In complex environment images, there are often foreground objects, such as pedestrians, vehicles, and buildings. Existing techniques for generating camouflage images for complex environments are not adaptable enough to dynamic environmental interference and cannot effectively remove the interference from foreground objects. The generated camouflage patterns inevitably retain the color and texture information of the foreground objects.

[0009] Furthermore, existing technologies cannot generate high-quality camouflage images. The camouflage image generated by the first paper contains artifacts; the camouflage images generated by the second and third papers only contain multiple discrete primary colors, which cannot cover all the color and texture information of a complex environment.

[0010] Therefore, there is a need to provide a data processing method and system for environmental textures, and a method and system for generating camouflage patterns, to eliminate interference from foreground objects, better integrate background patterns, and improve the generation quality of camouflage patterns.

[0011] The information disclosed in the background section is only intended to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0012] The main objective of this invention is to overcome the problems of interference from foreground objects and low quality in camouflage pattern generation methods, and to provide a data processing method and system for environmental textures, as well as a method and system for generating camouflage patterns, which eliminates interference from foreground objects, better integrates background patterns, and improves the generation quality of camouflage patterns.

[0013] To achieve the above objectives, the first aspect of the present invention provides a data processing method for environmental textures, comprising the following steps: S11: Acquire environmental images; S12: Segment the foreground and background objects in the environment image to form a single-channel image; S13: Extract background features from the environmental image and single-channel image, and reconstruct the environmental texture map from the background features.

[0014] According to an exemplary embodiment of the present invention, the data processing method for environmental texture further includes: S14: slicing the environmental image to form an environmental slice map; determining whether the environmental texture map is similar to the environmental image based on the environmental slice map; if so, outputting the environmental texture map; otherwise, returning to step S13 to reconstruct the environmental texture map.

[0015] According to an exemplary embodiment of the present invention, in step S12, the foreground objects and background patterns of the environment image are segmented using a SAM model.

[0016] According to an exemplary embodiment of the present invention, step S13 further includes: training the background features to conform to a normal distribution.

[0017] As a second aspect of the present invention, the present invention provides an environmental texture data processing system, comprising: an image acquisition module, an image segmentation module, and an environmental texture map reconstruction module; The image acquisition module is used to acquire environmental images; The image segmentation module is used to segment the foreground objects and background patterns of the environmental image to form a single-channel image; The environment texture map reconstruction module is used to extract background features based on the environment image and single-channel image, and reconstruct the environment texture map from the background features.

[0018] According to an exemplary embodiment of the present invention, the environmental texture data processing system further includes a first supervision module and a second supervision module; The first supervision module is used to train the background features to conform to a normal distribution; The second supervision module is used to slice the environment image to form an environment slice map; based on the environment slice map, it determines whether the environment texture map is similar to the environment image. If so, the environment texture map is output; otherwise, it returns to the environment texture map reconstruction module to reconstruct the environment texture map.

[0019] As a third aspect of the present invention, the present invention provides a method for generating camouflage patterns, comprising the following steps: S1: Reconstruct the environmental texture map according to the environmental texture data processing method described above; S2: Generate camouflage patterns based on the environment texture map.

[0020] According to an exemplary embodiment of the present invention, in step S2, generating the camouflage pattern based on the environmental texture map includes: Obtain a reference camouflage pattern and generate a camouflage pattern based on the reference camouflage pattern; The reference camouflage image, the environment texture image, and the camouflage pattern are respectively input into a multi-layer convolutional network; The first style loss function is calculated by comparing each convolutional layer of the reference camouflage image with the convolutional layer of the corresponding camouflage pattern. Compare each convolutional layer of the environment texture map with the convolutional layer of the camouflage pattern to calculate the second style loss function. Generate a new camouflage pattern based on the reference camouflage pattern, the first style loss function, and the second style loss function; The process involves repeatedly inputting the data into a multi-layer convolutional network, calculating the first and second style loss functions, generating new camouflage patterns, and using the final generated camouflage pattern as the final camouflage pattern.

[0021] According to an exemplary embodiment of the present invention, the multilayer convolutional network is VGG19.

[0022] As a fourth aspect of the present invention, the present invention provides a camouflage pattern generation system, including the aforementioned environmental texture data processing system and camouflage pattern processing system; The camouflage pattern processing system is used to generate camouflage patterns based on the environmental texture map reconstructed by the environmental texture data processing system.

[0023] The advantages of this invention are: This scheme employs dual-layer supervision to achieve adversarial training, eliminating interference from foreground objects and better integrating background patterns. It can eliminate artifacts and generate clearer, noise-free camouflage patterns that cover all color and texture information of complex environments, thereby improving the quality of camouflage pattern generation. Attached Figure Description

[0024] The above and other objects, features, and advantages of this application will become more apparent from the detailed description of exemplary embodiments with reference to the accompanying drawings. The drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0025] Figure 1 The flowchart illustrating the generation process of the content-adaptive camouflage pattern based on style transfer from the first literature in the background art is shown.

[0026] Figure 2 The flowchart of the small dot camouflage generation algorithm in the second literature of the background art is illustrated.

[0027] Figure 3 The flowchart of the CAPGEN method in the third document of the background art is illustrated schematically.

[0028] Figure 4 A schematic diagram of the camouflage pattern generation system is shown.

[0029] Figure 5 A schematic diagram of the structure of the environmental texture data processing system is shown.

[0030] Figure 6 The flowchart illustrating the process of generating camouflage patterns is shown schematically.

[0031] Figure 7 The basic structure of AAE is illustrated schematically.

[0032] Figure 8 The diagram illustrates the basic principles of generative adversarial networks.

[0033] Figure 9 The diagram illustrates the steps involved in generating a camouflage pattern.

[0034] Figure 10 The diagram illustrates the steps of a data processing method for environmental textures.

[0035] Figure 11 The schematic diagram illustrates the structure of the SAM model.

[0036] Figure 12 The illustration shows the effect of partial convolution in image restoration. Figure 12 (a) represents the image that needs repair. Figure 12 (b) shows the result obtained using the PatchMatch method. Figure 12 (c) shows the results obtained using the method of Iizuka et al. Figure 12 (d) shows the effect obtained by using the method of Yu et al. Figure 12 (e) shows the result obtained using the Hole=127.5 method. Figure 12 (f) shows the result obtained using the Hole-IN_Mean method. Figure 12 (g) represents the result obtained using the partial conv method. Figure 12 (h) represents the original image.

[0037] Figure 13 The flowchart of the style transfer algorithm is illustrated.

[0038] Figure 14 A schematic comparison diagram of camouflage patterns generated by this method and existing technologies is shown. Among them, Figure 14 (a) represents the original environmental image. Figure 14 (b) represents a camouflage pattern generated by existing technology. Figure 14 (c) represents the camouflage pattern generated by this scheme.

[0039] Figure 15 The illustration shows a schematic diagram of how this method generates a camouflage pattern and applies it to a foreground object. Wherein, Figure 15 (a) represents the original environmental image. Figure 15 (b) indicates that the camouflage pattern generated by this scheme is attached to Figure 15 (a) A schematic diagram of the foreground object; Figure 15 (c) represents another original environmental image. Figure 15 (d) indicates that the camouflage pattern generated by this scheme is attached to Figure 15 (c) Schematic diagram of the foreground object. Detailed Implementation

[0040] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that this application will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0041] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0042] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0043] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0044] It should be understood that although the terms first, second, third, etc., may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another. Therefore, the first component discussed below may be referred to as the second component without departing from the teachings of this application. As used herein, the term "and / or" includes all combinations of any one and more of the associated listed items.

[0045] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of exemplary embodiments, and the modules or processes in the drawings are not necessarily essential for implementing this application, and therefore cannot be used to limit the scope of protection of this application.

[0046] According to a first specific embodiment of the present invention, the present invention provides a camouflage pattern generation system, such as... Figure 4 As shown, it includes an environmental texture data processing system and a camouflage pattern processing system.

[0047] Environmental texture data processing systems are used to reconstruct environmental texture maps. For example... Figure 5 As shown, the environmental texture data processing system includes: an image acquisition module, an image segmentation module, an environmental texture map reconstruction module, a first supervision module, and a second supervision module.

[0048] The image acquisition module is used to acquire environmental images.

[0049] The image segmentation module is connected to the image acquisition module and is used to segment the foreground objects and background patterns of the environment image to form a single-channel image.

[0050] The environment texture map reconstruction module is connected to the image segmentation module. It is used to extract background features from the environment image and single-channel image, and then reconstruct the environment texture map from the background features. For example... Figure 6 As shown, the environment texture map reconstruction module includes encoder E1 and decoder E2. Encoder E1 is used to extract background features, and decoder E2 is used to reconstruct the environment texture map from the background features.

[0051] The first supervised module is connected to the environment texture map reconstruction module and is used to train the background features to conform to a normal distribution. For example... Figure 6 As shown, the first supervision module includes a normal distribution sampling module and a first discriminator D1. The normal distribution sampling module is used to sample normally distributed data, and the first discriminator D1 is trained to distinguish background features and train the background features to conform to a normal distribution.

[0052] The second supervisory module is connected to the environment texture map reconstruction module, image acquisition module, and image segmentation module. It is used to slice the environment image into environment slice maps; based on the environment slice maps, it determines whether the environment texture map is similar to the environment image. If so, it outputs the environment texture map; otherwise, it returns to the environment texture map reconstruction module to reconstruct the environment texture map. Figure 6 As shown, the second supervision module includes a slicing module and a second discriminator D2. The slicing module is used to slice the environment image to form an environment slice map. The second discriminator D2 is used to determine whether the environment texture map is similar to the environment image based on the environment slice map. If it is, the environment texture map is output; otherwise, the environment texture map is returned to the environment texture map reconstruction module to reconstruct the environment texture map.

[0053] Encoder E1, decoder E2, first discriminator D1, and second discriminator D2 form an AAE (Adversarial Autoencoders) network.

[0054] Adversarial autoencoders (AAEs) are deep learning models that combine autoencoders and generative adversarial networks (GANs). They introduce adversarial mechanisms to regularize the latent space and incorporate a discriminator to enhance the quality of generated images. Traditional autoencoders, consisting of an encoder and a decoder, learn to compress input data into latent vectors and reconstruct the original data, but they offer weak control over the structure of the latent space. AAEs, while maintaining reconstruction capabilities, introduce a discriminator to align latent variables, allowing the latent space to better conform to prior distributions and enhancing image constraints.

[0055] like Figure 7 As shown, the core idea of ​​AAE is to apply a discriminator on top of the autoencoder. This discriminator can not only help the latent space better conform to the prior distribution, but also enhance the generative ability of the model and the interpretability of the semantic representation.

[0056] Generative Adversarial Networks (GANs) achieve end-to-end image generation by constructing separate generators and discriminators. Their core mechanism involves the generator continuously optimizing the generated images to deceive the discriminator, while the discriminator continuously improves its ability to distinguish between generated patterns and real images. Through adversarial training, both continuously improve the quality of the generated images. Their basic principle is as follows: Figure 8 As shown, samples of real images are obtained, and samples are generated by a generator. The samples generated by the generator are fake images. Both types of samples are input into a discriminator, which outputs a supervision signal (the discrimination result of the discriminator loss module) and a generated pattern. The supervision signal is a 0 / 1 signal, where 0 represents fake and 1 represents real. The supervision signal is used to continuously improve the ability to distinguish real images (optimize the generator). To better optimize the generator so that the generated pattern continuously approaches the real pattern, a generator loss module is introduced to maximize the generated image.

[0057] The AAE model consists of three parts: an encoder, a decoder, and a discriminator. The AAE structure used in this scheme consists of four parts: an encoder E1, a decoder E2, and two discriminators, D1 and D2 (e.g., ...). Figure 6 As shown in the diagram, this scheme employs a dual-discriminator adversarial training approach. Encoder E1 extracts effective background features from the original image, while decoder E2 reconstructs the environmental texture map using these background features; the first discriminator D1 then trains the background features on a random multivariate normal distribution of the same dimension. The first discriminator performs adversarial training with the second discriminator to ensure that the generated features conform to a normal distribution as much as possible, thereby achieving better interpretability and applicability. The second discriminator, D2, performs adversarial training with the environment slice map of the original image and the generated environment texture map to generate an environment texture map that is adapted to the environment slice map and improve the quality of the generated texture map.

[0058] The camouflage pattern processing system is used to generate camouflage patterns from environmental texture maps reconstructed by an environmental texture data processing system.

[0059] According to a second embodiment of the present invention, the present invention provides a method for generating camouflage patterns, employing the camouflage pattern generation system of the first embodiment, such as... Figure 9 As shown, it includes the following steps: S1: Reconstruct the environmental texture map based on the environmental texture data processing method.

[0060] like Figure 10As shown, the method for processing environmental texture data using an environmental texture data processing system includes the following steps: S11: Acquire environmental images.

[0061] Environmental images are images taken in the environment where the camouflage pattern needs to be applied, including background patterns and foreground objects.

[0062] S12: Segment the foreground objects and background patterns of the environment image to form a single-channel image.

[0063] As a preferred implementation, the SAM model is used to segment the foreground objects and background patterns of an environmental image using an arbitrary segmentation model.

[0064] The basic principle of the SAM model is data-driven and belongs to the category of neural network models. This neural network model is trained in the early stages using a large amount of labeled foreground and background data. After the model is trained, it can automatically distinguish between foreground and background.

[0065] The Segment Anything Model (SAM) is a general-purpose image segmentation model capable of segmenting any object in an image. Users provide cues such as points, bounding boxes, or text, and the SAM model outputs a high-quality segmentation mask, such as... Figure 11As shown, the SAM model architecture mainly consists of three parts: Image Encoder, Prompt Encoder, and Mask Decoder. The Image Encoder uses a ViT (Vision Transformer) structure, performing sliding window segmentation on the input image and extracting high-dimensional features to form a global semantic representation of the image. Essentially, it is responsible for converting the input image into serialized feature vectors containing rich information from the image. The Prompt Encoder encodes user-provided input prompts. Points and boxes are encoded into vectors using an MLP, while text prompts use a text encoder (such as CLIP) to generate embedded representations. Essentially, it converts the input prompts (such as points, boxes, and text) into corresponding feature vectors that describe the target information the user wants to segment. The Mask Decoder is a lightweight Transformer structure responsible for fusing image features and prompt information to output a high-quality mask result. Essentially, it fuses the outputs of the Image Encoder and Prompt Encoder to generate the final segmentation mask. SAM is trained on the large-scale dataset SA-1B (containing 1.1 billion masks), has extremely strong generalization ability, supports various tasks such as zero-shot segmentation, multi-object segmentation, and instance segmentation, and can segment any unknown image excellently.

[0066] Figure 6 The truck in the pattern on the left is a foreground object and is considered interference information that must be removed. To generate camouflage that blends into the environment, there must be no foreground objects interfering.

[0067] The single-channel image described in this patent, i.e., the mask (specifically, in other documents or patents, a single-channel image is not just a mask; a mask is only one form of a single-channel image), is a single-channel black and white image. Figure 6 For example, the foreground object is a truck, and the area where the truck is located is black with a pixel value of 0; the background area is white with a pixel value of 1. During the convolution operation, the weight of the position with a value of 0 is set to 0, and the weight of the position with a value of 1 is set to 1, which is equivalent to keeping the importance unchanged. In this way, the influence of the foreground object on the generated result is eliminated.

[0068] S13: Extract background features from the environmental image and single-channel image, and reconstruct the environmental texture map from the background features.

[0069] As a preferred implementation, after extracting the background features, the background features are trained to conform to a normal distribution.

[0070] The encoder E1 extracts background features based on the environmental image and the single-channel image, the first discriminator D1 trains the background features to conform to a normal distribution, and the decoder E2 reconstructs the environmental texture map from the background features.

[0071] Encoder E1 introduces partial convolutions to more effectively extract background features and eliminate foreground object interference. Encoder E1 mainly consists of four 3×3 partial convolutional blocks, a batch normalization layer, and a non-linear activation layer. Partial convolutions can effectively combine with the mask image of the foreground object, focusing only on regions with a mask value of 1, meaning that the foreground portion of the image will disappear into the latent space after encoding. Introducing a batch normalization layer after the convolutional layers enhances the stability of the convolutional data, mitigates the risks of gradient explosion and vanishing gradients, and accelerates convergence. The non-linear activation layer utilizes ReLU (Rectified Linear Unit) to enhance the model's expressive power, improving its ability to capture complex patterns and relationships within the image. Decoder E2 adopts a symmetrical structure relative to encoder E1 (where partial convolutions are replaced by deconvolutions). Both encoder E1 and decoder E2 consist of four convolutional blocks.

[0072] Partial convolution is a convolution operation specifically designed to handle missing regions. Unlike traditional convolution, the core idea of ​​partial convolution is to use only valid pixels for convolution computation when calculating each convolution output and normalize the results. This makes the network more robust when handling missing regions and avoids interference from invalid pixels in the convolution results. Figure 12 As shown, this method is commonly used for image inpainting, where, Figure 12 (a) represents the image that needs repair. Figure 12 (b) shows the result obtained using the PatchMatch method. Figure 12 (c) shows the results obtained using the Iizuka et al. method. Figure 12 (d) shows the effect obtained by using the method of Yu et al. Figure 12 (e) shows the result obtained using the Hole=127.5 method. Figure 12 (f) shows the result obtained using the Hole-IN_Mean method. Figure 12 (g) represents the result obtained using the partial conv method. Figure 12 (h) represents the original image, and it can be seen that this method is the most effective.

[0073] In partial convolution, the input image is typically accompanied by a binary mask, indicating which parts of the image are known and which need to be occluded. Traditional convolution, however, cannot distinguish between these parts within a sliding window, easily leading to blurred colors or structural errors in the repaired area. Partial convolution, in each convolution, only uses pixels marked as "known" in the mask for computation, and normalizes the output based on the number of pixels involved, ensuring that the output does not shift due to changes in the number of participating pixels. Furthermore, in partial convolution networks, the mask is updated at each layer. If an output location has at least one valid input pixel in the current convolution, that location is marked as "valid" in the mask of the next layer, achieving a mechanism of progressive "repair" from the edge to the center. This dynamic mask update method ensures that the network's layer-by-layer structural repair better conforms to the local continuity and contextual semantics of the image.

[0074] This solution uses a partial convolution (pconv) algorithm to address the artifact problem. The principle is that partial convolution accurately matches the position of foreground objects in the mask and the original image during each convolution process. In contrast, the first paper on background techniques simply scaled the mask. Scaling and convolution operations cannot be correlated in terms of position, so areas that are originally foreground might be mistaken for background areas. Furthermore, the first paper did not perform adversarial training (AAE) (only style transfer), resulting in blocky artifacts.

[0075] Compared to the second and third literatures in the background, this scheme does not perform discrete sampling of colors, so it can cover all color and texture information in complex environments.

[0076] The first discriminator D1 consists of three dense fully connected layers, designed to guide the data in the latent space to conform to the characteristics of a normal distribution.

[0077] The first discriminator D1 has two inputs: one is the background features extracted by the encoder E1, and the other is a normal distribution sample. The first discriminator D1 calculates the difference between these two and then returns the difference (usually called loss, in the form of gradient) to the encoder E1, allowing the encoder E1 to continuously adjust and optimize its internal neural network so that the background features extracted by the encoder E1 are closer and closer to the normal distribution.

[0078] S14: Slice the environment image to form an environment slice map; determine whether the environment texture map is similar to the environment image based on the environment slice map. If it is, output the environment texture map; otherwise, return to step S13 to reconstruct the environment texture map.

[0079] A series of environment slices are cropped from the background region of the original image, ensuring that these slices do not overlap with the foreground target region. The specific steps are as follows: Set the size of the slices to be sampled to 128×128, and set the step size of the sampling window on the original image to 0.2 times the slice's width and height. Using a sliding window, slices are generated sequentially horizontally and vertically across the entire image, starting from the top left corner. For each slice, overlap with the segmented foreground mask (foreground object) is checked; if there is no overlap, the slice is considered part of the background region; otherwise, the sampling is skipped. Through this sampling process, a series of environment slices are obtained, which are subsequently used to train the AAE network.

[0080] The method for determining overlap is to avoid it by comparing it with a mask (single-channel image). If the area where the foreground object is located is 0, and the sampling range includes an area where the mask is 0, it means that the foreground has been sampled, and it is discarded. If the sampling range does not include an area where the mask is 0 when compared with the mask, it means that the sampling range is entirely the background area, and it is retained.

[0081] The second discriminator, D2, uses four convolutional blocks consisting of a standard convolutional layer and LeakyReLU (Leaky Rectified LinearUnit), along with two dense fully connected layers. Its primary function is to assist the decoder in reconstructing feature maps. LeakyReLU mainly avoids neuron "death" by adding a small slope in the negative interval and helps alleviate the gradient vanishing problem.

[0082] The image acquisition module and image segmentation module obtain an environment slice image, which is obtained by sampling the environment background. This sampled image is saved and used as label data during training. The second discriminator D2 has two inputs: an image reconstructed by decoder E2 (Xt) and an environment slice image (Xb). The second discriminator D2 calculates the difference between these two images and returns this difference (generally called loss, expressed as a gradient) to encoder E1 and decoder E2. This allows encoder E1 and decoder E2 to continuously adjust and optimize their internal neural networks, ensuring that the image reconstructed by decoder E2 increasingly closely resembles the environment background.

[0083] Training the entire AAE network (including training in steps S13 and S14): When constructing the reconstruction loss for decoder E2, since encoder E1 uses partial convolution to remove the foreground portion outside the encoding process, background is needed to guide the reconstruction of the removed foreground region. Let the input image be... The target image is The sampled environment tile images are used as label data: the label images are constructed by copying and stitching the sampled environment tile images (128×128) according to the original image size (256×256). Assuming the mask for the image foreground is M, the reconstruction loss consists of minimizing two loss terms: and . Image used to reconstruct the background area The image used to reconstruct the foreground region is shown in Equations (1) and (2): (1); (2); in, It is an image used to reconstruct the background area. This is the image used to reconstruct the foreground region, where c is the number of channels in the image, and h and w are the height and width of the image, respectively. I Is with M For a unit image of the same dimension, all pixel values ​​are 1. For the first discriminator D1 and the second discriminator D2, loss functions are designed as shown in formula (3) and formula (4), respectively, where K1 and K2 are the dimensions of the output vectors of the first discriminator D1 and the second discriminator D2, respectively. Let the background feature be a one-dimensional vector l, and let the random normal sampling with the same size as the background feature vector be z.

[0084] (3); (4); Formula 3: L D1 This represents the loss function of the first discriminator D1. The first discriminator D1 has two inputs: the first is the background feature vector obtained by encoder E1 encoding the input image. l The other is a vector (z) sampled from a multivariate random normal distribution. These two inputs result in two outputs from the first discriminator D1, namely D1( l The outputs are also vectors, but their lengths are K1 and L. D1 The process involves calculating the absolute value of the difference between two output vectors one by one, then summing them and taking the average. This yields the difference between the two vectors, which is then used to optimize and adjust the neural network in encoder E1 using the gradient.

[0085] Formula 4: L D2 This represents the loss function of the second discriminator D2, which has two inputs: the first is the input background feature vector from the decoder E2. l The environment texture map obtained by decoding The other is a label image randomly selected from the background sampling image. These two inputs result in two outputs from the second discriminator D2, namely... and Both outputs are also vectors, but their lengths are K² and L. D2 The process involves calculating the absolute value of the difference between two output vectors one by one, then summing them and taking the average. This yields the difference between the two vectors, which is then used to simultaneously optimize and adjust the neural networks in encoder E1 and decoder E2 using the gradient.

[0086] Finally, the loss function for training AAE is shown in Equations (5) and (6): (5).

[0087] (6).

[0088] The AAE is trained using an alternating training method: First, encoder E1 and decoder E2 are trained using formula (5), while keeping the parameters of the first discriminator D1 and the second discriminator D2 unchanged. Then, the first discriminator D1 and the second discriminator D2 are trained using formula (6), while keeping the parameters of encoder E1 and decoder E2 unchanged. The two are trained alternately until convergence.

[0089] Once the entire AAE network is trained, simply input the original image directly into encoder E1 and decoder E2 in AAE to obtain the environment texture map. Note that the input at this point does not include the foreground object mask. This is because the trained encoder E1 network can automatically distinguish between the foreground and background, while encoder E2 can automatically encode the background texture information into the foreground region to obtain the complete environment texture map.

[0090] S2: Generate camouflage patterns based on the environment texture map.

[0091] After generating the environmental texture map, this solution uses style transfer to generate the camouflage pattern. Style transfer can effectively transfer the environmental texture map extracted by AAE to the camouflage template pattern (refer to the camouflage image) to adapt to different environments. Using style transfer to generate camouflage patterns not only enables automatic generation of camouflage patterns, but also allows them to blend more naturally into the background environment, improving concealment.

[0092] Style transfer algorithms break through the single dimension of color matching by capturing high-order statistical features of background texture through a Gram matrix. The algorithm utilizes a pre-trained network model to extract content features from the target template and style features from the background image, respectively. By minimizing a weighted combination of content and style losses, it achieves the camouflage pattern while preserving the target outline and integrating the microscopic texture of the background. The basic principle of style transfer is as follows: Figure 13 As shown, the generation effect of this method is significantly better than that of traditional clustering, especially in maintaining multi-scale structural similarity when simulating complex backgrounds.

[0093] like Figure 13 The style transfer algorithm shown first extracts and stores content features and style features using a VGG19 network. The style image 'a' is input into the VGG19 network and passes through multiple convolutional layers from bottom to top, calculating and storing its style features A across all relevant layers. L (Left side). The content image p is input into the VGG19 network, passes through multiple convolutional layers from bottom to top, and its content features P in all relevant layers are calculated and stored. l (Right side). Then, a random white noise image x is input into the VGG19 network to compute its style features G. L and content features F l In all layers contained within the style representation, G is calculated. L With A L The element-wise mean squared error is used to obtain the style loss L. style (Left side). Simultaneously calculate F. l With P l The mean squared error is used to obtain the content loss L. content (Right side). Total loss L total It is a linear combination of content loss and style loss. Its derivative with respect to pixel values ​​can be calculated through error backpropagation (middle). The gradient descent algorithm is used to continuously update image x until it simultaneously matches the style features of style image a and the content features of content image p (bottom middle).

[0094] like Figure 6 As shown, generating camouflage patterns based on environmental texture maps includes: S21: Obtain a reference camouflage pattern and generate a camouflage pattern based on the reference camouflage pattern.

[0095] S22: Input the reference camouflage image, environment texture image, and camouflage pattern into a multi-layer convolutional network. The multi-layer convolutional network is VGG19.

[0096] S23: Compare each convolutional layer of the reference camouflage image with the convolutional layer of the corresponding camouflage pattern, and calculate the first style loss function.

[0097] S24: Compare each convolutional layer of the environment texture map with the convolutional layer corresponding to the camouflage pattern, and calculate the second style loss function; S25: Generate a new camouflage pattern based on the reference camouflage pattern, the first style loss function, and the second style loss function; S26: Repeat the input of the multi-layer convolutional network, calculate the first style loss function and the second style loss function, and generate a new camouflage pattern (i.e., repeat steps S21 to S25), and use the finally generated camouflage pattern as the final camouflage pattern.

[0098] This invention uses a pre-trained VGG19 network as the backbone network for style transfer. For example... Figure 6 As shown in the second stage on the right, the content map (referencing the camouflage image) and style map (environmental texture image) are first processed through a pre-trained VGG19 network (with fixed parameters). The output features of the content map and style map at different layers of the VGG19 network are extracted and saved as supervision signals for style transfer. While typical style transfer methods use random noise images as input for generation, this scheme, to improve convergence speed, directly uses the content map (referencing the camouflage image) as the initial input image. Based on the initial input image, the style transfer algorithm continuously modifies and adjusts the input image through backpropagation using the aforementioned supervision signals until the algorithm converges and outputs a fused content and style image. The output image retains the structural information of the content map while incorporating the style features of the style map.

[0099] The fixed parameters of the VGG19 network mean that the parameters remain unchanged throughout the entire training process. To optimize the input image, the calculated gradient is transmitted to the input layer (i.e., the input image) in its entirety by fixing the parameters. The derivative of the gradient with respect to the pixel value is calculated to force the input image to change continuously (gradient descent algorithm) and eventually become the final camouflage image.

[0100] This invention uses three loss terms for style transfer: content loss Style loss Total variation loss Content loss ensures that the initial camouflage template retains its content, while style loss captures the texture and color of the environment texture map. Furthermore, total variation loss effectively minimizes image noise. Assuming the initial content map (reference camouflage map) is... The style map (environment texture map) is The generated target image is P, and the three loss functions are shown in formulas (7), (8), and (9).

[0101] (7); (8); (9); in, It is the content loss function: where F l It is the corresponding layer of the VGG19 network (such as...) Figure 6 As shown, this invention mainly involves three layers: conv_1, conv_2, and conv_3. X c and P These are the initial content image (reference camouflage image) and the generated target image (final camouflage image). X s This is a style map (environment texture map). The content loss function aims to preserve content as much as possible by minimizing the difference between the two.

[0102] This represents the style loss function. The Gram matrix is ​​calculated as follows: The Gram matrix is ​​a covariance matrix, essentially an inner product of a feature map across each channel. It represents the correlation between two feature maps, and this is a general calculation formula.

[0103] Let represent the total variation loss function, where P The image is represented by the following: i, j, k represent pixel positions, c represents image channels, h represents image height, and w represents image width. Its purpose is to reduce noise by minimizing the differences between each pixel and its neighboring pixels in the horizontal and vertical directions to improve the output image. P Smoother.

[0104] In formula (7) F l It is the corresponding layer 1 of the VGG19 network (e.g. Figure 6 As shown, this invention mainly involves three layers: conv_1, conv_2, and conv_3. In formula (8)... Gram A matrix (Gram matrix) is essentially a non-centered covariance matrix of eigenvectors, used to measure the similarity between two matrices. The specific calculation is shown in formula (10). Formula (9) Total variation loss Noise is reduced by decreasing the difference between each pixel and its neighboring pixels in the horizontal and vertical directions to improve the output image. P The style transfer is smoother. The total loss function for style transfer is shown in Equation (10).

[0105] (10); (11).

[0106] Among them, in formula (10) Fl This refers to the convolutional layer operation corresponding to the VGG19 network. The result of this convolutional operation is the feature map of the corresponding layer. Assuming that The number of channels is c, the height is h, and the width is w. Then... The shape is c×h×w. During calculation, the spatial dimensions (h, w) are first flattened into a vector of length hw. F l i，k ( X ) represents the feature map The value of the i-th channel at the k-th position (k takes values ​​between [1, hw]) after flattening. F l i，k ( X ) represents the feature map The value of the j-th channel at the k-th position after flattening, where i and j are respectively The different number of channels also represents the row and column indices of the Gram matrix. The formula then expresses the inner product of channel i and channel j across all spatial locations, used to measure their feature correlation at that layer.

[0107] L style_transfer It is the total loss function, which includes content loss, style loss, and total variation loss.

[0108] like Figure 14 As shown, on the left Figure 14 (a) is an environmental image, from which a camouflage pattern is generated, according to existing technology. Figure 14 (b) Clearly contains foreground interference information (red blocks), this solution is Figure 14 (c) can remove foreground interference, and Figure 14 Within the purple box, the red color and texture of the remaining off-road vehicle are artifacts, and fewer camouflage artifacts are generated.

[0109] like Figure 15 As shown, Figure 15 (a) is an environmental image with a forest background, showing the camouflage pattern generated according to this scheme. Figure 15 (b) When placed on a foreground object (truck), it almost completely blends into the background; Figure 15 (c) is another environmental image with a desert background. A camouflage pattern is generated according to this scheme. Figure 15 (d) When placed on a foreground object (off-road vehicle), it almost completely blends with the background, and it can be seen that it plays a good role in camouflage.

[0110] In summary, this solution proposes an intelligent method and system for generating high-quality camouflage patterns that adapts to the environment. The method includes two stages: environmental texture map extraction and final camouflage pattern generation.

[0111] Stage 1: Environment Texture Extraction. This stage first segments foreground objects in the environment and generates corresponding masks. Then, it randomly samples the background to generate background slices and uses an adversarial autoencoder with a partially convolutional network to extract high-quality environment texture maps. Applying partial convolution to encoder E1 in the dual discriminator AAE effectively removes foreground interference. Stage 2: Camouflage Generation. This stage mainly uses the environment texture map and reference camouflage map to generate high-quality camouflage patterns adapted to specific environments through style transfer. In style transfer, random noisy image input is replaced with a content map, greatly improving the training convergence speed of the style transfer network.

[0112] Therefore, this scheme adopts dual-layer supervision to achieve adversarial training, separates the background and foreground regions, effectively removes the interference of irrelevant foreground on the generated camouflage image, better integrates the background pattern, eliminates artifacts and generates a clearer, noise-free texture feature map that covers all color and texture information of the complex environment, thereby improving the generation quality of camouflage patterns.

[0113] Exemplary embodiments of the present invention have been specifically shown and described above. It should be understood that the present invention is not limited to the detailed structures, arrangements, or implementations described herein; rather, the present invention is intended to cover various modifications and equivalent arrangements contained within the spirit and scope of the appended claims.

Claims

1. A method for processing environmental texture data, characterized in that, Includes the following steps: S11: Acquire environmental images; S12: Segment the foreground objects and background patterns of the environment image to form a single-channel image; S13: Extract background features from the environmental image and single-channel image, and reconstruct the environmental texture map from the background features.

2. The environmental texture data processing method according to claim 1, characterized in that, It also includes: S14: slice the environment image to form an environment slice map; determine whether the environment texture map is similar to the environment image based on the environment slice map. If it is, output the environment texture map; otherwise, return to step S13 to reconstruct the environment texture map.

3. The environmental texture data processing method according to claim 1, characterized in that, In step S12, the SAM model is used to segment the foreground objects and background patterns of the environment image.

4. The environmental texture data processing method according to claim 1, characterized in that, Step S13 also includes: training the background features to conform to a normal distribution.

5. A data processing system for environmental textures, characterized in that, include: Image acquisition module, image segmentation module, and environmental texture map reconstruction module; The image acquisition module is used to acquire environmental images; The image segmentation module is used to segment the foreground objects and background patterns of the environmental image to form a single-channel image; The environment texture map reconstruction module is used to extract background features based on the environment image and single-channel image, and reconstruct the environment texture map from the background features.

6. The environmental texture data processing system according to claim 5, characterized in that, It also includes a first supervision module and a second supervision module; The first supervision module is used to train the background features to conform to a normal distribution; The second supervision module is used to slice the environment image to form an environment slice map; based on the environment slice map, it determines whether the environment texture map is similar to the environment image. If so, the environment texture map is output; otherwise, it returns to the environment texture map reconstruction module to reconstruct the environment texture map.

7. A method for generating a camouflage pattern, characterized in that, Includes the following steps: S1: The environmental texture data processing method according to any one of claims 1-4 reconstructs the environmental texture map; S2: Generate camouflage patterns based on the environment texture map.

8. The method for generating camouflage patterns according to claim 7, characterized in that, In step S2, generating the camouflage pattern based on the environmental texture map includes: Obtain a reference camouflage pattern and generate a camouflage pattern based on the reference camouflage pattern; The reference camouflage image, the environment texture image, and the camouflage pattern are respectively input into a multi-layer convolutional network; The first style loss function is calculated by comparing each convolutional layer of the reference camouflage image with the convolutional layer of the corresponding camouflage pattern. Compare each convolutional layer of the environment texture map with the convolutional layer of the camouflage pattern to calculate the second style loss function. Generate a new camouflage pattern based on the reference camouflage pattern, the first style loss function, and the second style loss function; The process involves repeatedly inputting the data into a multi-layer convolutional network, calculating the first and second style loss functions, generating new camouflage patterns, and using the final generated camouflage pattern as the final camouflage pattern.

9. The method for generating camouflage patterns according to claim 8, characterized in that, The multilayer convolutional network is VGG19.

10. A camouflage pattern generation system, characterized in that, include: The environmental texture data processing system and camouflage pattern processing system as described in claim 5 or 6; The camouflage pattern processing system is used to generate camouflage patterns based on the environmental texture map reconstructed by the environmental texture data processing system.

Images