A multi-source remote sensing image dodging and color uniformization method based on style transfer

Abstract

The application discloses a kind of based on style migration multi-source remote sensing image light uniformity color method, belong to computer vision and remote sensing science and technology field. Including: based on standard optical model, make remote sensing satellite cloud image dataset;Through category semantic label, construct positive and negative prompt word, and the degraded cloud image is introduced into noise estimation network as condition to estimate conditional noise distribution, train conditional diffusion model;Convolution network and the remote sensing image semantic extraction model of the combination of Transformer are constructed, and the semantic extraction of multi-source remote sensing image is carried out, and the ground object category label of each pixel point is output;For the content and style information of multi-source remote sensing image, construct the style migration model based on Transformer, reference radiation characteristic suitable benchmark image, realize the light uniformity color of remote sensing image. It can be applied to realize the color coordination between adjacent images, contrast is moderate, make color image approach natural real color.

Description

Technical Field

[0001] This invention relates to a method for homogenizing light and color in multi-source remote sensing images based on style transfer, belonging to the fields of computer vision and remote sensing science and technology. Background Technology

[0002] With the rapid advancement of remote sensing technology, remote sensing imagery plays an increasingly important role in several key areas such as urban planning, land resource management, and environmental monitoring. Against the backdrop of the national push for smart city development, the application of remote sensing technology not only promotes the rational use of land resources but also provides strong support for high-quality economic development. As a core tool for acquiring geospatial information, the color consistency and accuracy of remote sensing imagery are crucial for subsequent analysis and application. However, due to the influence of various factors such as lighting conditions, sensor characteristics, and atmospheric effects, acquired remote sensing images often exhibit color differences and unevenness, which seriously affect the mosaicking effect and practical application value of the images.

[0003] In the field of remote sensing image processing, the research on uniform illumination and color matching methods has profound significance. It not only significantly improves the usability and analytical accuracy of remote sensing images, but is also particularly important in processing large-scale, multi-source, and multi-temporal remote sensing images. By eliminating color differences caused by different sensors, imaging conditions, and environmental variations, uniform illumination and color matching methods ensure the consistency of color and brightness in remote sensing images, providing a solid, high-quality data foundation for subsequent image analysis and applications. Currently, research on uniform illumination and color matching of multi-source remote sensing images still faces many challenges, such as cloud and fog interference, significant color differences, insufficient automation, difficulty in extracting stitching seams, and unsatisfactory stitching seam removal results. Summary of the Invention

[0004] This invention provides a method for homogenizing the light and color of multi-source remote sensing images based on style transfer, solving the problems disclosed in the background art. This method can be divided into three technical modules: cloud and fog removal based on style transfer, semantic extraction, and style transfer. Through deep learning technology, the color and brightness of remote sensing images can be intelligently adjusted to achieve homogenization of light and color. First, a remote sensing cloud and fog image dataset is created based on crawled OSM images and a standard optical model, and a conditional diffusion model is trained to preprocess the multi-source remote sensing images for cloud and fog removal. Then, semantic extraction is performed on the preprocessed remote sensing images to achieve semantic segmentation. Finally, style transfer technology is used to ensure the consistency of land cover styles between different images, achieving seamless mosaicking. This method optimizes the visual presentation of images while preserving land cover features, enhancing the application value of remote sensing images.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0006] A method for homogenizing and coloring multi-source remote sensing images based on style transfer includes:

[0007] Based on the standard optical model, images containing different degrees of cloud and fog are synthesized to create a remote sensing satellite cloud and fog image dataset.

[0008] Prompt words are constructed using category semantic labels to generate text conditions; degraded cloud and fog images from the remote sensing satellite cloud and fog image dataset are introduced as conditions into a noise estimation network to estimate the conditional noise distribution and generate cloud and fog control conditions. A latent diffusion model is trained based on the text conditions and the cloud and fog control conditions; the result after applying the cloud and fog image control conditions is added to the result output by the latent diffusion model, and the remote sensing image with cloud and fog removed is output by the VAE decoder.

[0009] Using a pre-built remote sensing image semantic extraction model, semantic extraction is performed on the remote sensing images after cloud and fog removal preprocessing, and the content and style information of multi-source remote sensing images are output.

[0010] To address the content and style information of multi-source remote sensing images, a Transformer-based style transfer model is constructed. This model references a baseline image with suitable radiometric characteristics to achieve uniform illumination and color in the remote sensing images.

[0011] Furthermore, the standard optical model is as follows: ;

[0012] in, These are the coordinates of the image pixels. For images with fog, For the dehazed image to be recovered, Transmittance, This refers to the global atmospheric light composition. In cloudy or foggy weather conditions, reduced reflected energy causes transmission attenuation. This leads to a decrease in image brightness; ambient light scattering creates atmospheric light. This enhances image brightness and reduces saturation; cloud and fog images are formed by a camera capturing light reflected from objects weakened by clouds and fog, as well as atmospheric light reflected from clouds and fog. This refers to the weakening and scattering of light by clouds and fog.

[0013] Furthermore, the training objective of the potential diffusion model is: ;

[0014] in, The actual noise value follows a standard normal distribution. , This represents the noise value predicted by the model; This indicates the time step in the diffusion process. It is a representation vector in the latent space. This represents the latent variables in the diffusion process at a certain moment during the diffusion process. state, Conditional embedding for degraded cloud and fog images, This indicates text conditional embedding. This represents the expected value.

[0015] Furthermore, the methods for constructing semantic extraction models from remote sensing images include:

[0016] A convolutional network-based encoder is constructed, using ResNet18 as the encoder to extract multi-scale semantic features. It consists of four residual blocks, each of which reduces the scale factor of the feature map by a factor of 2 through downsampling. The feature map generated by each stage is fused with the corresponding feature map in the decoder through 1x1 convolution. The semantic features generated by the residual blocks and the features generated by the global-local transformation blocks in the decoder are aggregated through a weighted summation operation.

[0017] Construct a Transformer-based decoder that utilizes three global-local attention Transformer blocks and a feature refinement head to build a lightweight Transformer-based decoder.

[0018] The global-local attention Transformer block constructs two parallel branches to extract global and local contexts, respectively, including:

[0019] The local branch uses two parallel convolutional layers with core sizes of 3 and 1 respectively to extract the local context; two batch normalization operations are added before the final summation operation.

[0020] The global branch uses a window-based multi-head self-attention mechanism to capture the global context; it uses a standard 1x1 convolution to expand the channel dimension of the input 2D feature map to three times; then it applies a window segmentation operation to segment the 1D sequence into query Q, key K, and value V vectors.

[0021] Furthermore, the formula for the weighted summation operation is: ;

[0022] in, Indicates fusion features, The characteristics representing the generation of residual blocks, This represents the features generated by the global and local Transformer blocks. Indicates the weight.

[0023] Furthermore, the method for constructing the remote sensing image semantic extraction model also includes building a cross-shaped window context interaction module. This module captures the global context by fusing two feature maps generated by horizontal average pooling layers and vertical average pooling layers; the horizontal average pooling layer establishes the horizontal relationship between windows; for any point in window 1... The corresponding point in window 2 The dependency model is as follows:

[0024] ;

[0025] ;

[0026] ;

[0027] ;

[0028] in, Indicates an index. It refers to the window size. This represents self-attention computation, which can model the dependencies between pixel pairs within a local window.

[0029] Furthermore, Transformer-based style transfer models include:

[0030] The Transformer encoder is used to feed the image sequence to be processed into the Transformer encoder. Each layer contains a multi-head self-attention module and a feedforward network to convert the input sequence into a query Q, a key K and a value V.

[0031] The multi-head self-attention module is used to process different heads in parallel through a multi-head self-attention mechanism, calculate attention, and effectively encode the input sequence.

[0032] The Transformer decoder is used to translate the encoded content sequence using a regression approach based on the encoded style sequence. All sequence blocks are input at once, and the output is predicted. The feature sequence is generated by using the content sequence to generate the query Q and the style sequence to generate the key K and value V, and then performing sequence translation. The output sequence of the Transformer decoder is further refined by a three-layer CNN decoder, including convolution, ReLU activation, and upsampling.

[0033] Furthermore, the method for achieving uniform illumination and color in remote sensing images by referencing a reference image with suitable radiometric characteristics in the style transfer model includes:

[0034] After the input sequence is embedded, it is fed into the Transformer encoder, where it is transformed into a representation of query Q, key K, and value V.

[0035] ;

[0036] in, This represents a sequence of input content. , Indicates the sequence length. Indicates the number of heads of attention;

[0037] Multi-head self-attention mechanisms compute attention by processing different heads in parallel, thus achieving efficient encoding of the input sequence.

[0038] ;

[0039] The Transformer decoder follows the coding style sequence Translate encoded content sequences using a regression approach The input to the Transformer decoder includes the encoded content sequence. and style sequence Generate queries using content sequences Use style sequence to generate keys Sum :

[0040] ;

[0041] Calculate the output sequence of the Transformer decoder :

[0042] ;

[0043] Output sequence of Transformer decoder Having shape ,in and These represent the output height and width, respectively. This represents the number of channels; a three-layer CNN decoder is used to further refine the output of the Transformer decoder; for each layer of the CNN decoder, the scale is increased, including 3x3 convolutions, ReLU activation functions, and 2x upsampling, ultimately obtaining a resolution-sensitive output. The output image is matched to the input image in terms of spatial resolution. Each pixel of the output image contains information from three color channels, generating a uniformly lit and colored remote sensing image.

[0044] Accordingly, the present invention also provides a computer-readable storage medium for storing one or more programs, said one or more programs including instructions that, when executed by a computing device, cause the computing device to perform any of the methods described above.

[0045] Accordingly, the present invention also provides a computing device, comprising:

[0046] One or more processors, one or more memories, and one or more programs, wherein the one or more programs are stored in the one or more memories and configured to be executed by the one or more processors, and the one or more programs include instructions for performing any of the methods described above.

[0047] The beneficial effects achieved by this invention are as follows:

[0048] This invention provides a method for homogenizing and colorizing multi-source remote sensing images based on style transfer. It involves crawling OSM16 level images of some towns in the New York area and their category semantic labels to create a remote sensing satellite cloud and fog image dataset based on a standard optical model. A conditional diffusion model is trained using combined textual and degraded cloud and fog image conditions to achieve cloud and fog removal. Based on the cloud and fog removal preprocessing, a remote sensing image semantic extraction model combining convolutional networks and Transformers is constructed to extract semantics from the remote sensing images, achieving semantic segmentation for different categories. For the content and style information of multi-source remote sensing images, a Transformer-based style transfer model is constructed to extract content and style information, achieving the fusion of content and style information between the remote sensing image to be homogenized and a reference image with suitable radiometric characteristics. This results in tonal harmony and appropriate contrast between adjacent images, aiming to enhance the application value of remote sensing images in key areas such as land use monitoring, urban planning, and resource management. Attached Figure Description

[0049] Figure 1 This is a schematic diagram of the overall process of the style transfer-based multi-source remote sensing image homogenization and color homogenization method of the present invention;

[0050] Figure 2 This is a schematic diagram of the framework of the remote sensing image cloud and fog removal model of the present invention;

[0051] Figure 3 This is a schematic diagram of the framework of the style transfer-based multi-source remote sensing image homogenization and color homogenization model of the present invention. Detailed Implementation

[0052] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0053] like Figure 1 As shown, this invention provides a method for homogenizing the illumination and color of multi-source remote sensing images based on style transfer, comprising the following steps:

[0054] Step 1: Crawl OSM16 level images of some towns in the New York area and the category semantic labels in the images, and create a remote sensing satellite cloud and fog image dataset based on the standard optical model;

[0055] Furthermore, the aforementioned crawling of OSM16 level images of some towns in the New York area, along with the category semantic labels within the images, and the creation of a remote sensing satellite cloud and fog image dataset based on a standard optical model, includes:

[0056] Based on existing industry or national standards, such as the *Map Features-OSM Wiki* and the *Basic Geographic Information 1:50000 Topographic Feature Data Specification*, as well as experience and statistics from other projects, a land feature classification system was constructed, including 54 land feature categories such as roads, rivers, grasslands, bare land, and buildings. This ensures that the generated dataset images contain a rich variety of land feature categories, making the trained model more widely applicable to diverse terrains.

[0057] Based on the established feature classification system, imagery of some towns and villages in the New York area at OSM level 16 was crawled. Some categories were not shown in the images, such as bridges, ships, and water towers. Therefore, images were crawled again focusing on these objects to supplement the classification.

[0058] Crawl the category semantic labels from each image for use in subsequent model training to construct prompt words;

[0059] The standard optical model, such as Equation (1), is used to synthesize clouds and fog in remote sensing images. Based on the center point synthesis method, the image brightness, cloud and fog concentration, and fog size are randomly initialized to randomly synthesize clouds and fog of different locations, ranges, and degrees, ensuring the diversity of samples.

[0060] (1);

[0061] in These are the coordinates of the image pixels. For images with fog, For the dehazed image to be recovered, Transmittance, This refers to the global atmospheric light composition. In cloudy or foggy weather conditions, reduced reflected energy causes transmission attenuation. This leads to a decrease in image brightness. Ambient illumination scattering creates atmospheric light. This enhances image brightness and reduces saturation. Cloud and fog images are formed by a camera capturing light reflected from objects (attenuated by clouds and fog) and atmospheric light reflected from clouds and fog. This can be understood as the weakening and scattering effect of clouds and fog on light.

[0062] Step 2: Construct positive and negative cue words using category semantic labels, and incorporate degraded cloud and fog images as conditions into the noise estimation network to estimate the conditional noise distribution, training the conditional diffusion model, such as... Figure 2 As shown;

[0063] Furthermore, the method of constructing positive and negative cue words through category semantic labels and incorporating degraded cloud and fog images as conditions into a noise estimation network to estimate conditional noise distribution, and training a conditional diffusion model, includes:

[0064] Label cue words are constructed based on the land cover category labels of OSM images, such as "river, forest, meadow, motorway, buildings, and...". Positive and negative image cue words are then constructed based on the label cue words. The positive image cue word is "A bright and clear remote sensing satellite image including - [class types]", where [class types] is the label cue word. The negative image cue word is "A remote sensing satellite image obscured by clouds and haze." The label cue words and image cue words are used together as text conditions and input into the network through a clipped text encoder.

[0065] Latent diffusion model (LDM) compresses the input image into a low-dimensional vector. Furthermore, by performing the diffusion process within the latent space, the cost required for training and inference of the diffusion model is reduced. The generated samples... Reconstruct a high-resolution image in pixel space.

[0066] The stable diffusion model LDM is fine-tuned using text-conditioning and cloud / fog image conditions from both publicly available and self-made cloud / fog remote sensing datasets. This avoids the significant time and computational costs of training from scratch and the risk of "catastrophic forgetting," allowing the model to leverage rich domain-specific knowledge without altering pre-trained weights. Two identical copies of the LDM model are created: a "locked" copy and a "trainable" copy. The "locked" copy is frozen, meaning its weights remain unchanged, preserving the capabilities of the LDM model after training on large-scale image-text datasets. Degraded cloud / fog image conditions are then used. Fine-tuning is performed on the "trainable" copy, which introduces degraded cloud and fog images as image conditions into the noise estimation network to estimate the conditional noise distribution. The degraded cloud and fog image conditions are passed through a learnable embedding layer and... The calculation and training objective can be expressed as formula (2). Then, the result after applying the cloud and fog image control conditions is added to the result of the original LDM model and passed through the VAE decoder to obtain the final clear remote sensing image.

[0067] (2);

[0068] in, The actual noise value follows a standard normal distribution. , This represents the noise value predicted by the model. This indicates the time step in the diffusion process. It is a representation vector in the latent space. This represents the latent variables in the diffusion process at a certain moment during the diffusion process. state, Conditional embedding for degraded cloud and fog images, This indicates text conditional embedding. This represents the expected value.

[0069] Step 3: Construct a remote sensing image semantic extraction model that combines convolutional networks and Transformers to extract semantics from multi-source remote sensing images and output the land cover category label for each pixel.

[0070] Furthermore, the aforementioned construction of a remote sensing image semantic extraction model combining convolutional networks and Transformers performs semantic extraction on multi-source remote sensing images, outputting the land cover category label for each pixel, including:

[0071] A convolutional network-based encoder is constructed, using ResNet18 as the encoder to extract multi-scale semantic features. It consists of four residual blocks, each reducing the scale factor of the feature map by a factor of 2 through downsampling. The feature map generated in each stage is fused with the corresponding feature map in the decoder via a 1x1 convolution. The semantic features generated by the residual blocks and the features generated by the Global-Local Transform Block (GLTB) in the decoder are aggregated through a weighted summation operation. The weighted summation operation selectively weights the two features based on their contribution to the segmentation accuracy, thereby learning more generalized fused features. The formula for the weighted summation operation is as follows (3):

[0072] (3);

[0073] in, Indicates fusion features, The characteristics representing the generation of residual blocks, This represents the features generated by the global and local Transformer blocks. Indicates weight;

[0074] The Transformer-based decoder is constructed by using three global-local Transformer blocks and a feature refinement head to build a lightweight Transformer-based decoder. Through this hierarchical and lightweight design, the decoder can capture global and local information at multiple scales while maintaining high efficiency.

[0075] Specifically, a global-local attention Transformer block is proposed, which constructs two parallel branches to extract global and local contexts respectively, including:

[0076] The local branch employs two parallel convolutional layers with kernel sizes of 3 and 1 respectively to extract local context. Two batch normalization operations are appended before the final summation operation.

[0077] The global branch uses a window-based multi-head self-attention mechanism to capture the global context. A standard 1x1 convolution is used to expand the channel dimension of the input 2D feature map by three times. Then, a window segmentation operation is applied to split the 1D sequence into query (Q), key (K), and value (V) vectors.

[0078] To capture cross-window relationships with high computational efficiency, a cross-shaped window context interaction module is proposed. This module captures the global context by fusing two feature maps generated by horizontal and vertical average pooling layers. The horizontal average pooling layer establishes the horizontal relationships between windows. For any point in window 1... It is related to the one in window 2. The dependency relationship can be modeled as follows:

[0079] (4);

[0080] (5);

[0081] (6);

[0082] (7);

[0083] in, Indicates an index. It refers to the window size. This represents self-attention computation, which can model the dependencies between pixel pairs within a local window. It is the corresponding point in window 2.

[0084] Shallow features preserve rich spatial details but lack semantic content. Deep features provide accurate semantic information but have lower spatial resolution. While directly summing these two types of features is fast, it may reduce segmentation accuracy. Therefore, a feature refinement head is added to construct the decoder, including:

[0085] We perform a weighted summation of shallow and deep features to make full use of precise semantic information and spatial details;

[0086] The fused features are used as input to the feature refinement head to enhance the feature representation;

[0087] The channel path uses a global average pooling layer to generate a channel attention map, and adjusts the channel dimension by reducing and expanding operations;

[0088] The spatial path uses depthwise convolution to generate a spatial attention map to reflect the spatial resolution of the feature map;

[0089] The attention features generated by the two paths are further fused using a summation operation. A 1x1 convolutional layer and upsampling are then applied to generate the final segmentation map. Residual connections are introduced to prevent network performance degradation.

[0090] Step 4: Based on the content and style information of multi-source remote sensing images, construct a Transformer-based style transfer model and refer to a reference image with suitable radiometric characteristics to achieve uniform illumination and color of the remote sensing images.

[0091] Furthermore, the aforementioned construction of a Transformer-based style transfer model based on the content and style information of multi-source remote sensing images, referencing a reference image with suitable radiometric characteristics, to achieve uniform illumination and color in the remote sensing images includes:

[0092] After the input sequence is embedded, it is fed into the Transformer encoder. Each layer of the encoder contains a multi-head self-attention module and a feedforward network. The input sequence is converted into a representation of query (Q), key (K) and value (V), and the calculation formula is shown in (8):

[0093] (8);

[0094] in, This represents a sequence of input content. , Indicates the sequence length. This indicates the number of heads of attention.

[0095] Multi-head self-attention mechanisms compute attention by processing different heads in parallel, thereby achieving efficient encoding of the input sequence. The calculation formula is shown in (7):

[0096] (9);

[0097] The decoder is used to determine the encoding style sequence. Translate encoded content sequences using a regression approach Unlike the autoregressive process in traditional natural language processing tasks, this method predicts the output by taking all sequence blocks as input at once. Each Transformer decoder layer contains two multi-head self-attention modules and a feedforward network. The input to the Transformer decoder includes the encoded content sequence. and style sequence As shown in formula (10), the query is generated using the content sequence. Use style sequence to generate keys Sum :

[0098] (10);

[0099] Then, the output sequence of the Transformer decoder can be computed in the following way. :

[0100] (11);

[0101] The output sequence of Transformer Having shape ,in and These represent the output height and width, respectively. This represents the number of channels. To construct the final result, a three-layer CNN decoder is used to further refine the output of the Transformer decoder. For each layer of the CNN decoder, the scale is increased through a series of operations, including 3x3 convolutions, ReLU activation functions, and 2x upsampling. Ultimately, a resolution-enhanced version can be obtained. The final result. This means that the output data not only matches the input image in spatial resolution, but each pixel also contains information from three color channels, thus enabling the generation of high-quality images with natural colors.

[0102] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

[0103] A computer-readable storage medium storing one or more programs, the programs including instructions that, when executed by a computing device, cause the computing device to perform a style transfer-based method for homogenizing and colorimetrically isolating multi-source remote sensing images.

[0104] A computing device includes one or more processors, one or more memories, and one or more programs, wherein the one or more programs are stored in the one or more memories and configured to be executed by the one or more processors, and the one or more programs include instructions for performing a style transfer-based method for homogenizing and colorimetrically isolating multi-source remote sensing images.

[0105] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0106] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0107] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0108] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0109] The above are merely embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of the claims of the present invention pending approval.

Claims

1. A method for homogenizing the illumination and color of multi-source remote sensing images based on style transfer, characterized in that, include: Based on the standard optical model, images containing different degrees of cloud and fog are synthesized to create a remote sensing satellite cloud and fog image dataset. Prompt words are constructed using category semantic labels to generate text conditions; degraded cloud and fog images are obtained from remote sensing satellite cloud and fog image datasets; a latent diffusion model is trained based on the text conditions and the degraded cloud and fog image conditions. The training process is as follows: the latent diffusion model is copied into two identical parts, namely a "locked" copy and a "trainable" copy; the "locked" copy is frozen, that is, the weights remain unchanged. Fine-tuning of the "trainable" copy is achieved using degraded cloud and fog image conditions. This involves incorporating the degraded cloud and fog image as image conditions into the noise estimation network to estimate the conditional noise distribution. The degraded cloud and fog image conditions are then passed through a learnable embedding layer and... The calculation and training objective are: ; in, The actual noise value follows a standard normal distribution. , This represents the noise value predicted by the model; This indicates the time step in the diffusion process. It is a representation vector in the latent space. This represents the latent variables in the diffusion process at a certain moment during the diffusion process. state, Conditional embedding for degraded cloud and fog images, This indicates text conditional embedding. Indicates the expected value; The fine-tuned "trainable" copy and the output of the trained potential diffusion model are added together, and the cloud-removed remote sensing image is output through the VAE decoder. Using a pre-built remote sensing image semantic extraction model, semantic extraction is performed on remote sensing images after cloud and fog removal, and the content and style information of multi-source remote sensing images are output. To address the content and style information of multi-source remote sensing images, a Transformer-based style transfer model is constructed. This model references a baseline image with suitable radiometric characteristics to achieve uniform illumination and color in the remote sensing images.

2. The method for homogenizing and color-matching multi-source remote sensing images based on style transfer according to claim 1, characterized in that: The standard optical model is as follows: ; in, These are the coordinates of the image pixels. For images with fog, For the dehazed image to be recovered, Transmittance, This refers to the global atmospheric light composition; under cloudy or foggy weather conditions, reduced reflected energy causes transmission attenuation. This leads to a decrease in image brightness; ambient light scattering creates atmospheric light. This enhances image brightness and reduces saturation; cloud and fog images are formed by a camera capturing light reflected from objects weakened by clouds and fog, as well as atmospheric light reflected from clouds and fog. This refers to the weakening and scattering of light by clouds and fog.

3. The method for homogenizing and color-matching multi-source remote sensing images based on style transfer according to claim 1, characterized in that: Methods for constructing semantic extraction models from remote sensing images include: A convolutional network-based encoder is constructed, using ResNet18 as the encoder to extract multi-scale semantic features. It consists of four residual blocks, each of which reduces the scale factor of the feature map by a factor of 2 through downsampling. The feature map generated by each stage is fused with the corresponding feature map in the decoder through 1x1 convolution. The semantic features generated by the residual blocks and the features generated by the global-local transformation blocks in the decoder are aggregated through a weighted summation operation. Construct a Transformer-based decoder that utilizes three global-local attention Transformer blocks and a feature refinement head to build a lightweight Transformer-based decoder. The global-local attention Transformer block constructs two parallel branches to extract global and local contexts, respectively, including: The local branch uses two parallel convolutional layers with core sizes of 3 and 1 respectively to extract the local context; two batch normalization operations are added before the final summation operation. The global branch uses a window-based multi-head self-attention mechanism to capture the global context; it uses a standard 1x1 convolution to expand the channel dimension of the input 2D feature map to three times; then it applies a window segmentation operation to segment the 1D sequence into query Q, key K, and value V vectors.

4. The method for homogenizing and color-matching multi-source remote sensing images based on style transfer according to claim 3, characterized in that: The formula for the weighted summation operation is: ; in, Indicates fusion features, The characteristics representing the generation of residual blocks, This represents the features generated by the global and local Transformer blocks. Indicates the weight.

5. The method for homogenizing and color-matching multi-source remote sensing images based on style transfer according to claim 3, characterized in that: The method for constructing a semantic extraction model for remote sensing images also includes building a cross-shaped window context interaction module. This module captures the global context by fusing two feature maps generated by horizontal average pooling layers and vertical average pooling layers. The horizontal average pooling layer establishes the horizontal relationship between windows; for any point in one of the windows in the horizontal relationship... The corresponding point in another window, in relation to the horizontal relationship. The dependency model is as follows: ； ； ； ； in, Indicates an index. It refers to the window size. This represents self-attention computation, used to model the dependencies between pixel pairs within a local window.

6. The method for homogenizing and color-matching multi-source remote sensing images based on style transfer according to claim 1, characterized in that: Transformer-based style transfer models include: The Transformer encoder is used to feed the image sequence to be processed into the Transformer encoder. Each layer contains a multi-head self-attention module and a feedforward network to convert the input sequence into a query Q, a key K and a value V. The multi-head self-attention module is used to process different heads in parallel through a multi-head self-attention mechanism, calculate attention, and effectively encode the input sequence. The Transformer decoder is used to translate the encoded content sequence in a regression manner based on the encoded style sequence. All sequence blocks are input at once and the output is predicted. The feature sequence is generated by using the content sequence to generate the query Q and the style sequence to generate the key K and value V, and then performing sequence translation. The output sequence of the Transformer decoder is further refined by a three-layer CNN decoder, including convolution, ReLU activation and upsampling.

7. The method for homogenizing and color-matching multi-source remote sensing images based on style transfer according to claim 6, characterized in that: The style transfer model, referencing a baseline image with suitable radiometric characteristics, employs methods to achieve uniform illumination and color in remote sensing images, including: After the input sequence is embedded, it is fed into the Transformer encoder, where it is transformed into a representation of query Q, key K, and value V. ； in, This represents a sequence of input content. , Indicates the sequence length. Indicates the number of heads of attention; Multi-head self-attention mechanisms compute attention by processing different heads in parallel, thus achieving efficient encoding of the input sequence. ； The Transformer decoder follows the coding style sequence Translate encoded content sequences using a regression approach The input to the Transformer decoder includes the encoded content sequence. and style sequence Generate queries using content sequences Use style sequence to generate keys Sum : ； Calculate the output sequence of the Transformer decoder : ； Output sequence of Transformer decoder Having shape ,in and These represent the output height and width, respectively. This represents the number of channels; a three-layer CNN decoder is used to further refine the output of the Transformer decoder; for each layer of the CNN decoder, the scale is increased, including 3x3 convolutions, ReLU activation functions, and 2x upsampling, ultimately obtaining a resolution-sensitive output. The output image is matched to the input image in terms of spatial resolution. Each pixel of the output image contains information from three color channels, generating a uniformly lit and colored remote sensing image.

8. A computer-readable storage medium for storing one or more programs, characterized in that: The one or more programs include instructions that, when executed by a computing device, cause the computing device to perform any of the methods according to claims 1 to 7.

9. A computing device, characterized in that, include: One or more processors, one or more memories, and one or more programs, wherein the one or more programs are stored in the one or more memories and configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods according to claims 1 to 7.

Images