Cloud image efficient synthesis method

Abstract

The invention provides an efficient cloud image synthesis method, which belongs to the field of image processing and comprises the following steps of: calculating edge distance transformation for a mask of a foreground image to generate a multi-level boundary information graph; converting the background image and the foreground image into a Lab color space, inputting the Lab channel, the mask and the multi-level boundary information graph into a generator network together, and outputting a preliminary fusion image in the Lab color space; a brightness discriminator and a chrominance discriminator are adopted to discriminate the local texture consistency of the preliminary fusion image and the real fusion image in the L channel and the global color harmony of the preliminary fusion image and the real fusion image in the a channel and the b channel respectively; and performing adversarial training on the generator network and the discriminator based on the total loss function until the network model converges, and generating a final fusion image by using the converged generator network. According to the method, the high reality sense and consistency of the fused image in detail texture and overall hue can be ensured.

Description

Technical Field

[0001] This invention belongs to the field of image processing, and in particular relates to a method for efficient cloud-based image synthesis. Background Technology

[0002] Image fusion technology aims to integrate complementary information from multiple source images acquired from different sensors or under different conditions to generate a single fused image that is richer in information and easier for human visual perception or computer processing. This technology has wide applications in fields such as medical imaging, remote sensing monitoring, digital photography, and video surveillance. Traditional image fusion methods mainly focus on multi-scale transform (MST). These methods typically involve three steps: First, using a multi-scale decomposition tool, such as Laplacian pyramid (LP), discrete wavelet transform (DWT), or non-subsampled contourlet transform (NSCT), the source image is decomposed into sub-band coefficients of different scales and orientations; second, for the decomposed low-frequency and high-frequency sub-band coefficients, corresponding fusion rules (such as "maximum selection" or "weighted average") are designed for merging; finally, the fused coefficients are subjected to inverse multi-scale transform to reconstruct the final fused image. However, these MST-based methods heavily rely on manually designed fusion rules, which are often heuristic, lack adaptability, and are difficult to fully extract and retain salient features from the source images. They are prone to introducing artifacts, reducing contrast, or causing loss of detail in the fusion result.

[0003] Early deep learning methods typically leveraged the powerful feature extraction capabilities of CNNs to assist or replace certain steps in traditional MST frameworks, such as for activity level measurement or fusion weight decisions. In recent years, Generative Adversarial Networks (GANs) have been introduced into the field of image fusion due to their outstanding performance in image generation tasks. GAN-based fusion methods drive the generator to learn a complex mapping from the source image to a high-quality fused image through an adversarial game between a generator and a discriminator. The discriminator learns to distinguish the differences in the distribution of specific features (such as gradients and textures) between the fused image and the real source image, providing more effective supervision signals for the generator's optimization. This end-to-end learning approach adaptively preserves key information in the source image, resulting in significant improvements in detail preservation and overall visual quality. Nevertheless, designing effective network structures and loss functions that better integrate with multi-scale analysis theory and effectively address the color fidelity issue of color images remains a current research hotspot and challenge. Summary of the Invention

[0004] To address the issues of color distortion and insufficient detail preservation in image fusion, this invention proposes an efficient cloud-based image synthesis method, comprising the following steps:

[0005] The edge distance transformation of the mask in the foreground image is calculated to generate a multi-level boundary information map; the background image and the foreground image are converted to the Lab color space, and the Lab channel, mask and multi-level boundary information map are input into the generator network. The generator network has a built-in Lab cross-channel attention module, which can realize bidirectional attention weighting between the L channel and the a and b channel feature maps, and outputs a preliminary fused image in the Lab color space.

[0006] A luminance discriminator and a chrominance discriminator are used to determine the local texture consistency of the preliminary fused image and the real fused image in the L channel, and the global color harmony in the a and b channels, respectively. The total loss function is obtained by weighting the adversarial loss generated by the discriminator, the L1 pixel loss of the preliminary fused image and the real fused image in the Lab color space, and the gradient correlation loss between Lab channels.

[0007] The generator network and discriminator are subjected to adversarial training based on the total loss function until the network model converges, and the converged generator network is used to generate the final fused image.

[0008] Preferably, the step of generating a multi-level boundary information map by performing mask calculation on the foreground image to calculate edge distance transformation includes:

[0009] A distance transformation map is generated by performing a distance transformation on the binary mask of the foreground image;

[0010] Set a first predetermined distance threshold and a second predetermined distance threshold, wherein the second predetermined distance threshold is greater than the first predetermined distance threshold;

[0011] Generate a first-level boundary information map, in which pixels on the distance transformation map whose values ​​are less than or equal to a first predetermined distance threshold are assigned a value of 1, and the rest are assigned a value of 0;

[0012] A second-level boundary information map is generated, in which pixels on the distance transformation map whose values ​​are greater than the first predetermined distance threshold and less than or equal to the second predetermined distance threshold are assigned a value of 1, and the rest are assigned a value of 0;

[0013] Generate a third-level boundary information map, in which pixels with values ​​greater than the second predetermined distance threshold on the distance transformation map are assigned a value of 1, and the rest are assigned a value of 0;

[0014] The first-level boundary information map, the second-level boundary information map, and the third-level boundary information map are stacked to form the multi-level boundary information map.

[0015] Preferably, the generator network incorporates a Lab cross-channel attention module, which enables bidirectional attention weighting between the L-channel and the a and b-channel feature maps, including:

[0016] Using the L-channel feature map as the query vector, and the concatenated a- and b-channel feature maps as the key vector and value vector, the attention weight of the L-channel on the a- and b-channels is calculated and applied to the value vector to obtain the weighted feature map of the a- and b-channels.

[0017] The concatenated a and b channel feature maps are used as query vectors, and the L channel feature map is used as key and value vectors. The attention weights of the a and b channels on the L channel are calculated and applied to the value vector to obtain the weighted feature map of the L channel.

[0018] The original L, a, and b channel feature maps are residually connected to their corresponding weighted feature maps to achieve the bidirectional attention weighting.

[0019] Preferably, the brightness discriminator is a PatchGAN structure, the input of which is a local image patch cropped from the L channel of the preliminary fused image and the L channel of the real fused image, and the output is the result of judging whether the local image patch is real or fake;

[0020] The color discriminator is a fully convolutional network structure. The input is the a and b channels of the preliminary fused image and the complete image of the a and b channels of the real fused image. The output is a single discrimination score for global color harmony.

[0021] Preferably, the total loss function is set. The calculation formula is:

[0022] ;

[0023] in, To combat the losses, For L1 pixel loss, For the gradient correlation loss between Lab channels, , , These are preset non-zero weighting coefficients.

[0024] Preferably, the calculation of the gradient correlation loss between Lab channels includes:

[0025] The gradient operator is used to calculate the L-channel gradient map, a-channel gradient map, and b-channel gradient map of the preliminary fused image, respectively; the cosine similarity between the L-channel gradient map and the a-channel gradient map is then calculated. And the cosine similarity between the L-channel gradient map and the b-channel gradient map. ;

[0026] The gradient correlation loss Calculated using the following formula: .

[0027] Preferably, the step of performing adversarial training on the generator network and discriminator based on the total loss function until the network model converges includes:

[0028] During training, perform the following steps alternately:

[0029] With the parameters of the generator network fixed, the network parameters of the luminance discriminator and chrominance discriminator are trained and updated.

[0030] The network parameters of the luminance discriminator and chrominance discriminator are fixed, and the parameters of the generator network are trained and updated based on the total loss function;

[0031] The adversarial training continues until a preset convergence condition is met.

[0032] This invention decouples luminance and chromaticity information by performing image fusion in the Lab color space. Through a Lab cross-channel attention module, it achieves precise coordination and transfer of lighting and color features, avoiding color distortion issues that arise when adjusting lighting and allowing foreground objects to blend better into the background environment. Simultaneously, a multi-level boundary information map-guided generator network refines the fusion boundaries, reducing blurring and artifacts at the stitching points and generating smoother, more natural transition regions. Furthermore, separate luminance and chromaticity discriminators constrain the fusion quality from two dimensions: local texture consistency and global color harmony. Combined with gradient correlation loss between Lab channels, this further ensures high realism and consistency in detail texture and overall tone of the fused image, resulting in a more realistic visual effect, more harmonious colors, and superior boundary processing. Attached Figure Description

[0033] Figure 1 This is a flowchart of Example 1;

[0034] Figure 2 A schematic diagram generated for a multi-level boundary information map;

[0035] Figure 3 A schematic diagram to enhance the characteristics of the L channel;

[0036] Figure 4 A schematic diagram to enhance the features of channels a and b. Detailed Implementation

[0037] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0038] Specific Implementation Example 1, such as Figure 1 As shown, a cloud-based efficient image synthesis method includes the following steps:

[0039] S1, calculate the edge distance transformation of the mask of the foreground image to generate a multi-level boundary information map; convert the background image and the foreground image to the Lab color space, and input the Lab channel, mask and multi-level boundary information map into the generator network. The generator network has a built-in Lab cross-channel attention module, which can realize bidirectional attention weighting between the L channel and the a and b channel feature maps, and output a preliminary fused image in the Lab color space.

[0040] Obtain a binarized mask of the foreground object region, and then use an edge detection algorithm such as the Sobal operator to extract its precise boundary contour. Perform a Euclidean distance transformation on the boundary contour map to generate a distance map, where the gray value of each pixel represents its distance to the nearest boundary point. By setting multiple incremental distance thresholds, such as 5 pixels, 15 pixels, and 30 pixels, binarize the distance map to generate three independent boundary region maps. Stack these three maps along the channel dimension to form a multi-level boundary information map, such as... Figure 2 As shown, both the RGB format background and foreground images are converted to the CIELab color space. A multi-channel input tensor is constructed by concatenating the L, a, and b channels of the background image, the L, a, and b channels of the foreground image, a single-channel mask, and the multi-channel, multi-level boundary information map generated in the previous step along the channel dimension. The input tensor is then fed into a generator network using a U-Net structure. In the decoder part of the network, a Lab cross-channel attention module is incorporated. This module establishes a bidirectional attention mechanism between the L-channel feature map and the a and b-channel feature maps. For example, it uses the L-channel features to generate queries and keys, and uses the a and b-channel features to generate values. Weights are calculated using a standard scaled dot product attention mechanism to adjust the color features of the a and b channels, and vice versa, thereby achieving mutual guidance and enhancement of illumination and color.

[0041] S2, a luminance discriminator and a chrominance discriminator are used to determine the local texture consistency of the preliminary fused image and the real fused image in the L channel, and the global color harmony in the a and b channels, respectively; the total loss function is obtained by weighting the adversarial loss generated by the discriminator, the L1 pixel loss of the preliminary fused image and the real fused image in the Lab color space, and the gradient correlation loss between Lab channels.

[0042] In a more specific embodiment, the luminance discriminator is a convolutional neural network based on the PatchGAN architecture. It receives a single L-channel of the pre-fused image or a real fused image as input and outputs a two-dimensional probability map. Each value in the map represents the authenticity of the corresponding local image patch in the input image, thereby judging whether the local texture details of the image are realistic. The chrominance discriminator is a standard global convolutional neural network. It receives the stitched a and b dual-channel images as input, passes through multiple convolutional and fully connected layers, and outputs a single scalar value to determine whether the color distribution of the entire image is globally harmonious and natural. The adversarial loss employs Least Squares Adversarial Loss (LSGAN), calculating the generator's loss relative to the two discriminators. The losses of the two discriminators are derived from local texture consistency loss and global color harmony loss. In a more specific embodiment, the adversarial loss is obtained by adding or weighting the local texture consistency loss corresponding to the luminance discriminator and the global color harmony loss corresponding to the chrominance discriminator. The L1 pixel loss is obtained by summing the absolute values ​​of the differences between corresponding pixel values ​​in the L, a, and b channels of the preliminary fused image and the real fused image. The Lab channel gradient correlation loss first uses the Sobel operator to calculate the gradient maps of each of the L, a, and b channels. Then, it calculates the Pearson correlation coefficients between the L channel gradient and the a channel gradient of the preliminary fused image and the real fused image, as well as the Pearson correlation coefficient between the L channel gradient and the b channel gradient. This loss term represents the L1 distance between these two sets of correlation coefficient pairs, used to maintain structural consistency between illumination and color. These three losses are then weighted and summed using preset weighting coefficients to obtain the total loss.

[0043] S3, perform adversarial training on the generator network and discriminator based on the total loss function until the network model converges, and use the converged generator network to generate the final fused image.

[0044] The training process employs an alternating optimization strategy, using the Adam optimizer for parameter updates. In each training step, the parameters of the generator network are first fixed, and the real image and the preliminary fused image generated by the generator are fed into two discriminators respectively. The loss of the discriminators is calculated, and their network parameters are updated. Then, the parameters of the two discriminators are fixed again, and the generator generates the preliminary fused image again. The total loss, including adversarial loss, L1 pixel loss, and gradient correlation loss, is calculated, and the parameters of the generator network are updated based on this total loss. This process is repeated for a specified number of epochs until the values ​​of each loss function tend to stabilize and the quality of the generated image no longer significantly improves, at which point the model is considered to have converged. After convergence, only the trained generator network is retained for fusing new foreground and background images, and the final fused result is directly output.

[0045] In a specific embodiment two, the step of generating a multi-level boundary information map by calculating the edge distance transformation of the mask of the foreground image includes:

[0046] A distance transformation map is generated by performing a distance transformation on the binary mask of the foreground image;

[0047] Set a first predetermined distance threshold and a second predetermined distance threshold, wherein the second predetermined distance threshold is greater than the first predetermined distance threshold;

[0048] Generate a first-level boundary information map, in which pixels on the distance transformation map whose values ​​are less than or equal to a first predetermined distance threshold are assigned a value of 1, and the rest are assigned a value of 0;

[0049] A second-level boundary information map is generated, in which pixels on the distance transformation map whose values ​​are greater than the first predetermined distance threshold and less than or equal to the second predetermined distance threshold are assigned a value of 1, and the rest are assigned a value of 0;

[0050] Generate a third-level boundary information map, in which pixels with values ​​greater than the second predetermined distance threshold on the distance transformation map are assigned a value of 1, and the rest are assigned a value of 0;

[0051] The first-level boundary information map, the second-level boundary information map, and the third-level boundary information map are stacked to form the multi-level boundary information map.

[0052] Multi-level boundary information maps are used to guide the generator network to pay different levels of attention to different regions of the image, thereby achieving a smoother and more natural transition at the fusion of foreground and background. For example, for a binary mask image of a foreground object, a distance transformation is used to calculate the nearest distance from each pixel in the image to the object's edge, forming a distance transformation map. Assume a first distance threshold of 8 pixels and a second distance threshold of 20 pixels. The first-level boundary information map marks the area within 8 pixels of the object's edge; this is the core fusion area most prone to imperfections such as halos or hard edges, requiring the finest adjustments. The second-level boundary information map marks the transition area between 8 and 20 pixels from the edge; the fusion importance of this area is secondary. The third-level boundary information map marks the foreground interior area more than 20 pixels from the edge; here, more attention should be paid to preserving the texture and content of the foreground itself. The first, second, and third-level boundary information maps are then combined... Figure 3 The single-channel images are stacked into a three-channel image, providing the generator with explicit spatial guidance information, enabling it to perform differentiated fusion processing strategies for different regions.

[0053] In a specific embodiment three, the generator network incorporates a Lab cross-channel attention module, which enables bidirectional attention weighting between the L-channel and the a and b-channel feature maps, including:

[0054] Using the L-channel feature map as the query vector, and the concatenated a- and b-channel feature maps as the key vector and value vector, the attention weight of the L-channel on the a- and b-channels is calculated and applied to the value vector to obtain the weighted feature map of the a- and b-channels.

[0055] The concatenated a and b channel feature maps are used as query vectors, and the L channel feature map is used as key and value vectors. The attention weights of the a and b channels on the L channel are calculated and applied to the value vector to obtain the weighted feature map of the L channel.

[0056] The original L, a, and b channel feature maps are residually connected to their corresponding weighted feature maps to achieve the bidirectional attention weighting.

[0057] This embodiment primarily addresses the issue of brightness and chromaticity inconsistency in image fusion. In the Lab color space, the L channel represents luminance, while the a and b channels represent chromaticity. By establishing a bidirectional attention mechanism between the two, it ensures that luminance adjustments are referenced to color information, and vice versa, thereby achieving light and color harmony. For example, when a warm-toned, bright foreground object is placed against a cool-toned, dark background, this module will engage in bidirectional information exchange.

[0058] On one hand, the network uses the L-channel feature map as a query to explore the a and b-channel color features corresponding to the highlighted areas. The calculated attention weights guide the network to reduce the saturation of the foreground object's color and integrate it with the cool tones of the background, making the color more compatible with the ambient lighting. Figure 3 As shown. On the other hand, the network uses the a and b channel feature maps as queries to explore the L channel brightness features corresponding to warm color regions, such as... Figure 4 As shown, this prompts the network to do more than simply darken the foreground when adjusting its brightness; instead, it simulates specific lighting effects caused by cool ambient light. By adding these weighted features to the original features through residual connections, the network integrates light and color correlation information while preserving important details of the original image, resulting in a more realistic final output in terms of both lighting and color.

[0059] In specific embodiment four, the brightness discriminator is a PatchGAN structure. The input is a local image patch cropped from the L channel of the preliminary fused image and the L channel of the real fused image, and the output is the real or fake discrimination result of the local image patch.

[0060] The color discriminator is a fully convolutional network structure. The input is the a and b channels of the preliminary fused image and the complete image of the a and b channels of the real fused image. The output is a single discrimination score for global color harmony.

[0061] This embodiment employs two independent discriminators to specifically and efficiently evaluate two core dimensions of image harmony: local lighting realism and global color harmony, providing more targeted optimization guidance for the generator. The luminance discriminator uses a PatchGAN architecture. Instead of judging the overall image's authenticity, it divides the input L-channel image into several overlapping local image patches, such as 64x64 pixel patches, and judges the authenticity of each patch. This approach forces the generator not only to produce an overall pleasing image but also to ensure that the lighting, texture, and details of each local area are sufficiently realistic, effectively identifying and penalizing subtle imperfections at the blending boundaries. The chroma discriminator, on the other hand, focuses on global color. It receives the complete a and b channel images and uses a fully convolutional network structure to evaluate whether the color scheme of the entire image is natural and unified. For example, it can determine whether an object placed in a blue-toned night scene has been correctly given a cooler hue and provides a total global harmony score, ensuring that the generated image achieves a high degree of realism in both detail and overall quality.

[0062] In a specific embodiment five, the total loss function is defined. The calculation formula is:

[0063] ;

[0064] in, To combat the losses, For L1 pixel loss, For the gradient correlation loss between Lab channels, , , These are preset non-zero weighting coefficients.

[0065] The overall loss function guides the training of the network model from multiple perspectives by weightedly combining various loss types to generate high-quality fused images. Each loss component plays a different role, and their importance is balanced by preset weight coefficients. For example, weights can be set... =1, It is 100. The value is 1. More specifically, combating losses. Provided by a discriminator network, its goal is to drive the generator to produce images that the discriminator cannot distinguish from real images, primarily responsible for improving the realism and naturalness of the generated results. L1 pixel loss. By calculating the sum of the absolute differences between each pixel value in the generated image and the real target image, the generated image is forced to approximate the real image in terms of overall content, color, and structure, which can accelerate model convergence and ensure basic quality. Gradient correlation loss This focuses on image details, ensuring consistency in brightness and color trends to avoid issues like color bleeding. Combining these three aspects allows the model to pursue both macroscopic similarity and microscopic realism and harmony of details during optimization.

[0066] In specific embodiment six, the calculation of the gradient correlation loss between Lab channels includes:

[0067] The gradient operator is used to calculate the L-channel gradient map, a-channel gradient map, and b-channel gradient map of the preliminary fused image, respectively; the cosine similarity between the L-channel gradient map and the a-channel gradient map is then calculated. And the cosine similarity between the L-channel gradient map and the b-channel gradient map. ;

[0068] The gradient correlation loss Calculated using the following formula: .

[0069] In natural images, brightness and color often change synchronously at object edges or where textures change drastically. The Lab channel gradient correlation loss aims to mathematically force the generated image to mimic this characteristic, resulting in more realistic details. During computation, operators such as Sobal are applied to the L, a, and b channels of the generated image, respectively, to obtain three gradient maps. , and These three images highlight the locations where brightness, red-green color pairs, and yellow-blue color pairs undergo dramatic changes. Then, the cosine similarity between the L-channel gradient image and the a-channel gradient image is calculated. This measures the correlation between changes in brightness and changes in red and green colors. A value close to 1 indicates that the changes in the two are highly synchronized. Similarly, calculating... This is used to measure the correlation between brightness and the yellow / blue color. During model training, the optimization goal is to minimize this loss value, which is equivalent to maximizing... and For example, if It is 0.95. If the value is 0.92, the loss is 0.13, which is a low value, indicating that the brightness and chromaticity gradients are well correlated and the image details transition naturally.

[0070] Specific embodiment seven, the step of performing adversarial training on the generator network and discriminator based on the total loss function until the network model converges includes:

[0071] During training, perform the following steps alternately:

[0072] With the parameters of the generator network fixed, the network parameters of the luminance discriminator and chrominance discriminator are trained and updated.

[0073] The network parameters of the luminance discriminator and chrominance discriminator are fixed, and the parameters of the generator network are trained and updated based on the total loss function;

[0074] The adversarial training continues until a preset convergence condition is met.

[0075] The model training process employs an alternating optimization approach, where progress in one area motivates further improvement in the other, ultimately achieving a high level of equilibrium. Each training iteration comprises two distinct steps. The first step involves training the discriminator. In this step, all parameters of the generator are temporarily locked and not updated. Then, a batch of real fused images and a batch of fake images generated by the current generator are input into the luminance discriminator and chrominance discriminator, respectively. The discriminator aims to learn how to accurately assign high scores to real images and low scores to fake images, updating its network parameters through backpropagation to improve its discrimination ability. The second step involves training the generator. At this point, the parameters of both discriminators are locked. The generator produces a new batch of images and feeds them into the fixed discriminators for evaluation. Adversarial loss is calculated based on the discriminator feedback and combined with L1 loss and gradient correlation loss to form the total loss. This total loss is used to update the generator network's parameters, enabling its generated images to better deceive the discriminators and more closely resemble real images in the next iteration. The alternating training cycle will continue, for example, for tens of thousands of iterations, until the model's performance no longer shows significant improvement, or the loss function value tends to stabilize, that is, it reaches the convergence state.

[0076] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. In addition, various different implementations of the embodiments of the present invention can be arbitrarily combined, as long as they do not violate the spirit of the embodiments of the present invention, and they should also be regarded as the content disclosed in the embodiments of the present invention.

Claims

1. A method for efficient cloud-based image synthesis, characterized in that, Includes the following steps: The edge distance transformation of the mask in the foreground image is calculated to generate a multi-level boundary information map; the background image and the foreground image are converted to the Lab color space, and the Lab channel, mask and multi-level boundary information map are input into the generator network. The generator network has a built-in Lab cross-channel attention module, which can realize bidirectional attention weighting between the L channel and the a and b channel feature maps, and outputs a preliminary fused image in the Lab color space. A luminance discriminator and a chrominance discriminator are used to determine the local texture consistency in the L channel and the global color harmony in the a and b channels of the preliminary fused image and the real fused image, respectively. The total loss function is obtained by weighting the adversarial loss generated by the discriminator, the L1 pixel loss of the preliminary fused image and the real fused image in the Lab color space, and the gradient correlation loss between Lab channels. The generator network and discriminator are subjected to adversarial training based on the total loss function until the network model converges, and the converged generator network is used to generate the final fused image.

2. The method according to claim 1, characterized in that, The step of calculating edge distance transformation of the foreground image using a mask to generate a multi-level boundary information map includes: A distance transformation map is generated by performing a distance transformation on the binary mask of the foreground image; Set a first predetermined distance threshold and a second predetermined distance threshold, wherein the second predetermined distance threshold is greater than the first predetermined distance threshold; Generate a first-level boundary information map, in which pixels on the distance transformation map whose values ​​are less than or equal to a first predetermined distance threshold are assigned a value of 1, and the rest are assigned a value of 0; A second-level boundary information map is generated, in which pixels on the distance transformation map whose values ​​are greater than the first predetermined distance threshold and less than or equal to the second predetermined distance threshold are assigned a value of 1, and the rest are assigned a value of 0; Generate a third-level boundary information map, in which pixels with values ​​greater than the second predetermined distance threshold on the distance transformation map are assigned a value of 1, and the rest are assigned a value of 0; The first-level boundary information map, the second-level boundary information map, and the third-level boundary information map are stacked to form the multi-level boundary information map.

3. The method according to claim 1, characterized in that, The generator network incorporates a Lab cross-channel attention module, which enables bidirectional attention weighting between the L-channel and the a and b-channel feature maps, including: Using the L-channel feature map as the query vector, and the concatenated a- and b-channel feature maps as the key vector and value vector, the attention weight of the L-channel on the a- and b-channels is calculated and applied to the value vector to obtain the weighted feature map of the a- and b-channels. The concatenated a and b channel feature maps are used as query vectors, and the L channel feature map is used as key and value vectors. The attention weights of the a and b channels on the L channel are calculated and applied to the value vector to obtain the weighted feature map of the L channel. The original L, a, and b channel feature maps are residually connected to their corresponding weighted feature maps to achieve the bidirectional attention weighting.

4. The method according to claim 1, characterized in that, The brightness discriminator is a PatchGAN structure. Its input is a local image patch cropped from the L channel of the preliminary fused image and the L channel of the real fused image. Its output is the result of judging whether the local image patch is real or fake.

5. The method according to claim 4, characterized in that, The colorimetric discriminator is a fully convolutional network structure.

6. The method according to claim 5, characterized in that, The input to the fully convolutional network structure is the complete image of the a and b channels of the preliminary fused image and the a and b channels of the real fused image, and the output is a single discrimination score for global color harmony.

7. The method according to claim 1, characterized in that, The total loss function The calculation formula is: ； in, To combat the losses, For L1 pixel loss, For the gradient correlation loss between Lab channels, , , These are preset non-zero weighting coefficients.

8. The method according to claim 5, characterized in that, The calculation of the gradient correlation loss between Lab channels includes: The gradient operator is used to calculate the L-channel gradient map, a-channel gradient map, and b-channel gradient map of the preliminary fused image, respectively; the cosine similarity between the L-channel gradient map and the a-channel gradient map is then calculated. And the cosine similarity between the L-channel gradient map and the b-channel gradient map. .

9. The method according to claim 8, characterized in that, The gradient correlation loss between Lab channels Calculated using the following formula: .

10. The method according to claim 1, characterized in that, The adversarial training of the generator network and discriminator based on the total loss function until the network model converges includes: During training, perform the following steps alternately: With the parameters of the generator network fixed, the network parameters of the luminance discriminator and chrominance discriminator are trained and updated. The network parameters of the luminance discriminator and chrominance discriminator are fixed, and the parameters of the generator network are trained and updated based on the total loss function; The adversarial training continues until a preset convergence condition is met.

Images