Remote sensing shadow removal method and system based on dynamic feature fusion and diffusion optimization

Abstract

The application discloses a remote sensing shadow removal method and system based on dynamic feature fusion and diffusion optimization, and the method comprises the following steps: S1, acquiring a non-shadow image, a shadow image and a corresponding shadow mask; S2, extracting global illumination features of a non-shadow area and high-frequency texture features of a shadow area; S3, performing dynamic weighted fusion on the global illumination features and the high-frequency texture features; S4, inputting a coarse shadow removal result into a lightweight conditional diffusion refinement network, performing iterative denoising under a shared time step scheduling, training a diffusion model by taking the non-shadow image as a supervision target, and obtaining a trained diffusion model; and S5, inputting a remote sensing image to be processed into the trained diffusion model, and outputting a final shadow removal result. Through self-supervised pre-training, the application reduces data dependence, and through progressive recovery guided by a shared time step, the application realizes the coordination and unification of global illumination correction and local texture enhancement.

Description

Technical Field

[0001] This invention relates to the technical field of shadow removal from remote sensing images, and specifically to a method and system for shadow removal from remote sensing images based on dynamic feature fusion and diffusion optimization. Background Technology

[0002] Shadow removal is crucial for ensuring the accuracy of scene interpretation in UAV remote sensing imagery and the effectiveness of downstream visual tasks. Due to the complex lighting conditions and diverse scene structures in UAV imagery, shadow areas introduce local radiometric distortion, severely degrading image quality. Existing methods often struggle to achieve a consistent balance between global illumination correction and fine-grained texture restoration, leading to problems such as over-smoothing or local artifacts in complex scenes.

[0003] Traditional shadow removal methods primarily rely on artificial priors constructed based on lighting, gradient, or chromaticity information. Representative methods recover shadow-free areas by suppressing shadow gradients or modeling them as relighting problems. However, these methods are often based on simplistic assumptions about lighting and scene structure, and lack robustness when dealing with drone scenes with diverse materials and varying perspectives.

[0004] With the development of deep learning, supervised methods have significantly improved reconstruction performance by learning the mapping from shadowed images to shadowless images using pairwise data. However, most of these methods employ a deterministic, single-step reconstruction framework, making it difficult to simultaneously maintain global illumination consistency and local texture detail. Furthermore, their performance is highly dependent on large-scale pairwise datasets, and the high cost of acquiring such data for UAV imagery limits their widespread adoption in practical applications.

[0005] To reduce the reliance on paired annotations, researchers have further explored unsupervised and weakly supervised methods to improve data flexibility. However, due to the lack of explicit supervised priors in the shadow region, accurate recovery of high-frequency structures remains difficult, often sacrificing local texture fidelity for global smoothness.

[0006] In recent years, diffusion models have provided a new technical approach for shadow removal due to their generative capabilities in realistic texture and illumination restoration. Existing diffusion model methods, such as ShadowDiffusion, DeS3, and BCDiff, have made some progress in shadow detection and removal through mask-aware frameworks, adaptive attention mechanisms, or boundary-aware strategies. However, existing diffusion-based methods still face a common technical bottleneck: the lack of a unified stage coordination mechanism for illumination correction, structure restoration, and detail enhancement. The stages are relatively loosely coupled, lacking unified control across stages. This makes it difficult for existing methods to achieve progressive coordination from global illumination correction to local texture enhancement during the restoration process, easily leading to problems such as overly smooth structures or inconsistent local details in complex UAV scenes. Summary of the Invention

[0007] To overcome the shortcomings of existing technologies, one of the objectives of this invention is to provide a remote sensing shadow removal method based on dynamic feature fusion and diffusion optimization. This method reduces data dependence through self-supervised pre-training and achieves coordinated unity between global illumination correction and local texture enhancement through progressive recovery guided by shared time steps.

[0008] To achieve one of the objectives of this invention, the following solution is adopted: The remote sensing shadow removal method based on dynamic feature fusion and diffusion optimization includes the following steps: S1: Obtain the shadowless image for training, the shadowed image for training and inference, and the corresponding shadow mask; use the shadowless image to repair the pre-trained initial dual-branch encoder network through self-supervised image repair to obtain the pre-trained encoder parameters; S2: Input the shadow image and the shadow mask into the background encoder and texture encoder initialized by the pre-trained encoder parameters, respectively, to extract the global illumination features of the non-shadow area and the high-frequency texture features of the shadow area; S3: Guided by the shared time step, the global illumination features and the high-frequency texture features are dynamically weighted and fused, and a coarse deshading result is generated after decoding; S4: Input the coarse deshading result into a lightweight conditional diffusion refinement network, perform iterative denoising under the shared time step scheduling, train the diffusion model with the shadowless image as the supervised target, and obtain the trained diffusion model. S5: Input the remote sensing image to be processed into the trained diffusion model and output the final shadow removal result.

[0009] Further, in step S1, the dual-branch encoder network is initialized through self-supervised image inpainting pre-training using the shadowless image, specifically as follows: An irregular binary mask is randomly generated from the shadowless image to simulate the missing region. The shadowless image is then multiplied element-wise with the inverse code of the mask to obtain the damaged input image. The pre-trained network takes the damaged input image and the mask as input, reconstructs the original shadowless image based on the visible context region, and optimizes it using the absolute error between the reconstructed image and the shadowless image as the loss function. After pre-training, the encoder parameters are transferred to the dual-branch feature encoder in the subsequent shadow removal framework, and the decoder is discarded.

[0010] Further, in step S2, the extraction of global illumination features of non-shaded areas and high-frequency texture features of shaded areas specifically involves: Two parallel encoders, a texture encoder and a background encoder, are used to extract complementary features; The texture encoder takes the stitching result of the shadow image and the shadow mask as input, preserves the high-frequency details inside the shadow area, and generates a texture feature map. The background encoder processes the image obtained by element-wise multiplication of the shadow image and the inverse code of the shadow mask, capturing global color and lighting information from the non-shadow areas to generate a background feature map.

[0011] Furthermore, in step S3, guided by the shared time step, the illumination features and texture features are dynamically weighted and fused, specifically as follows: A time step is introduced to represent the stage variable of the recovery stage. This time step is encoded by sinusoidal position embedding combined with a multilayer perceptron to obtain the stage representation. The stage representation is then mapped to a bias term. The texture feature map and the background feature map are concatenated along the channels and then input into the convolutional layer to obtain static fusion information. The static fusion information is added element by element to the bias term and then mapped by the Sigmoid activation function to generate two time-step-aware dynamic fusion weights. The texture feature map and the background feature map are multiplied element-wise with their respective dynamic fusion weights and then added together. The results are then fused through a convolutional layer to obtain the fused features at the current time step. A lightweight decoder consisting of eight residual blocks, two transposed convolutions, and one standard convolutional layer upsamples the fused features to generate a coarse, shadowless prediction result.

[0012] Further, in step S4, the coarse deshading result is input into a lightweight conditional diffusion refinement network for iterative denoising, specifically as follows: The coarse deshading result is used as the structured initial state, and together with the shadow image and the shadow mask, a conditional signal is constructed and input into a lightweight conditional diffusion refinement network. Using a diffusion model framework that predicts clean images, the forward diffusion process obtains a noisy state by gradually adding Gaussian noise to the coarse prediction results. The diffusion thinning network takes noisy state, conditional signal and time step as input, and outputs a thinned shadowless image. During training, the parameters of the coarse prediction network are frozen, and only the diffusion refinement network is optimized. The loss function is a weighted combination of diffusion reconstruction loss and perception loss, where diffusion reconstruction loss is the mean square error between the refinement result and the real shadowless image, and perception loss is the perceptual similarity loss between the two.

[0013] Further, in step S4, the shared time step scheduling specifically includes: Align the feature fusion stage and the diffusion refinement stage to the same monotonically decreasing time step scheduling sequence, ensuring that the time steps of the fusion stage are equal to the time steps of the diffusion refinement stage. Under the coordination of the shared time step, a larger time step causes the fusion process to be biased towards global illumination correction and the denoising process to be biased towards large-scale structure restoration, while a smaller time step causes the fusion process to be biased towards texture detail enhancement and the denoising process to be biased towards local fine-tuning, thereby achieving a progressive restoration from global illumination correction to local texture enhancement.

[0014] Furthermore, step S5 specifically includes: The remote sensing image to be processed is input into the trained coarse prediction network to obtain the coarse prediction result. The coarse prediction result is input into the trained diffusion model, and deterministic denoising diffusion implicit model sampling is performed. The coarse prediction result is used as a low-noise initialization state corresponding to the maximum sampling time step. The coarse prediction network is executed only once at the initial time step to generate a time-step-aware initialization result. Then, deterministic sampling is performed in descending order of time steps to gradually refine the prediction result. In each sampling step, the diffusion refinement network predicts the clean estimate at the current time step and calculates the refined sample for the next time step according to the deterministic denoising update rule, combining the noisy sample at the current time step, the clean estimate at the current time step, and the signal retention ratio of the previous and next time steps, and finally obtains the refined shadowless image.

[0015] Furthermore, the remote sensing images include UAV remote sensing images.

[0016] Furthermore, the self-supervised image inpainting pre-training is completed using only shadowless images and random masks, requiring only a small amount of paired shadow-no-shadow training data to complete subsequent training.

[0017] The second objective of this invention is to provide a remote sensing shadow removal system based on dynamic feature fusion and diffusion optimization, which reduces data dependence through self-supervised pre-training and achieves coordinated unity of global illumination correction and local texture enhancement through progressive recovery guided by shared time steps.

[0018] To achieve the second objective of this invention, the following solution is adopted: A remote sensing shadow removal system based on dynamic feature fusion and diffusion optimization includes: The data acquisition and pre-training module is used to acquire shadowless images for training, shadowed images for training and inference, and corresponding shadow masks; and to use the shadowless images to perform self-supervised image repair pre-training to initialize the dual-branch encoder network and obtain the pre-trained encoder parameters. The feature extraction module is used to input the shadow image and the shadow mask into the background encoder and texture encoder initialized by the pre-trained encoder parameters, respectively, to extract the global illumination features of the non-shadow area and the high-frequency texture features of the shadow area; The dynamic fusion module is used to dynamically weight and fuse the global illumination features and the high-frequency texture features under the guidance of a shared time step, and generate a coarse deshading result after decoding; The diffusion refinement module is used to input the coarse deshading result into a lightweight conditional diffusion refinement network, perform iterative denoising under the shared time step scheduling, train the diffusion model with the shadowless image as the supervised target, and obtain the trained diffusion model. The inference module is used to input the remote sensing image to be processed into the trained diffusion model and output the final shadow removal result.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention initializes a bi-branch encoder network through self-supervised image inpainting pre-training, enabling the encoder to learn prior knowledge of image structure and texture without any paired labeled data during the pre-training stage. In subsequent shadow removal training, this pre-training mechanism significantly reduces the need for paired shadow-free training data, effectively alleviating the problems of high cost and difficulty in acquiring paired remote sensing image data, and significantly improving the feasibility of deploying the method in practical applications.

[0020] 2. This invention unifies the scheduling of the feature fusion stage and the diffusion refinement stage through a shared time step mechanism. This allows both stages to undertake global illumination correction and local detail restoration tasks respectively under the coordination of the same time step, achieving a progressive restoration from coarse-grained structure restoration to fine-grained texture enhancement. This coordination mechanism effectively overcomes the shortcomings of existing single-step restoration frameworks that struggle to balance global illumination consistency and local texture fidelity, significantly improving the visual consistency and realism of shadow removal results in complex scenes.

[0021] 3. This invention employs a dual-branch encoder to extract global illumination features from non-shaded areas and high-frequency texture features from shaded areas, achieving separate extraction of illumination and texture information. This avoids mutual interference and feature aliasing between the two types of information during a single encoding process, providing higher-quality feature input for the subsequent fusion stage.

[0022] 4. Under the guidance of a shared time step, this invention performs dynamic weighted fusion of global illumination features and high-frequency texture features, so that the fusion weight can be automatically adjusted according to the current recovery progress. In the early stage of recovery, the focus is on illumination correction, and in the later stage of recovery, the focus is on detail enhancement. This achieves an adaptive and smooth transition from global illumination to local texture, avoiding the problems of oversmoothing or loss of detail caused by fixed fusion weights.

[0023] 5. This invention uses the coarse shadow removal result as the input to a lightweight conditional diffusion refinement network, and refines the coarse prediction result step by step through iterative denoising, effectively eliminating the residual artifacts that are difficult to avoid in a single-step deterministic recovery framework, and improving the clarity and texture realism of the shadow removal result.

[0024] 6. This invention uses shared time step scheduling to unify feature fusion and diffusion refinement under the same progressive restoration framework, enabling the entire shadow removal process to complete a coordinated transition from illumination correction to texture restoration under a unified control signal. High-quality shadow removal results can be obtained without relying on complex post-processing steps, simplifying the processing flow and reducing computational overhead. Attached Figure Description

[0025] Figure 1 This is a flowchart of the remote sensing shadow removal method based on dynamic feature fusion and diffusion optimization in an embodiment of the present invention; Figure 2 This is a schematic diagram of the dynamic feature fusion and diffusion optimization network model structure in an embodiment of the present invention; Figure 3 This is a schematic diagram of the dynamic fusion module in an embodiment of the present invention; Figure 4 This is a schematic diagram of a test for shadow removal from UAV remote sensing images in an embodiment of the present invention; Figure 5 This is a block diagram of the remote sensing shadow removal system based on dynamic feature fusion and diffusion optimization in an embodiment of the present invention. Detailed Implementation

[0026] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments. It should be noted that, without conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.

[0027] Example 1 like Figure 1-4 As shown, this invention provides a remote sensing shadow removal method based on dynamic feature fusion and diffusion optimization. It can effectively solve the problems of difficulty in coordinating global illumination correction and local texture restoration, shadow boundary artifact residue and high-frequency detail loss in shadow removal of UAV remote sensing images. It significantly improves the shadow removal quality and visual consistency in complex scenes, while greatly reducing the dependence on paired shadow training data and saving data acquisition and annotation costs.

[0028] The remote sensing shadow removal method based on dynamic feature fusion and diffusion optimization according to embodiments of the present invention includes the following steps: S1: Obtain the shadowless image for training, the shadowed image for training and inference, and the corresponding shadow mask; use the shadowless image to repair the pre-trained initial dual-branch encoder network through self-supervised image repair to obtain the pre-trained encoder parameters.

[0029] S2: Input the shadow image and the shadow mask into the background encoder and texture encoder initialized by the pre-trained encoder parameters, respectively, to extract the global illumination features of the non-shadow area and the high-frequency texture features of the shadow area.

[0030] S3: Guided by the shared time step, the global illumination features and the high-frequency texture features are dynamically weighted and fused, and a coarse deshading result is generated after decoding.

[0031] S4: Input the coarse deshading result into a lightweight conditional diffusion refinement network, perform iterative denoising under the shared time step scheduling, and train the diffusion model with the shadowless image as the supervised target to obtain the trained diffusion model.

[0032] S5: Input the remote sensing image to be processed into the trained diffusion model and output the final shadow removal result.

[0033] The remote sensing shadow removal method based on dynamic feature fusion and diffusion optimization according to the embodiments of the present invention will be further described in detail below.

[0034] The remote sensing shadow removal method based on dynamic feature fusion and diffusion optimization in this invention includes: acquiring shadowless UAV images, shadowed images, and corresponding shadow masks; initializing a dual-branch encoder network through self-supervised image inpainting pre-training; inputting the shadow images and masks into a background encoder and a texture encoder, respectively, to extract global illumination features of the non-shadowed areas and high-frequency texture features of the shadowed areas; dynamically weighting and fusing the illumination features and texture features under the guidance of a shared time step, and generating a coarse shadow removal result after decoding; inputting the coarse shadow removal result into a lightweight conditional diffusion refinement network, performing iterative denoising under shared time step scheduling, and training the diffusion model; inputting the UAV image to be processed into the trained diffusion model, and outputting the final shadow removal result.

[0035] The remote sensing shadow removal method based on dynamic feature fusion and diffusion optimization in this invention provides structural and texture priors for the dual-branch encoder through self-supervised image inpainting pre-training, enabling the model to achieve good restoration capabilities with a small amount of paired data. A shared time-step mechanism unifies the scheduling of the dynamic fusion and diffusion refinement processes, achieving progressive restoration from global illumination correction to local texture enhancement. Time-step-aware dynamic fusion weights adaptively integrate illumination and texture features, effectively avoiding the oversmoothing and detail loss problems common in single-step restoration frameworks. A lightweight conditional diffusion refinement network iteratively refines the coarse shadow removal results, further eliminating residual artifacts and restoring high-frequency details. This method significantly improves the quality and visual consistency of shadow removal in UAV remote sensing images with only a small amount of paired training data, while reducing data acquisition and annotation costs.

[0036] Specifically, in step S1, the process of acquiring the UAV's shadowless image, shadowed image, and corresponding shadow mask, and initializing the dual-branch encoder network through self-supervised image inpainting pre-training includes: Given an image without shadows Randomly generate an irregular binary mask. To simulate missing regions. Damaged input. Represented as: Where ⊙ denotes element-wise multiplication. Pre-trained network The aim is to reconstruct the original image based on the visible context region: The network optimizes by reconstructing the target: After pre-training is complete, the encoder parameters will be transferred to the dual-branch feature encoder in the subsequent shadow removal framework, while the decoder will be discarded.

[0037] In this embodiment, the unshadowed image, the shadowed image, and the corresponding shadow mask of the UAV are obtained. The dual-branch encoder network is initialized by pre-training self-supervised image inpainting, which can provide structural and texture priors for the dual-branch encoder, enabling the model to obtain good recovery ability under the condition of a small amount of paired data.

[0038] Specifically, in step S2, the process of inputting the shadow image and the mask into the background encoder and the texture encoder respectively, and extracting the global illumination features of the non-shadow area and the high-frequency texture features of the shadow area includes: Given a shadow image and its corresponding mask It employs two parallel encoders, namely texture encoders. and background encoder To extract complementary features.

[0039] Texture encoder with stitched As input, to preserve high-frequency details within the shaded area, a feature map is generated. .

[0040] The background encoder processes the mask image. It captures global color and lighting information from non-shaded areas to generate features. .

[0041] In this embodiment, by inputting the shadow image and the mask into the background encoder and the texture encoder respectively, the global illumination features of the non-shadow area and the high-frequency texture features of the shadow area are extracted. This enables the decoupling extraction of texture details in the shadow area and illumination information in the non-shadow area, providing a feature basis for subsequent stage-aware dynamic fusion.

[0042] Specifically, in step S3, the process of dynamically weighting and fusing illumination features and texture features under the guidance of a shared time step, and then decoding to generate a coarse deshading result includes: To achieve feature integration for stage awareness, a stage variable, time step t, is introduced to represent the recovery stage. This variable is encoded using sinusoidal position embedding combined with a multilayer perceptron (MLP) to obtain the stage representation. Subsequently, the embedding is mapped to a bias term: The fusion weights are calculated as follows: in This represents a convolutional layer with a two-channel output, where σ is the sigmoid function.

[0043] Time step dependent bias A global stage-aware modulation is introduced into the fusion process. As the time step t decreases, the fusion focus gradually shifts from background features dominated by illumination to texture-sensitive shadow features. Since larger time steps correspond to the early stages of restoration, where the reliability of details is limited, the fusion process naturally prioritizes globally consistent illumination cues before gradually focusing on fine-grained texture restoration. Subsequently, the fused features are obtained using the following formula: A lightweight decoder consisting of eight residual blocks, two transposed convolutions, and one standard convolutional layer will... Upsampling is performed to generate a coarse, shadowless prediction result. .

[0044] In this embodiment, by dynamically weighting and fusing illumination features and texture features under the guidance of a shared time step, the time step-dependent bias adaptively adjusts the fusion weight as the recovery stage progresses. Larger time steps are biased towards global illumination correction, while smaller time steps are biased towards texture detail enhancement, thus achieving a progressive recovery from coarse to fine.

[0045] Specifically, in step S4, the coarse deshading result is input into a lightweight conditional diffusion refinement network, and iterative denoising is performed under a shared time-step scheduling. The process of training the diffusion model includes: The coarse deshading result generated in step S3 As the initial structured state, with the original shadow image Shadow mask Jointly construct conditional signals Input lightweight conditional diffusion refinement network ; use - The predicted diffusion model framework, with the forward diffusion process coarsely predicted by progressively injecting Gaussian noise perturbations: in Indicates time step The noisy coarse prediction results are as follows. The cumulative signal retention ratio; Diffusion Refinement Network In noisy state , condition signal and time step As input, output a refined, shadowless image: During training, the parameters of the coarse prediction network are frozen, and only the diffusion refinement network is optimized. The loss function is a weighted combination of diffusion reconstruction loss and perception loss: in To produce a true, shadow-free image. The weighting coefficients for perceived loss.

[0046] Understandably, by inputting the coarse deshading result into a lightweight conditional diffusion refinement network for iterative denoising, and maintaining semantic consistency with the feature fusion stage under shared time step scheduling, residual shadow artifacts can be effectively eliminated and high-frequency texture details can be restored.

[0047] Furthermore, the specific implementation methods for iterative denoising under shared time step scheduling include: Align the feature fusion stage and the diffusion refinement stage to the same monotonically decreasing time step scheduling. Below, satisfy ,in and These represent the time steps of the feature fusion stage and the diffusion refinement stage, respectively. With the coordination of shared time steps, larger time steps make the fusion process biased towards global illumination correction and the denoising process biased towards large-scale structure restoration, while smaller time steps make the fusion process biased towards texture detail enhancement and the denoising process biased towards local fine-tuning, thus achieving a progressive restoration from global illumination correction to local texture enhancement.

[0048] Specifically, in step S5, the process of inputting the UAV image to be processed into the trained diffusion model and outputting the final shadow removal result includes: The drone imagery to be processed is input into a trained coarse prediction network to obtain coarse prediction results. Input the trained diffusion model and perform deterministic denoising diffusion implicit model (DDIM) sampling. As for the maximum DDIM time step The corresponding low-noise initialization state; The coarse prediction network only predicts at the initial time step Execute once to generate the initialization result for time-step awareness; subsequent executions will follow the same steps. DDIM sampling is performed in decreasing order to gradually refine the prediction results; In each sampling step, the diffusion refinement network predicts the clean estimate at the current time step. Update the sample according to the following formula: Final prediction results That is, the thinned image without shadows. .

[0049] Build as Figure 2 The shadow removal network shown includes a time-step-aware dynamic fusion module, as follows: Figure 3 As shown. The fusion phase introduces a time step variable. This time step is used to characterize the current recovery progress. It is first sinusoidally position-encoded and then input into a multilayer perceptron (MLP) network to generate learnable bias terms. Meanwhile, the texture features output by the texture encoder Background features output by the background encoder After concatenating along the channel dimension, a 3×3 convolutional layer is input to extract preliminary fusion information; then the output of this convolution is compared with a bias term. Element-wise summation, followed by mapping using a sigmoid activation function, yields time-step-aware dynamic fusion weights. and Finally, the texture features and background features are multiplied by their corresponding weights and then summed. These sums are then fused through a 3×3 convolutional layer to output the fused features at the current time step. .

[0050] Without loss of generality, This is a sample of shadow images of the drone to be tested. For the restored image after shadow removal by the method of the present invention, then It can be represented as: in This represents a coarse-shading network. This indicates a diffusion-refined network. For shadow masking, For conditional signals, For shared time step scheduling sequences.

[0051] like Figure 4 The diagram shown illustrates a test of shadow removal in drone images.

[0052] In an embodiment, by means of, Figure 2 (a) shows a self-supervised image inpainting pre-training method that provides structural and textural priors for the bi-branch encoder, enabling the model to achieve good recovery capabilities even with limited paired data. Through methods such as... Figure 2 The shared time-step mechanism shown in (b) and (c) unifies the scheduling of the dynamic fusion and diffusion refinement process, achieving progressive restoration from global illumination correction to local texture enhancement; through methods such as Figure 3 The time-step-aware dynamic fusion weights adaptively integrate illumination and texture features, effectively avoiding the oversmoothing and detail loss problems common in single-step restoration frameworks. A lightweight conditional diffusion refinement network iteratively refines the coarse shadow removal results, further eliminating residual artifacts and restoring high-frequency details. This method significantly improves the quality and visual consistency of shadow removal in UAV remote sensing images with only a small amount of paired training data, while reducing data acquisition and annotation costs.

[0053] The remote sensing shadow removal method based on dynamic feature fusion and diffusion optimization in this invention provides structural and texture priors for the dual-branch encoder through self-supervised image inpainting pre-training, enabling the model to achieve good restoration capabilities with a small amount of paired data. A shared time-step mechanism unifies the scheduling of the dynamic fusion and diffusion refinement processes, achieving progressive restoration from global illumination correction to local texture enhancement. Time-step-aware dynamic fusion weights adaptively integrate illumination and texture features, effectively avoiding the oversmoothing and detail loss problems common in single-step restoration frameworks. A lightweight conditional diffusion refinement network iteratively refines the coarse shadow removal results, further eliminating residual artifacts and restoring high-frequency details. This method significantly improves the quality and visual consistency of shadow removal in UAV remote sensing images with only a small amount of paired training data, while reducing data acquisition and annotation costs.

[0054] Example 2 like Figure 5 As shown, this embodiment of the invention provides a remote sensing shadow removal system based on dynamic feature fusion and diffusion optimization, comprising: The data acquisition and pre-training module is used to acquire shadowless images for training, shadowed images for training and inference, and corresponding shadow masks. It then uses the shadowless images to initialize a dual-branch encoder network through self-supervised image inpainting pre-training to obtain pre-trained encoder parameters. Specifically, the data acquisition and pre-training module acquires shadowless UAV images, generates damaged images using random masks, reconstructs the original images via an encoder-decoder network, learns structural and texture priors through self-supervised image inpainting pre-training, and transfers the encoder parameters to a dual-branch feature encoder.

[0055] The feature extraction module is used to input the shadow image and the shadow mask into the background encoder and texture encoder initialized by the pre-trained encoder parameters, respectively, to extract global illumination features of non-shadow areas and high-frequency texture features of shadow areas. Specifically, the feature extraction module is used to input the UAV shadow image and the corresponding shadow mask into the background encoder and texture encoder, respectively, to extract global illumination features of non-shadow areas and high-frequency texture features of shadow areas.

[0056] A dynamic fusion module, guided by a shared time step, dynamically weights and fuses the global illumination features and the high-frequency texture features, generating a coarse deshading result after decoding. Specifically, guided by a shared time step, the dynamic fusion module maps the time step encoding to a bias term, adds it to the static weights of the convolution output, and activates it to obtain time step-aware dynamic fusion weights. These weighted weights are then used to fuse the illumination features and texture features, and a lightweight decoder generates a coarse deshading result.

[0057] The diffusion refinement module is used to input the coarse deshading result into a lightweight conditional diffusion refinement network, perform iterative denoising under the shared time step scheduling, and train the diffusion model using the shadowless image as the supervised target to obtain the trained diffusion model. Specifically, the diffusion refinement module is used to construct a conditional signal by combining the coarse deshading result with the original shadow image and the mask, input it into the lightweight conditional diffusion refinement network, perform iterative denoising under the shared time step scheduling, and generate a refined shadowless image.

[0058] The inference module is used to input the remote sensing image to be processed into the trained diffusion model and output the final shadow removal result. Specifically, the model inference module is used to input the UAV image to be processed into the trained fusion and diffusion network and output the final shadow removal result.

[0059] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A remote sensing shadow removal method based on dynamic feature fusion and diffusion optimization, characterized in that, Includes the following steps: S1: Obtain the shadowless image for training, the shadowed image for training and inference, and the corresponding shadow mask; use the shadowless image to repair the pre-trained initial dual-branch encoder network through self-supervised image repair to obtain the pre-trained encoder parameters; S2: Input the shadow image and the shadow mask into the background encoder and texture encoder initialized by the pre-trained encoder parameters, respectively, to extract the global illumination features of the non-shadow area and the high-frequency texture features of the shadow area; S3: Guided by the shared time step, the global illumination features and the high-frequency texture features are dynamically weighted and fused, and a coarse deshading result is generated after decoding; S4: Input the coarse deshading result into a lightweight conditional diffusion refinement network, perform iterative denoising under the shared time step scheduling, train the diffusion model with the shadowless image as the supervised target, and obtain the trained diffusion model. S5: Input the remote sensing image to be processed into the trained diffusion model and output the final shadow removal result.

2. The remote sensing shadow removal method based on dynamic feature fusion and diffusion optimization according to claim 1, characterized in that, In step S1, the shadowless image is used to initialize the pre-trained dual-branch encoder network through self-supervised image inpainting, specifically as follows: An irregular binary mask is randomly generated from the shadowless image to simulate the missing region. The shadowless image is then multiplied element-wise with the inverse code of the mask to obtain the damaged input image. The pre-trained network takes the damaged input image and the mask as input, reconstructs the original shadowless image based on the visible context region, and optimizes it using the absolute error between the reconstructed image and the shadowless image as the loss function. After pre-training, the encoder parameters are transferred to the dual-branch feature encoder in the subsequent shadow removal framework, and the decoder is discarded.

3. The remote sensing shadow removal method based on dynamic feature fusion and diffusion optimization according to claim 1, characterized in that, In step S2, the extraction of global illumination features of non-shadow areas and high-frequency texture features of shadow areas specifically involves: Two parallel encoders, a texture encoder and a background encoder, are used to extract complementary features; The texture encoder takes the stitching result of the shadow image and the shadow mask as input, preserves the high-frequency details inside the shadow area, and generates a texture feature map. The background encoder processes the image obtained by element-wise multiplication of the shadow image and the inverse code of the shadow mask, capturing global color and lighting information from the non-shadow areas to generate a background feature map.

4. The remote sensing shadow removal method based on dynamic feature fusion and diffusion optimization according to claim 3, characterized in that, In step S3, guided by the shared time step, the lighting features and texture features are dynamically weighted and fused, specifically as follows: A time step is introduced to represent the stage variable of the recovery stage. This time step is encoded by sinusoidal position embedding combined with a multilayer perceptron to obtain the stage representation. The stage representation is then mapped to a bias term. The texture feature map and the background feature map are concatenated along the channels and then input into the convolutional layer to obtain static fusion information. The static fusion information is added element by element to the bias term and then mapped by the Sigmoid activation function to generate two time-step-aware dynamic fusion weights. The texture feature map and the background feature map are multiplied element-wise with their respective dynamic fusion weights and then added together. The results are then fused through a convolutional layer to obtain the fused features at the current time step. A lightweight decoder consisting of eight residual blocks, two transposed convolutions, and one standard convolutional layer upsamples the fused features to generate a coarse, shadowless prediction result.

5. The remote sensing shadow removal method based on dynamic feature fusion and diffusion optimization according to claim 1, characterized in that, In step S4, the coarse deshading result is input into a lightweight conditional diffusion refinement network for iterative denoising, specifically as follows: The coarse deshading result is used as the structured initial state, and together with the shadow image and the shadow mask, a conditional signal is constructed and input into a lightweight conditional diffusion refinement network. Using a diffusion model framework that predicts clean images, the forward diffusion process obtains a noisy state by gradually adding Gaussian noise to the coarse prediction results. The diffusion thinning network takes noisy state, conditional signal and time step as input, and outputs a thinned shadowless image. During training, the parameters of the coarse prediction network are frozen, and only the diffusion refinement network is optimized. The loss function is a weighted combination of diffusion reconstruction loss and perception loss, where diffusion reconstruction loss is the mean square error between the refinement result and the real shadowless image, and perception loss is the perceptual similarity loss between the two.

6. The remote sensing shadow removal method based on dynamic feature fusion and diffusion optimization according to claim 5, characterized in that, In step S4, the shared time step scheduling specifically includes: Align the feature fusion stage and the diffusion refinement stage to the same monotonically decreasing time step scheduling sequence, ensuring that the time steps of the fusion stage are equal to the time steps of the diffusion refinement stage. Under the coordination of the shared time step, a larger time step causes the fusion process to be biased towards global illumination correction and the denoising process to be biased towards large-scale structure restoration, while a smaller time step causes the fusion process to be biased towards texture detail enhancement and the denoising process to be biased towards local fine-tuning, thereby achieving a progressive restoration from global illumination correction to local texture enhancement.

7. The remote sensing shadow removal method based on dynamic feature fusion and diffusion optimization according to claim 1, characterized in that, Step S5 is as follows: The remote sensing image to be processed is input into the trained coarse prediction network to obtain the coarse prediction result. The coarse prediction result is input into the trained diffusion model, and deterministic denoising diffusion implicit model sampling is performed. The coarse prediction result is used as a low-noise initialization state corresponding to the maximum sampling time step. The coarse prediction network is executed only once at the initial time step to generate a time-step-aware initialization result. Then, deterministic sampling is performed in descending order of time steps to gradually refine the prediction result. In each sampling step, the diffusion refinement network predicts the clean estimate at the current time step and calculates the refined sample for the next time step according to the deterministic denoising update rule, combining the noisy sample at the current time step, the clean estimate at the current time step, and the signal retention ratio of the previous and next time steps, and finally obtains the refined shadowless image.

8. The remote sensing shadow removal method based on dynamic feature fusion and diffusion optimization according to claim 1, characterized in that, The remote sensing images include UAV remote sensing images.

9. The remote sensing shadow removal method based on dynamic feature fusion and diffusion optimization according to claim 1, characterized in that, The self-supervised image inpainting pre-training is completed using only shadowless images and random masks, and only pairs of shadow-no-shadow training data are needed to complete subsequent training.

10. A remote sensing shadow removal system based on dynamic feature fusion and diffusion optimization, characterized in that, include: The data acquisition and pre-training module is used to acquire shadowless images for training, shadowed images for training and inference, and corresponding shadow masks; and to use the shadowless images to perform self-supervised image repair pre-training to initialize the dual-branch encoder network and obtain the pre-trained encoder parameters. The feature extraction module is used to input the shadow image and the shadow mask into the background encoder and texture encoder initialized by the pre-trained encoder parameters, respectively, to extract the global illumination features of the non-shadow area and the high-frequency texture features of the shadow area; The dynamic fusion module is used to dynamically weight and fuse the global illumination features and the high-frequency texture features under the guidance of a shared time step, and generate a coarse deshading result after decoding; The diffusion refinement module is used to input the coarse deshading result into a lightweight conditional diffusion refinement network, perform iterative denoising under the shared time step scheduling, train the diffusion model with the shadowless image as the supervised target, and obtain the trained diffusion model. The inference module is used to input the remote sensing image to be processed into the trained diffusion model and output the final shadow removal result.

Images