A method of multi-scale adaptive gating and dual attention shadow removal network

Abstract

The application provides a multi-scale adaptive gating and double attention shadow removal network method, and belongs to the technical field of computer vision and digital image processing. The method uses adaptive mask gating to continuously guide multi-scale shadow perception feature learning in the encoding stage, enhances the representation ability of the model to the shadow area and boundary structure, constructs a mask guided regional attention and windowed global attention two complementary branches at the bottleneck layer, and dynamically integrates local enhanced features and global illumination features through an adaptive feature fusion mechanism, so that the detail recovery and overall illumination consistency are considered. The application achieves excellent shadow removal performance on the ISTD+ and SRD two public data sets, fully verifies the effectiveness of the mask prior guidance and local-global collaborative modeling strategy, and provides an effective solution for single image shadow removal in complex natural scenes.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and digital image processing technology, and in particular to a method for multi-scale adaptive gating and dual attention shadow removal networks. Background Technology

[0002] Shadows are a common lighting phenomenon in natural scenes, causing localized brightness attenuation and color shifts in images, thus disrupting visual consistency. While shadows can provide important clues about light source direction, scene structure, and occlusion relationships, they are generally considered significant interfering factors in most computer vision tasks. For example, in autonomous driving scenarios, shadows may cause perception systems to misjudge road edges or obstacles; in medical imaging, shadow artifacts can affect the segmentation and detection accuracy of lesion areas; and in tasks such as object detection and tracking, localized lighting inconsistencies caused by shadows directly reduce the generalization performance and stability of models. Therefore, removing shadows from natural scenes and restoring image lighting consistency is crucial for improving the robustness of downstream vision tasks.

[0003] Traditional shadow removal methods are mostly based on lighting models and physical priors for reasoning, but these methods often rely on strict imaging assumptions and have limited effectiveness in complex natural scenes. With the development of deep learning, researchers have proposed a series of learning-based methods based on CNNs and Transformers, which can learn the differences between shadow and non-shadow regions from data. However, existing methods still face the following challenges: insufficient sensitivity of attention mechanisms to shadow boundaries: shadow boundary regions are often the most difficult to recover, and ordinary self-attention struggles to accurately perceive the local structure of the boundaries; imbalance between local and global modeling capabilities: for example, RASM (Regional Attention Shadow Removal Model) emphasizes capturing local region information, while ShadowFormer focuses more on global context, but lacks effective integration of the two; insufficient utilization of shadow mask information: although some methods use shadow masks, most only treat the shadow mask as an additional input, lacking multi-scale, dynamically adjusted feature guidance, resulting in the shadow mask failing to fully realize its potential to distinguish between shadow and non-shadow regions; these problems make it difficult for existing models to achieve ideal recovery results in complex, low-quality, or strongly projected shadow scenes. Summary of the Invention

[0004] This invention aims to at least solve the technical problems existing in the prior art, and in particular, it innovatively proposes a method for multi-scale adaptive gating and dual attention shadow removal network.

[0005] To achieve the above-mentioned objectives of this invention, this invention provides a method for multi-scale adaptive gating and dual-attention shadow removal networks, comprising the following steps:

[0006] S1, input the shadow image and shadow mask into the adaptive mask gated encoder, use the shadow mask to generate gated weight adaptive modulation coding features at each coding level, and extract multi-scale shadow perception features;

[0007] S2 inputs the input features into the dual attention bottleneck layer, and extracts local enhancement features and global illumination features in parallel through mask-guided region attention and windowed global attention. Deep semantic features are obtained through adaptive feature fusion.

[0008] S3, input the deep semantic features into the shadow-aware decoder, combine them with the multi-scale shadow-aware features of the skip connection fusion encoder, upsample step by step to restore the image spatial resolution, and reconstruct a shadow-free image.

[0009] In the preferred embodiment of the above technical solution, the adaptive mask gating encoder described in S1 constructs two representations of the shadow mask, generates spatial gating weights through lightweight convolution mapping, and uses the gating weights to adaptively modulate the current encoded features, continuously enhancing the feature response of the shadow region and boundary.

[0010] In the preferred embodiment of the above technical solution, the two representation forms include a standard binary shadow mask and a zero-centered shadow mask, which are used to accurately distinguish between shadowed and unshadowed regions and to amplify the response differences between shadowed and unshadowed regions, respectively.

[0011] In the preferred embodiment of the above technical solution, the mask-guided region attention in S2 sequentially extracts feature information through lightweight local convolution, recalibrates channel attention, optimizes local spatial response through spatial attention, and combines explicit shadow mask prior to amplify the response difference between shadow and non-shadow regions, outputting local enhanced features.

[0012] In the preferred embodiment of the above technical solution, the explicit shadow mask prior utilizes a zero-centered shadow mask to participate in spatial response modulation, enabling the model to enhance the response in the shadow region while avoiding damage to the features in the non-shadow region, thus achieving selective protection of local feature modulation by the explicit spatial prior.

[0013] In the preferred embodiment of the above technical solution, the windowed global attention described in S2 divides the projected features into non-overlapping windows, performs multi-head self-attention calculation within each window, enhances the nonlinear expressive ability through a feedforward network and residual connections, and outputs global illumination features.

[0014] In the preferred embodiment of the above technical solution, the adaptive feature fusion described in S2 concatenates local enhancement features and global illumination features along the channel dimension, generates a spatial adaptive weight map through lightweight convolutional mapping, and dynamically adjusts the fusion ratio of local enhancement features and global illumination features according to different spatial locations.

[0015] In the preferred embodiment of the above technical solution, the shadow perception decoder described in S3, during the step-by-step upsampling process, fuses the multi-scale shadow perception features of the encoder through skip connections, so as to fully combine the shallow texture information and the deep semantic information, and guide the texture restoration and illumination reconstruction of the shadow area.

[0016] In the preferred embodiment of the above technical solution, the method is implemented based on an encoder-bottleneck layer-decoder architecture, and an adaptive optimizer is used during training, combined with a cosine annealing learning rate strategy.

[0017] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0018] An adaptive mask-gated encoder (AMG) is proposed, which continuously enhances the structural representation of shadow regions and boundaries at different coding stages by dynamically modulating coding features through multi-scale mask priors. Two parallel module branches, Mask-Guided Regional Attention (MGRA) and Window-based Global Attention (WGA), are designed, and an adaptive feature fusion strategy is used to dynamically integrate local enhancement features and global illumination features to achieve more accurate shadow restoration. A complete multi-scale adaptive gating and dual-attention shadow removal network (MSAG-DANet) architecture is constructed and validated on the ISTD+ and SRD datasets. Experimental results show that the model can effectively remove shadows and restore natural illumination.

[0019] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0020] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0021] Figure 1 This is the overall framework diagram of the present invention;

[0022] Figure 2 This is a schematic diagram of the core module structure of the present invention;

[0023] Figure 3 This is an example image of the present invention without shadows on the ISTD+ dataset;

[0024] Figure 4 This is a comparison image of shadowless image instances of the present invention on the SRD dataset. Detailed Implementation

[0025] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0026] like Figure 1 As shown, this invention discloses a method for shadow removal using a multi-scale adaptive gating and dual attention network (MSAG-DANet). This method uses shadow images... and corresponding shadow mask As input, the overall architecture adopts an encoder-bottleneck-decoder structure, consisting of a multi-scale adaptive mask-gated encoder (AMG), a dual-attention bottleneck layer (MGRA+WGA), and a shadow-aware decoder. During the encoding stage, shadow masks guide multi-scale shadow-aware feature extraction. The bottleneck layer enhances deep semantic features through collaborative local region modeling and global illumination modeling. During the decoding stage, spatial resolution is gradually restored and shadow-free images are reconstructed. Unlike traditional U-Net, which primarily relies on convolutional layer-by-layer feature extraction, MSAG-DANet is designed around hierarchical feature learning driven by shadow priors. This allows shadow masks to continuously participate in the feature modeling process, thereby enhancing the model's ability to express the structure of shadow regions and changes in illumination. Figure 2 The specific structures of each core module of the present invention are further provided.

[0027] S1: Input the shadow image and shadow mask into the adaptive mask gated encoder. At each coding level, use the shadow mask to generate gated weight adaptive modulation coding features and extract multi-scale shadow perception features.

[0028] Shadow masks accurately describe the location distribution of shadow regions and have therefore been widely used in supervised shadow removal tasks. However, most existing methods treat shadow masks as network inputs or supervisory signals, utilizing their prior information only in the input stage or loss function, making it difficult to continuously guide the hierarchical feature extraction process. For example, ShadowMaskFormer uses shadow mask representations to improve the input features of the Transformer, but shadow mask information mainly participates in the Patch Embedding stage. As the network deepens, its guiding ability for modeling deep semantic features gradually weakens. To further leverage the role of shadow priors, this invention proposes the AMG module, which directly integrates shadow masks into the multi-scale shadow-aware feature learning process during the encoding stage. Unlike simply using shadow masks to enhance input representations, AMG uses a learnable gating mechanism to continuously modulate feature responses at different encoding layers, allowing shadow priors to permeate the entire hierarchical feature extraction process, thereby improving the network's ability to perceive shadow regions and their boundary structures.

[0029] During the encoding stage, while multiple downsamplings can expand the receptive field, they inevitably lose high-frequency texture and shadow boundary information. Existing methods typically only use shadow masks as network input or supervision information, which has limited guiding role for deep semantic feature learning. To fully utilize the spatial prior provided by shadow masks, this invention introduces AMG (Advanced Motion Gathering) during the encoding stage. AMG first performs synchronous downsampling of the shadow mask according to the current scale, and uses lightweight convolution to generate corresponding gating weights to adaptively modulate the encoded features. This allows the shadow region to obtain more sufficient feature responses at different scales, thereby alleviating the problem of shadow information gradually weakening during continuous downsampling and improving the encoder's sensitivity to shadow regions and their boundary structures.

[0030] To satisfy the two different requirements of shadow region localization and modulation feature, two representations of the shadow mask are first constructed. and Its definition is as follows:

[0031] ,

[0032] ;

[0033] in, This represents a shadow mask, with pixel values ​​ranging from [0, 255], where a pixel value of 0 represents a non-shadow area and a pixel value of 255 represents a shadow area. This represents a standard binary shadow mask, used to accurately distinguish between shadowed and unshadowed areas; The zero-centered shadow mask representation can further amplify the response difference between shadowed and unshadowed regions, providing a more stable shadow prior for subsequent local feature modulation.

[0034] During the encoding process, as image spatial resolution gradually decreases with continuous downsampling, while semantic features are continuously enhanced, high-frequency texture and shadow boundary information are easily weakened. Therefore, in each encoder stage, the shadow mask is first synchronously modulated to the corresponding scale, and lightweight convolutional mapping is used to generate gate weights.

[0035] ;

[0036] in, Represents the Sigmoid function; This represents the spatial gating weights generated by the shadow mask, and their numerical range is... ; Indicates the kernel size as Two-dimensional convolutional layers, i.e., pointwise convolution; Indicates the kernel size as A two-dimensional convolutional layer is then used. Subsequently, gated weights are used to adaptively modulate the current encoded features:

[0037] ;

[0038] in, This represents the encoded features after gating and weighted adaptive modulation. This represents the output features of the current coding layer. This represents element-wise multiplication. This represents a learnable scalar parameter, initially set to 0, which learns the importance of shadow priors at different scales during training.

[0039] Encoded features after gating weight adaptive modulation Multi-scale shadow perception features are constructed at each coding level, denoted as... The output of the Lth layer As input features of the bottleneck layer .

[0040] Unlike traditional attention modules that learn responses through features themselves, AMG utilizes explicit spatial priors provided by shadow masks to enhance the responses of encoded features in a targeted manner. This allows the network to pay more attention to features related to shadow regions while maintaining stable representations of features in non-shadow regions. Since this gating operation continues throughout the entire encoding stage, the shadow prior can continuously participate in multi-scale shadow perception feature learning, effectively mitigating the problem of gradual decay of shadow information in deep networks and providing a more reliable feature foundation for subsequent local region modeling and global illumination restoration.

[0041] S2 inputs the input features into the dual attention bottleneck layer, and extracts local enhancement features and global illumination features in parallel through mask-guided region attention and windowed global attention. Deep semantic features are obtained through adaptive feature fusion.

[0042] The bottleneck layer contains the deepest semantic features of the network and is a crucial stage for establishing the relationship between local enhancement features and global illumination features. Addressing the challenge that shadow recovery relies on both local texture details and requires maintaining overall illumination consistency, this invention designs a dual-branch enhancement structure consisting of Mask-Guided Region Attention (MGRA) and Windowed Global Attention (WGA). MGRA utilizes explicit region priors provided by shadow masks to recalibrate channel attention on local convolutional features, making the model focus more on structural details in and around shadow regions, thereby improving local recovery quality. WGA employs a windowed multi-head self-attention mechanism, modeling long-distance illumination dependencies between different regions while ensuring computational efficiency, providing a more stable global illumination reference for shadow region recovery. Subsequently, a spatially adaptive feature fusion module dynamically fuses local enhancement features and global illumination features, enabling the network to automatically balance the contributions of local enhancement features and global illumination features according to different spatial locations, obtaining a more complete deep semantic feature representation.

[0043] S2-1 uses masking to guide regional attention and extract local enhancement features.

[0044] While global contextual information is crucial for shadow restoration, shadow removal is essentially a task of restoring local details. Shadow regions and their boundaries often contain the most complex brightness variations and texture degradations. Relying solely on global illumination features can easily lead to problems such as blurred boundaries, missing textures, and inaccurate local color reproduction. Therefore, in the bottleneck layer, a Mask-Guided Region Attention (MGRA) module is designed. By fusing local convolution, channel attention recalibration, and shadow mask priors, it performs more refined local structural modeling of shadow regions and their neighborhoods. By introducing explicit shadow mask priors into the local feature optimization process, MGRA enables attention computation to fully incorporate the positional constraints of shadow regions, more effectively enhancing shadow restoration-related features.

[0045] For input features MGRA first employs a lightweight local convolutional structure for preliminary feature extraction. Continuous convolutions can fully preserve local feature information, while depthwise separable convolutions further reduce computational complexity and enhance the modeling ability of each channel for local spatial structures. Therefore, the local features are represented as:

[0046] ;

[0047] in, This represents the feature representation after local convolution, providing more stable local feature information for subsequent channel attention recalibration; This represents the input features, i.e., the original feature tensor that enters the MGRA module; This indicates a modified linear unit activation function; express Depthwise separable convolution.

[0048] Because different channels respond differently to shadow textures, color variations, and boundary information, MGRA further employs a channel attention mechanism to adaptively model the importance of each channel. Specifically, global average pooling and global max pooling are performed on local features respectively, and then a two-layer mapping function with shared parameters is used. Feature compression and restoration are completed, resulting in two sets of channel description vectors. Subsequently, the two feature vectors are summed, and channel weights are generated using the Sigmoid function.

[0049] ;

[0050] in, and These represent the channel descriptions obtained by max pooling and average pooling, respectively. This indicates a shared mapping function.

[0051] Subsequently, the channel weights are applied to the local features to obtain the channel-enhanced local feature representation:

[0052] ;

[0053] After completing channel recalibration, to further enhance the model's ability to perceive shadow regions and their boundary locations, MGRA introduces a spatial attention mechanism to optimize local spatial responses. This is achieved by... Perform average pooling and max pooling along the channel dimension respectively, and then concatenate the results along the channel dimension. Convolution generates spatial attention weights:

[0054] ;

[0055] in, This represents a spatial attention map, which can adaptively learn the important spatial locations of local regions; This indicates a feature concatenation operation along the channel dimension; This represents average pooling along the channel dimension, generating a single-channel average feature description. This represents max pooling along the channel dimension, generating the maximum feature description for a single channel.

[0056] MGRA further utilizes zero-centered shadow masks As an explicit shadow prior, the spatial response is modulated again:

[0057] ;

[0058] in, This represents the spatial enhancement feature after joint modulation by spatial attention weights and explicit shadow priors; It is not used as additional input in feature extraction, but rather as spatial prior in the local feature modulation process. Because... Using zero-center representation, the shaded area corresponds to ,at this time Spatial augmentation features This is further magnified, causing the model to focus more on important structures in the shaded areas and near the boundaries; while the non-shaded areas correspond to... ,at this time , The output in this region is a zero vector. After layer normalization and linear mapping, this zero vector still retains an approximately zero feature response, and is then connected via residuals. Afterwards, the output of the non-shaded region is approximately equal to the original input features. This allows for the enhancement of the response in the shadowed region while avoiding damage to the features in the non-shadowed region, thus achieving selective protection of local feature modulation by explicit spatial priors.

[0059] To further stabilize the feature distribution and enhance the stability of module training, MGRA finally employs layer normalization and linear mapping to further optimize the spatially augmented features, and obtains the final output through residual connections:

[0060] ;

[0061] in, This represents the local enhancement features ultimately output by the MGRA module; Represents linear mapping operations; Representation layer normalization operation. Residual connections not only alleviate the gradient decay problem during deep network training, but also continuously enhance the local structural representation of shadow regions while preserving the original semantic information, providing more reliable regional features for subsequent global illumination modeling and local-global feature fusion.

[0062] S2-2 extracts global illumination features through windowed global attention.

[0063] Although shadow removal is a local restoration task, global illumination consistency remains crucial for shadow reconstruction. For the same object, different regions typically have consistent reflectivity, but shadows disrupt this global illumination relationship. Therefore, relying solely on local structure modeling can easily lead to inconsistent brightness recovery between different regions, resulting in problems such as local overexposure, color shifts, or overall lighting inconsistencies. Therefore, in addition to local structure modeling, it is also necessary to establish long-distance illumination dependencies between shadowed and unshadowed regions.

[0064] This invention designs a windowed global attention (WGA) branch parallel to MGRA at the bottleneck layer. This module takes the bottleneck layer as input features. As input, a global context representation is established through windowed multi-head self-attention, which effectively enhances the network's ability to model global illumination features while maintaining low computational complexity.

[0065] First, the input features are processed... After convolution, we get Then, the projected features Divided into sizes Non-overlapping windows, and flatten them into a window sequence. ,in It is the embedded dimension. The number of non-overlapping windows; It refers to the batch size; It is the feature map height; It is the feature map width; This represents the side length of the non-overlapping window. To stabilize the training process, the window sequence is first normalized layer by layer:

[0066] ;

[0067] in, This represents the window sequence after the projected features have been divided and flattened. This represents the window sequence after layer normalization. Subsequently, multi-head self-attention computation is performed within each window. For the ... Each attention head has an output defined as:

[0068] ;

[0069] in, , and They represent the first The matrix of Query, Key, and Value corresponding to each attention point. The feature dimension of a single attention head; This represents the Softmax activation function; It is the transpose symbol.

[0070] The outputs of all attention heads are concatenated and then mapped back to the unified feature space through the output projection matrix:

[0071] ;

[0072] in, Indicates the number of attention heads. Indicates the output projection matrix. This indicates that the focus is on the bulls.

[0073] To further enhance training stability, this invention introduces a learnable scalar parameter to adaptively adjust the contribution of the windowed global attention branch to the final feature:

[0074] ;

[0075] in, These are learnable scalar parameters that are automatically optimized during training to balance information fusion between the residual branch and the windowed global attention branch. This represents the window sequence after multi-head self-attention residual connection.

[0076] Subsequently, a feedforward network is used to further enhance the nonlinear representation capability, and a second residual connection is employed to obtain the final feature representation:

[0077] ;

[0078] in, This represents the window sequence after passing through the feedforward network and the second residual connection; This represents a feedforward network.

[0079] Finally, restore the window sequence to its original spatial layout by... Convolution maps the embedding dimension back to the original number of channels and performs residual fusion with the input features to obtain the global illumination features of the WGA output:

[0080] ;

[0081] S2-3, deep semantic features are obtained through adaptive feature fusion.

[0082] MGRA and WGA extract complementary features from the perspectives of local structure modeling and global illumination modeling, respectively. Since different image regions have varying degrees of dependence on local enhancement features and global illumination features, using fixed weights for fusion makes it difficult to simultaneously achieve shadow boundary detail restoration and overall illumination consistency. Therefore, this invention designs an adaptive feature fusion module that dynamically generates spatially adaptive weights based on the two branch features, achieving collaborative fusion of local and global information.

[0083] Output the branches of MGRA and WGA and By splicing along the channel dimension, the features are obtained. Subsequently, a spatially adaptive weight graph is generated through lightweight convolutional mapping:

[0084] ;

[0085] in, Represents the convolution mapping function, spatially adaptive weight graph Used to represent the degree of dependence of different spatial locations on local enhancement features and global illumination features.

[0086] Finally, the two feature streams are adaptively fused according to their spatial weights to obtain deep semantic features:

[0087] ;

[0088] This mechanism allows the network to autonomously decide during training whether to rely more on local features or global information, depending on the different image regions.

[0089] S3, the deep semantic features are input into the shadow-aware decoder, and combined with the multi-scale shadow-aware features of the skip connection fusion encoder, the image spatial resolution is restored by upsampling step by step, and a shadow-free image is reconstructed.

[0090] During the decoding stage, the network combines skip connections with multi-scale shadow-perceived features from the encoder and employs progressive upsampling to restore the image's spatial resolution, ensuring a full integration of shallow texture information and deep semantic information. Simultaneously, the deep semantic features, enhanced by dual attention, are progressively passed to each level of the decoder to continuously guide texture restoration and illumination reconstruction in shadow areas. Finally, the output reconstruction module generates the final shadow-free image, achieving a more natural, continuous, and consistent brightness and color transition between shadow and non-shadow areas.

[0091] Considering the characteristics of image shadow removal tasks, in order to ensure that the shadow-free image is as close as possible to the real shadow-free image at the pixel and semantic feature levels, a composite loss function including content loss and perceptual loss is adopted. The following is an introduction to it.

[0092] Content loss is a metric used in deep learning to measure the content difference between a shadowless image and the original image, and is commonly used in image processing tasks. This invention employs the L1 loss function to minimize the absolute difference between the shadowless image and the true shadowless image:

[0093] ;

[0094] in, The image output by the model without shadows. For a true, shadow-free image, N is the total number of pixels.

[0095] Perceptual loss is a loss function based on deep learning. Its core idea is not to directly compare the differences between the original data, but rather to extract high-level semantic features through a pre-trained neural network and then measure the differences between these features. This example uses features extracted from layers {conv1_2,conv2_2,conv3_2,conv4_2,conv5_2} of a pre-trained VGG-19 model and calculates the L1 difference between features from each layer.

[0096] ;

[0097] in, Represents the L1 norm; Let j represent the j-th feature extraction layer in the VGG-19 network, with weight coefficients. Weight vector The j-th component corresponds to a weight value of the layer. The values ​​are determined based on the differences in feature representation capabilities of different convolutional layers in the VGG-19 network: low-level features {conv1_2, conv2_2} focus on detailed textures and have relatively limited constraints on the overall structural consistency in the shadow removal task, corresponding to the weights. , The value is 0.1; the high-level features {conv3_2, conv4_2, conv5_2} focus on semantic structure and are more critical for maintaining global consistency of the image, corresponding to the weights. , , A value of 1 makes the contribution of high-level features to the perceptual loss ten times that of low-level features. Multiple comparative experiments have verified that this set of weights achieves a balance between detail fidelity and semantic consistency.

[0098] Finally, the composite loss function is a weighted sum of content loss and perceptual loss:

[0099] ;

[0100] Among them, the weighting coefficient Set to 0.05 to balance content loss. and perceived loss The contribution. The weighting coefficient By conducting multiple sets of comparative experiments within a preset interval, it was determined that when At this time, the convergence speed of content loss and perceptual loss and the final reconstruction quality reach the optimal balance, which can effectively restore image texture details while ensuring pixel-level reconstruction accuracy.

[0101] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0102] 1. Parameter settings

[0103] The model proposed in this invention is implemented in PyTorch. During training, an adaptive moment estimator with weight decay is used, where the momentum is (0.9, 0.999) and the weight decay is set to 0.02. The initial learning rate is set to 3 × 10⁻⁶. -4 The learning rate was gradually reduced to 1×10 using a cosine annealing strategy. -6 All experiments were conducted on an NVIDIA GeForce RTX 3090 GPU.

[0104] Datasets: The model's performance was evaluated on two benchmark datasets:

[0105] (1) ISTD+: contains 1870 sets of images. Each set of images contains a shadowed image, a shadowless image and a corresponding shadow mask. 1330 sets are used for training and 540 sets are used for testing. It is widely used for supervised shadow detection and removal tasks.

[0106] (2) SRD: The dataset consists of 2680 pairs of training images and 408 pairs of images without shadows. It does not contain ground truth shadow masks and uses shadow masks provided by DHAN (Dual Hierarchical Aggregation Network) for training and testing.

[0107] Evaluation metrics: To better evaluate model performance, the root mean square error (RMSE) between the shadowless image and the real shadowless image in the LAB color space is used as a quantitative evaluation metric for shadow removal effect. The lower the value, the better the effect. In addition, peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) are also used to compare the performance of various methods in the RGB color space. The higher the value, the better the effect.

[0108] 2. Performance Evaluation

[0109] 2.1 Quantitative comparison

[0110] To comprehensively evaluate the shadow removal performance, quantitative comparative experiments were conducted on two widely used datasets, ISTD+ and SRD, against several state-of-the-art methods. The evaluation was carried out from three perspectives: shadow area, non-shadow area, and overall image, using PSNR, SSIM, and RMSE metrics.

[0111] Table 1 presents the evaluation results of this invention on the ISTD+ dataset. This invention achieves the highest PSNR and SSIM scores among the comparative methods in both the overall image (ALL) and non-shaded regions (NS), while maintaining low error. It also achieves a high level in the shaded region, second only to RASM, but significantly outperforming other models. Overall, this invention demonstrates significant advantages in preserving the integrity of non-shaded regions and overall image restoration on the ISTD+ dataset, reflecting its comprehensive performance in the shadow removal task.

[0112] Table 1. Quantitative comparison of the removal performance of the proposed model with the nearest shadow removal model on the ISTD+ dataset.

[0113]

[0114] Table 1 shows that high PSNR and SSIM were achieved in both shadow and non-shadow regions, indicating that it can effectively recover shadow region details and maintain brightness consistency. In the table, "-" indicates unavailable results; DHAN represents a two-level aggregation network; G2R represents a generator-removal framework; DC-ShadowNet represents domain classifier-guided shadow removal; AutoExposure represents automatic exposure fusion; BMNet represents a bijective mapping network; ShadowFormer represents global context shadow removal; ShadowDiffusion represents diffusion model shadow removal; Li et al. represents repair-based shadow removal; Liu et al. represents region illumination reconstruction; and RASM represents region attention shadow removal.

[0115] As shown in Table 2, on the more complex SRD dataset, the PSNR of this invention is improved by 0.35 dB and 0.73 dB in the overall image and non-shaded areas, respectively, compared to RASM. This indicates that the invention still has strong generalization ability and overall image restoration capability in different distributions and more complex scenes. Because a smoother strategy is adopted in the processing of textures in the shadow area, some compromises may be made in some indicators when reducing the overall RMSE, resulting in the invention not achieving optimal performance in the shadow area. Considering the overall performance, this trade-off is reasonable and worthwhile, ensuring that the final output shadowless image is harmonious and natural overall.

[0116] Table 2. Quantitative comparison of the removal performance of the proposed model with the nearest shadow removal model on the SRD dataset.

[0117]

[0118] As shown in Table 2, the model of this invention performs best in non-shadow regions, demonstrating better cross-scene generalization ability. Here, DSC represents orientation-aware spatial context, DHAN represents a two-level aggregation network, AutoExposure represents automatic exposure fusion, DC-ShadowNet represents domain classifier-guided shadow removal, BMNet represents a bijective mapping network, ShadowFormer represents global context shadow removal, Liu et al. represents region illumination reconstruction, DeS3 represents an adaptive attention-driven shadow removal model, and RASM represents region attention shadow removal.

[0119] 2.2 Qualitative Comparison

[0120] To further evaluate the visual effect of shadow removal by the method of the present invention, Figure 3 Qualitative comparison results on the ISTD+ dataset are presented. It can be seen that, compared to ShadowDiffusion and ShadowMaskFormer, the present invention achieves more accurate color reproduction in low-brightness areas; compared to RASM and ShadowFormer, the transition at shadow boundaries is smooth with no obvious artifacts, and the brightness of non-shadow areas remains consistent.

[0121] Figure 4 This paper demonstrates shadow removal performance on the more challenging SRD dataset, which features irregular shadow distribution and complex scenes, demanding higher generalization capabilities from the model. Compared to ShadowMaskFormer, DeS3 (Adaptive Attention-Driven Shadow Removal Model), and ShadowDiffusion, the model in this invention exhibits significant advantages in color consistency and structure recovery in large shadow areas, effectively reducing over-brightening or detail loss. This further validates the robustness and generalization performance of this invention in complex real-world scenes.

[0122] 3. Ablation test

[0123] To verify the effectiveness of each component of MSAG-DANet, ablation experiments were conducted on the ISTD+ dataset. The experimental results are shown in Table 3. The experiments removed the Adaptive Mask Gated Encoder (AMG), Mask Guided Region Attention (MGRA), Windowed Global Attention (WGA), and both attention modules simultaneously to analyze the contribution of each module to shadow removal performance.

[0124] As shown in Table 3, AMG can effectively enhance the shadow perception feature representation during the encoding stage. After removing AMG, the PSNR of the shadow region decreased from 40.52dB to 40.42dB, the RMSE increased from 4.87 to 5.02, and the overall image PSNR also decreased to 36.22dB. This indicates that multi-scale mask gating can continuously enhance the shadow region features and alleviate the loss of detail information during downsampling.

[0125] The dual attention modules are responsible for local structure modeling and global illumination modeling, respectively. The most significant performance degradation occurred after removing the WGA, with the overall image PSNR decreasing from 36.41 dB to 36.00 dB and the RMSE in shadow areas increasing from 4.87 to 5.73, indicating that global illumination modeling is crucial for maintaining consistency between shadow and non-shadow areas. In contrast, removing the MGRA module reduced the PSNR in non-shadow areas from 39.72 dB to 39.66 dB, while the RMSE in shadow areas increased from 4.87 to 4.97, demonstrating that the MGRA module effectively optimizes shadow boundaries and local texture recovery using shadow mask priors. When both the MGRA and WGA modules were removed simultaneously, the overall image PSNR further decreased to 35.97 dB, validating the good complementarity between local structure modeling and global illumination modeling.

[0126] In summary, AMG, MGRA, and WGA improve model performance in three aspects: multi-scale shadow feature enhancement, local structure restoration, and global illumination consistency modeling, respectively. Their synergistic effect enables MSAG-DANet to achieve optimal results in shadow regions, unshaded regions, and the overall image, validating the effectiveness of the proposed network design.

[0127] Table 3. Quantitative comparison of ablation studies of the proposed components on the ISTD+ dataset.

[0128]

[0129] The ablation study evaluated the complete MSAG-DANet and its variants by removing the adaptive mask-gated encoder (w / o AMG), mask-guided region attention (w / o MGRA), windowed global attention (w / o WGA), and simultaneously removing both attention modules (w / o MGRA+WGA) to analyze the contribution of each component.

[0130] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for shadow removal using a multi-scale adaptive gating and dual-attention network, characterized in that, include: S1, input the shadow image and shadow mask into the adaptive mask gate encoder. The adaptive mask gate encoder constructs two representations of the shadow mask at each coding level, generates spatial gate weights through lightweight convolution mapping, and uses the gate weights to adaptively modulate the current coding features to continuously enhance the feature response of the shadow area and boundary, and extract multi-scale shadow perception features. S2, the output features of the deepest encoding layer in the multi-scale shadow perception features are used as input features and input to the dual attention bottleneck layer; in the dual attention bottleneck layer, the mask-guided region attention sequentially extracts feature information through lightweight local convolution, recalibrates channel attention, optimizes local spatial response through spatial attention, and expands the response difference between shadow and non-shadow regions by combining explicit shadow mask prior, outputting local enhanced features; windowed global attention divides the projected features into non-overlapping windows, performs multi-head self-attention calculation within each window, enhances nonlinear expressive ability through feedforward network and residual connection, and outputs global illumination features; The local enhancement features and the global illumination features are fused through adaptive feature fusion to obtain deep semantic features; S3, input the deep semantic features into the shadow-aware decoder, combine them with the multi-scale shadow-aware features of the skip connection fusion encoder, upsample step by step to restore the image spatial resolution, and reconstruct a shadow-free image.

2. The method according to claim 1, characterized in that, The two representations described in S1 include the standard binary shadow mask and the zero-centered shadow mask, which are used to accurately distinguish between shadowed and unshadowed regions and to amplify the response differences between shadowed and unshadowed regions, respectively.

3. The method according to claim 1, characterized in that, The explicit shadow mask prior described in S2 utilizes zero-centered shadow masks to participate in spatial response modulation, enabling the model to enhance the response in shadowed regions while avoiding damage to features in non-shadowed regions, thus achieving selective protection of local feature modulation by the explicit spatial prior.

4. The method according to claim 1, characterized in that, The adaptive feature fusion described in S2 concatenates local enhancement features and global illumination features along the channel dimension, generates a spatial adaptive weight map through lightweight convolutional mapping, and dynamically adjusts the fusion ratio of local enhancement features and global illumination features according to different spatial locations.

5. The method according to claim 1, characterized in that, The shadow-aware decoder described in S3, during the step-by-step upsampling process, fuses multi-scale shadow-aware features from the encoder through skip connections, thereby fully combining shallow texture information with deep semantic information to guide texture restoration and illumination reconstruction in shadow areas.

6. The method according to claim 1, characterized in that, The method is based on an encoder-bottleneck layer-decoder architecture. During training, an adaptive moment estimation optimizer with weight decay is used in combination with a cosine annealing learning rate strategy.

Images