Mask-free image shadow removal method based on illumination guided dual-flow mamba

Abstract

This invention relates to the fields of image processing and computer vision technology, specifically disclosing a maskless image shadow removal method based on illumination-guided dual-stream Mamba. The method includes: passing the shadow image through a feature encoder, and then, based on Retinex theory, obtaining illumination flow features and reflectance flow features through a feature decoupling mapping operator; passing the illumination flow features through an illumination prior Mamba module to obtain illumination prior features; inputting the reflectance flow features and illumination prior features into an illumination-guided reflectance Mamba module to obtain a reflectance feature representation under illumination prior constraints; and inputting the illumination prior features and reflectance feature representation into a decoder to obtain a shadow-free image. This invention, by explicitly introducing illumination physical priors and reconstructing the feature sequence organization, achieves differentiated modeling of shadow and non-shadow regions without relying on shadow masks, significantly improving the structural consistency and color fidelity of the shadow removal results.

Description

Technical Field

[0001] This invention relates to the fields of image processing and computer vision technology, and in particular to a maskless image shadow removal method based on illumination-guided dual-stream Mamba. Background Technology

[0002] Existing shadow removal methods can be broadly categorized into physically based methods and data-driven deep learning methods. Traditional physically based methods typically rely on illumination consistency assumptions or Retinex theory, achieving shadow correction by analyzing image brightness or gradient information. However, these methods often require strict imaging condition assumptions, making them difficult to adapt to the complex and varied illumination distributions in real-world scenes. They are prone to over-enhancement or loss of detail in the presence of soft shadows, complex occlusions, or color changes. In recent years, with the development of deep learning technology, shadow removal methods based on convolutional neural networks or Transformer structures have made significant progress. One type of method introduces pixel-level shadow masks as prior information to explicitly distinguish between shadow and non-shadow areas, thereby guiding the network to perform targeted repair. These methods typically achieve good repair results on controlled datasets. However, in practical applications, obtaining high-quality shadow masks is costly, and existing shadow detection algorithms struggle to generate masks with accurate boundaries in complex lighting, soft shadows, or dynamic scenes. If there is an error in the mask, the error information will be further amplified during the subsequent repair process, which can easily produce halo artifacts at the shadow boundary or introduce color distortion in non-shadow areas, thus seriously affecting visual quality and algorithm stability.

[0003] To break free from the reliance on explicit shadow masks, researchers have proposed maskless shadow removal methods that learn shadow removal mappings directly from the input image via an end-to-end network. However, in the absence of strongly supervised priors, these methods generally face color ambiguity problems, namely, difficulty in distinguishing inherent dark regions from true shadow regions based solely on pixel intensity information. Existing maskless methods often employ single-stream convolutional networks or Transformer structures for implicit feature modeling, lacking explicit constraints on physical properties such as illumination and reflectivity. This can easily lead to indiscriminate brightness enhancement of dark regions, thereby compromising the original semantic consistency and color authenticity of the image. Furthermore, shadow regions typically exhibit discrete and irregular spatial distributions. Traditional convolutional operations are limited by local receptive fields, making it difficult to capture long-term dependencies in global illumination distribution; while existing sequence-based visual networks (such as state-space models with fixed scan paths) often rely on predefined spatial scan orders, mapping two-dimensional features to a one-dimensional sequence. This fixed-space scanning strategy is difficult to effectively model spatially discontinuous but physically similar shadow regions under maskless conditions. It is easy to sever long-range correlations between shadow regions, thus limiting the model's ability to model global illumination consistency.

[0004] In summary, existing technologies still have the following shortcomings in maskless shadow removal scenarios: 1. Over-reliance on explicit shadow masks limits practical applications; 2. Lack of explicit modeling of the physical properties of illumination and reflectivity easily leads to color ambiguity; 3. Fixed spatial scanning or local modeling methods are difficult to effectively characterize the global illumination relationship of discrete shadow regions. Summary of the Invention

[0005] This invention aims to solve the aforementioned problems. To this end, this invention provides a maskless image shadow removal method based on illumination-guided dual-stream Mamba. By explicitly introducing illumination physical priors and reconstructing the feature sequence organization, it achieves differentiated modeling of shadow and non-shadow regions without relying on shadow masks, significantly improving the structural consistency and color fidelity of the shadow removal results.

[0006] This invention provides a maskless image shadow removal method based on illumination-guided dual-stream Mamba, the technical solution of which includes: S1: Obtain the shadow image; S2: The shadow image is processed by a feature encoder to obtain a high-dimensional latent feature representation, and then, based on Retinex theory, a feature decoupling mapping operator is used to obtain illumination flow features and reflectivity flow features. S3: The illumination flow features are obtained through the illumination prior Mamba module to obtain illumination prior features; S4: Input the reflectivity flow features and illumination prior features into the illumination-guided reflectivity Mamba module to obtain the reflectivity feature representation under illumination prior constraints; S5: Input the prior illumination features and reflectivity features into the decoder to obtain a shadowless image.

[0007] Furthermore, the pixels in the illumination flow features are quantized into multiple illumination levels to obtain illumination level indexes. These illumination level indexes are used to guide the sequence construction of the illumination prior Mamba module and the illumination-guided reflectivity Mamba module. During the quantization process, pixels with similar illumination intensities are grouped into the same illumination level.

[0008] Furthermore, the sorting process for constructing the sequences of the illumination prior Mamba module and the illumination-guided reflectivity Mamba module both adopts a hierarchical constraint strategy based on lexicographical order, using the illumination level index as the primary sorting key and the scanning order of pixels in the original two-dimensional space as the secondary sorting key.

[0009] Further, lighting level index for: in, This indicates a round-down operation. To control the hyperparameters of stratification granularity, The spatial location of a single-channel illumination intensity feature map pixels, This represents the minimum value of the illumination component of a pixel in a single-channel illumination intensity feature map. This represents the maximum value of the illumination component of a pixel in a single-channel illumination intensity feature map.

[0010] Furthermore, in step S4, the prior illumination features are passed through the illumination histogram guidance module and then input into the illumination-guided reflectance Mamba module. The illumination histogram guidance module is used to perform statistical modeling of illumination prior features based on illumination hierarchy index, and to map the illumination response in the spatial domain to the statistical domain.

[0011] Furthermore, the calculation formula for the illumination histogram guidance module is as follows: in, Lighting levels The corresponding histogram eigenvalues, Lighting levels The set of pixels, for The number of elements, Let be the feature value of the prior illumination features of the l-th layer at spatial location (x, y). The pixel at spatial location (x, y) corresponding to illumination level k.

[0012] Furthermore, the working process of the illumination prior Mamba module includes: The illumination flow features are sequentially passed through the first linear layer, the first DW-Conv module, and SiLU activation to obtain the first intermediate illumination features; The second illumination intermediate feature is obtained by transforming the first illumination intermediate feature through the first SSM and dimension transformation; The illumination flow features are sequentially passed through the second linear layer and activated by SiLU to obtain the third intermediate illumination feature; Multiplying the intermediate features of the second illumination by the intermediate features of the third illumination yields the prior illumination features; The operation of the light-guided reflectivity Mamba module includes: The reflectivity flow feature is passed sequentially through the third linear layer, the second DW-Conv module, and SiLU activation to obtain the first intermediate reflectivity feature; After multiplying the first intermediate reflectance feature with the prior illumination feature, and then performing a second SSM and dimensional transformation, the second intermediate reflectance feature is obtained. The reflectivity flow characteristics are sequentially passed through the fourth linear layer and SiLU activation to obtain the third intermediate reflectivity characteristics; Multiplying the second intermediate reflectance feature with the third intermediate reflectance feature yields the reflectance feature representation.

[0013] This invention also provides a maskless image shadow removal system based on illumination-guided dual-stream Mamba, the technical solution of which includes: The image acquisition module is used to acquire shadow images; The encoding module is used to obtain a high-dimensional latent feature representation of the shadow image through a feature encoder, and then obtain illumination flow features and reflectivity flow features based on Retinex theory and through a feature decoupling mapping operator. The illumination stream processing module is used to obtain illumination prior features by passing the illumination stream features through the illumination prior Mamba module; The reflectivity stream processing module is used to input the reflectivity stream features and illumination prior features into the illumination-guided reflectivity Mamba module to obtain a reflectivity feature representation under illumination prior constraints. The decoding module is used to input the illumination prior features and reflectivity features into the decoder to obtain a shadowless image.

[0014] The present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described method for maskless image shadow removal based on illumination-guided dual-stream Mamba.

[0015] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-described method for maskless image shadow removal based on illumination-guided dual-stream Mamba.

[0016] The above-described one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects: 1. This invention is based on the two-stream modeling paradigm of Retinex theory, which explicitly decouples illumination features and reflectivity features at the network structure level. It models the scene brightness distribution through illumination flow and guides the restoration process of reflectivity flow with illumination priors, thereby effectively alleviating the color ambiguity problem under maskless conditions.

[0017] 2. To address the limitation of traditional fixed-scan strategies in capturing global illumination consistency due to the spatially discrete distribution of shadow regions, this invention proposes an Illumination Hierarchical Ranking (IHS) mechanism. This mechanism uses illumination intensity as an implicit attention signal to dynamically reorder spatial features. At the macroscopic level, pixels are hierarchically clustered based on illumination intensity; at the microscopic level, the spatial scanning order within each level is maintained. This ensures local texture continuity while allowing spatially discontinuous but physically similar shadow regions to be aggregated and modeled in the sequence domain. Through this sequence recombination method, the Mamba state-space model can stably perceive the same illumination modality across consecutive time steps, significantly enhancing its ability to model long-range dependencies in shadow regions.

[0018] 3. In the dual-stream Mamba collaborative modeling stage, the illumination prior Mamba module is used to learn the global illumination distribution pattern and generate stable illumination priors. Meanwhile, the illumination-guided reflectivity Mamba module, while sharing the same sequence organization, introduces illumination guidance information to conditionally modulate the state-space model parameters, thereby achieving targeted compensation for shadow-affected features. Simultaneously, by constructing an illumination statistical guidance mechanism, illumination features in the spatial domain are mapped to the statistical domain for global interactive modeling. The refined illumination priors are then fed back layer by layer to the reflectivity feature evolution process, enabling illumination information to continuously and stably exert a constraining effect throughout the network, further improving the naturalness of shadow boundaries and the consistency of the overall structure.

[0019] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0021] Figure 1 This is a flowchart of the method provided by the present invention.

[0022] Figure 2 This is an overall architecture diagram of the IGDM framework provided by the present invention.

[0023] Figure 3 The image shows the effect of shadow removal on the ISTD+ dataset provided by this invention.

[0024] Figure 4 The image shows the effect of shadow removal on the SRD dataset provided by this invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention. The following embodiments are used to illustrate this invention but should not be used to limit the scope of this invention.

[0026] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0027] The following is combined Figures 1 to 4 The present invention will be further described in detail below, which describes a maskless image shadow removal method based on illumination-guided dual-stream Mamba: Example 1 In this embodiment, as Figure 1 As shown, a maskless image shadow removal method based on illumination-guided dual-stream Mamba is provided, including the following steps: S1: Get the shadow image.

[0028] In practical implementation, the network is sized as Shadow image As the sole input, no additional shadow mask or prior annotation information is required. For width, For height.

[0029] S2: The shadow image is processed by a feature encoder to obtain a high-dimensional latent feature representation. Then, based on Retinex theory, the illumination flow feature and reflectivity flow feature are obtained through a feature decoupling mapping operator.

[0030] After normalization, the shadow image is fed into the feature encoder, which is used to extract high-dimensional latent feature representations from the pixel space, providing a unified feature basis for subsequent physical property modeling.

[0031] From the perspective of imaging physics, according to classical Retinex theory, an observed image affected by shadows can be modeled as an element-wise product of the scene reflectivity component and the illumination component, that is: in, It represents the Hadamah accumulation. It represents the inherent reflectivity information of an object's surface, mainly including texture and color attributes, and ideally should be independent of changes in lighting. Representing the spatial distribution of ambient light in a scene, shadow regions typically represent the illumination component in an image, exhibiting significant attenuation in localized areas. Therefore, the maskless shadow removal task can be formalized as: given only the shadow image and lacking prior shadow masking, inversely estimating the reflectance image unaffected by illumination. .

[0032] To avoid color ambiguity caused by traditional single-stream end-to-end networks confusing inherent dark colors with true shadow regions under no physical constraints, this method introduces a two-stream modeling paradigm based on explicit decoupling of physical properties at the network structure level. Specifically, the latent features output by the encoder are explicitly divided into two feature streams with different physical meanings in the high-dimensional feature space: in, Represents the feature encoder. Indicates the feature decoupling mapping operator; These are illumination flow features used to characterize the brightness distribution and shadow structure information of a scene. Reflectivity flow features are used to characterize the inherent texture and color attributes of an object.

[0033] S3: The illumination flow features are obtained by using the Illumination Prior Mamba (IP-Mamba) module to obtain illumination prior features.

[0034] In the dual-stream modeling process (steps S3 and S4), the two feature streams are modeled using Mamba modules based on state-space models, but they have clear differences in functional division and information flow direction. Specifically, the Mamba module (lighting prior Mamba module) in the illumination stream is used for long-range modeling of the global illumination distribution, learning the illumination consistency relationship between different spatial regions, and generating stable lighting prior features. The calculation formula is as follows: in, Represents prior features of illumination. This represents the lighting prior Mamba module. This lighting prior feature reflects the relative light intensity and shadow structure characteristics of different areas in the scene, and can be regarded as an implicit shadow cue.

[0035] The working process of the illumination prior Mamba module includes: The illumination flow features are sequentially passed through the first linear layer, the first DW-Conv module (depth convolution module), and SiLU activation to obtain the first intermediate illumination features; The first intermediate illumination feature is transformed into the second intermediate illumination feature through the first SSM (Selective State-Space Model) and dimensional transformation. The illumination flow features are sequentially passed through the second linear layer and activated by SiLU to obtain the third intermediate illumination feature; Multiplying the intermediate features of the second illumination by the intermediate features of the third illumination yields the prior illumination features.

[0036] S4: Input the reflectance flow features and illumination prior features into the illumination-guided reflectance Mamba module (Illumination-Guided Reflectance Mamba, IGR-Mamba) to obtain the reflectance feature representation under illumination prior constraints.

[0037] Based on step S3, the reflectivity flow does not evolve independently. Instead, prior constraints from the illumination flow are explicitly introduced during the Mamba state-space modeling process to modulate its state update process. Its modeling form can be expressed as: in, The Mamba module represents the reflectivity of light-guided illumination. This represents reflectance features under prior lighting constraints. By adopting a collaborative strategy of first modeling lighting and then guiding reflectance, the network can effectively distinguish between lighting changes and inherent object properties without explicit shadow masks, alleviating color ambiguity at the structural level and laying the foundation for high-fidelity reconstruction of shadowless images.

[0038] The operation of the light-guided reflectivity Mamba module includes: The reflectivity flow feature is passed sequentially through the third linear layer, the second DW-Conv module, and SiLU activation to obtain the first intermediate reflectivity feature; After multiplying the first intermediate reflectance feature with the prior illumination feature, and then performing a second SSM and dimensional transformation, the second intermediate reflectance feature is obtained. The reflectivity flow characteristics are sequentially passed through the fourth linear layer and SiLU activation to obtain the third intermediate reflectivity characteristics; Multiplying the second intermediate reflectance feature with the third intermediate reflectance feature yields the reflectance feature representation.

[0039] In this embodiment, both the illumination flow and reflectivity flow employ multi-layer processing. Specifically, the illumination flow utilizes multiple illumination prior Mamba modules, and the reflectivity flow utilizes multiple illumination-guided reflectivity Mamba modules. This embodiment has three layers, and in each layer, the reflectivity flow features are constrained using corresponding illumination prior features.

[0040] S5: After splicing the prior illumination features and reflectivity features, input them into the decoder to obtain a shadowless image.

[0041] This embodiment only accepts the original shadow image as input. Through illumination-guided two-stream state space modeling and the collaborative effect between modules, it finally outputs a high-quality restored shadowless image without relying on any explicit shadow mask.

[0042] Example 2 In this embodiment, as Figure 2 As shown, a maskless image shadow removal method based on illumination-guided dual-stream Mamba is provided, which introduces an illumination layer sorting mechanism based on Embodiment 1.

[0043] After completing the explicit decoupling modeling of the two-stream Mamba, the prior lighting features output by the lighting stream have modeled the overall lighting distribution and shadow structure in the scene. However, the shadow regions in natural scenes are usually discrete and discontinuous in spatial distribution. Directly modeling the sequence based on the two-dimensional spatial structure is difficult to effectively reflect the real lighting variation relationship between different regions.

[0044] Based on this, this embodiment introduces an Illumination Hierarchical Sorting (IHS) mechanism, which utilizes illumination priors to rearrange and structure features, thereby enhancing the ability of the subsequent state-space model to model the order of illumination changes. This mechanism adopts an illumination-driven, spatially constrained hierarchical strategy: at the macro level, pixels are divided into several illumination levels based on illumination intensity to achieve clustering based on physical attributes; at the micro level, each illumination level is forced to strictly follow the original spatial scanning order, thereby preserving the local texture structure to the greatest extent.

[0045] In this embodiment, the pixels in the illumination stream feature are quantized into multiple illumination levels using an illumination layer sorter to obtain an illumination layer index. The illumination layer index is used to guide the sequence construction of the illumination prior Mamba module and the illumination-guided reflectivity Mamba module. During the quantization process, pixels with similar illumination intensities are grouped into the same illumination layer.

[0046] The sorting process for constructing the sequences of the illumination prior Mamba module and the illumination-guided reflectivity Mamba module both adopts a hierarchical constraint strategy based on lexicographical order, using the illumination level index as the primary sorting key and the scanning order of pixels in the original two-dimensional space as the secondary sorting key.

[0047] Specifically, let the single-channel illumination intensity characteristic map of the illumination flow characteristics output by the illumination flow branch be as follows: , To suppress the influence of small fluctuations in illumination values ​​on the sorting results, the continuous illumination intensity range was discretized into... Each lighting level. For locations in space... The pixel, and its corresponding illumination level index Defined as: in, This indicates a round-down operation. To control the hyperparameters of stratification granularity, The spatial location of a single-channel illumination intensity feature map pixels, This represents the minimum value of the illumination component of a pixel in a single-channel illumination intensity feature map. This represents the maximum value of the illumination component of a pixel in a single-channel illumination intensity feature map.

[0048] Through this quantization process, pixels with similar illumination intensities are grouped into the same illumination level, thereby eliminating the sorting instability caused by slight brightness differences and distinguishing the shadow core area, transition area and highlight area at the physical property level.

[0049] After completing the illumination hierarchy partitioning, a final global sort index needs to be generated to guide sequence construction.

[0050] To avoid destroying the texture structure by completely ignoring spatial location information, the sorting process adopts a hierarchical constraint strategy based on lexicographical order, that is, indexed by illumination level. The primary sorting key is the scanning order of pixels in the original two-dimensional space, while the secondary sorting key is the same. For any two pixels, the system prioritizes comparing their illumination levels, with the pixel with the lower level value (corresponding to a darker area) appearing first in the sequence. When two pixels belong to the same illumination level, they are arranged strictly according to a predefined spatial scanning order (such as raster scanning). The feature sequences generated based on this sorting strategy have the characteristics of intra-layer continuity and inter-layer order, enabling the state-space model to traverse the complete illumination level region within consecutive time steps while maintaining the relative compactness of local textures.

[0051] In this embodiment, when the illumination prior Mamba module generates the first intermediate illumination feature and when the illumination-guided reflectance Mamba module generates the first intermediate reflectance feature, a hierarchical constraint strategy based on lexicographical order and guided by the illumination level index is adopted.

[0052] Example 3 In this embodiment, as Figure 2 As shown, a maskless image shadow removal method based on illumination-guided dual-stream Mamba is provided. Based on the first embodiment, an illumination statistics guidance mechanism is introduced, which is implemented through the illumination histogram guidance module.

[0053] To ensure that illumination information continuously and stably guides reflectivity restoration during feature evolution, this embodiment introduces an Illumination Histogram Guidance Block (IHGB) between the illumination prior Mamba module and the illumination-guided reflectivity Mamba module for each layer. This block enables layer-by-layer information transmission from the illumination stream to the reflectivity stream. This structure avoids the information lag problem caused by performing feature fusion only once at the network end, allowing the illumination prior to continuously apply physical constraints throughout the entire evolution of reflectivity features, thereby improving the stability and consistency of the shadow region restoration process.

[0054] The illumination histogram guidance module only receives illumination prior features output from the corresponding layer's illumination prior Mamba module as input, denoted as: , C is the number of channels, used to characterize the first channel. The brightness distribution and shadow structure information of the scene in the layer feature space are explicit modeling results of the physical properties of illumination. Based on the prior illumination features, the illumination histogram guidance module does not directly participate in the calculation of reflectivity features. Instead, it first uses the illumination level index generated by the illumination layer sorting mechanism to statistically model the prior illumination features, mapping the illumination response in the spatial domain to the statistical domain. For the k-th illumination level, its corresponding histogram feature is calculated using ensemble pooling: in, Lighting levels The corresponding histogram eigenvalues, Lighting levels The set of pixels, for The number of elements, Let be the feature value of the prior illumination features of the l-th layer at spatial location (x, y). The pixel at spatial location (x, y) corresponding to illumination level k.

[0055] Through the above operations, spatial location differences are eliminated, and illumination features are uniformly modeled in the dimension of illumination intensity, thus forming a histogram prior representation that reflects the overall illumination distribution relationship. In the statistical domain, the obtained histogram feature sequence is further enhanced by context interaction operators to obtain updated statistical features. Subsequently, the enhanced statistical prior is inversely mapped back to the spatial domain according to the corresponding illumination level index, generating illumination guidance features aligned with the original illumination features in spatial dimensions as conditional information. This information is passed layer by layer to the corresponding illumination guidance reflectivity Mamba module and participates in the generation process of state-space model parameters during its input projection stage, constraining and modulating the state transition and update rate of reflectivity features. Through this unidirectional information transmission method, the global illumination distribution prior learned in the illumination flow can continuously play a guiding role throughout the entire evolution of reflectivity features without introducing reflectivity features into the IHGB for calculation, thereby ensuring the structural decoupling and physical consistency of illumination modeling and reflectivity restoration.

[0056] In this embodiment, the prior illumination features are used to obtain illumination-guided features through the illumination histogram guidance module. The reflectivity flow features and illumination-guided features are then input into the illumination-guided reflectivity Mamba module. The operation of the illumination-guided reflectivity Mamba module includes: The reflectivity flow feature is passed sequentially through the third linear layer, the second DW-Conv module, and SiLU activation to obtain the first intermediate reflectivity feature; After multiplying the first intermediate reflectance feature with the illumination guidance feature, and then performing a second SSM and dimensional transformation, the second intermediate reflectance feature is obtained. The reflectivity flow characteristics are sequentially passed through the fourth linear layer and SiLU activation to obtain the third intermediate reflectivity characteristics; Multiplying the second intermediate reflectance feature with the third intermediate reflectance feature yields the reflectance feature representation.

[0057] Example 4 In this embodiment, as Figure 2 As shown, a maskless image shadow removal method based on illumination-guided dual-stream Mamba is provided. Building upon Embodiment 1, it introduces both an illumination layering and sorting mechanism and an illumination statistical guidance mechanism. For details, please refer to Embodiments 2 and 3, which will not be repeated here.

[0058] The present invention was verified through the following experiments to verify Examples 1 to 4.

[0059] Datasets: The experiments used three benchmark datasets: SRD, ISTD, and ISTD+. The SRD dataset contains 3088 pairs of shadowed images and their corresponding shadowless images, of which 2680 pairs were used for training and 408 pairs for testing. This dataset was collected from natural scenes, containing complex textured backgrounds and diverse shadow shapes. In the preprocessing stage, this embodiment used bilinear interpolation to uniformly adjust all images to a resolution of 256×256, using shadow masks predicted by DHAN as input. The ISTD dataset provides 1870 triplet samples, each containing a shadowed image, its corresponding binary shadow mask, and a shadowless image. The dataset is divided into 1330 training samples and 540 test samples according to the standard. ISTD+ is an improved version of ISTD, correcting the illumination inconsistency between shadowed and shadowless images through post-processing techniques. This dataset contains more accurate registration and color correction, providing a more reliable benchmark for evaluating the performance of shadow removal algorithms. To verify the generalization performance of this method, additional experiments were conducted on the SBU dataset in this embodiment. This dataset includes more complex indoor lighting environments and object occlusion scenarios, testing the model's adaptability in real-world scenes.

[0060] Data Preprocessing and Augmentation Strategies: During the training phase, this embodiment employs multiple data augmentation techniques to enhance the model's robustness. First, the input image is randomly horizontally flipped (probability 0.5), then color dithering is applied to adjust the image's brightness, contrast, and saturation (adjustment increment 0.1). Simultaneously, random scaling (range [0.8, 1.2]) is used to simulate inputs at different scales. All images are normalized, scaling pixel values ​​to the range [0, 1]. During the testing phase, a sliding window strategy is used to process high-resolution images. The window size is 256×256, with an overlap of 32 pixels. The prediction results from each window are fused using a weighted average.

[0061] Performance Evaluation Metrics: This embodiment uses Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), and Root Mean Square Error (RMSE). PSNR is a fundamental metric for measuring image reconstruction quality, evaluating fidelity by calculating the pixel-level difference between the shadow-removed result and the true shadow-free image. SSIM assesses image quality from a human visual perception perspective, focusing on similarity in brightness, contrast, and structure. RMSE is calculated in the LAB color space, primarily evaluating the accuracy of color restoration.

[0062] Ablation experiment: To systematically verify the effectiveness of each core component in the illumination-guided two-stream Mamba architecture proposed in this invention, and to explore its specific contribution to the maskless shadow removal task, this invention conducted detailed ablation experiments on the ISTD+ dataset. This embodiment uses the complete two-stream Mamba model (IGDM, Implementation Example 4) as a benchmark, and constructs three variant configurations by progressively removing or replacing key modules, aiming to analyze the independent role of each component using the controlled variable method. Specifically, to verify the necessity of the Retinex-based dual-stream decoupling strategy, this embodiment first constructs a variant of Mamba without dual-stream (w / o Dual-Mamba), degrading the network to a single-stream architecture and directly utilizing a single Mamba encoder-decoder for end-to-end mapping learning. Second, to evaluate the effectiveness of the illumination hierarchical ranking mechanism in solving the shadow space discretization problem, this embodiment replaces the IHS mechanism with the raster scanning strategy commonly used in standard visual Mamba, constructing a variant of hierarchical ranking without IHS. Finally, to verify the importance of cross-stream guidance for feature recovery, this embodiment removes the illumination histogram guidance module and constructs a variant of guidance module removal (w / o IG Block), retaining only simple feature concatenation operations.

[0063] Table 1 Ablation experimental results on the ISTD+ dataset

[0064] Table 1 presents the quantitative test results of the aforementioned variant models on the ISTD+ dataset. In-depth analysis of the data reveals the key roles of each component. First, experimental results show that the two-stream decoupling architecture is the cornerstone of model performance. Removing this strategy significantly reduced all model metrics. This phenomenon strongly confirms that, in the absence of explicit mask supervision, single-stream networks are prone to "color ambiguity," struggling to effectively distinguish between inherent dark areas and shadow regions. The two-stream architecture, by forcing the network to separate illumination and reflectivity attributes, successfully avoids this problem. Second, the illumination hierarchical sorting mechanism is crucial for maintaining the structural fidelity of the image. Data shows that replacing IHS with ordinary raster scanning significantly reduced the SSIM metric. This confirms the limitations of traditional spatial scanning strategies in handling irregular, fragmented shadow regions, severing long-range dependencies between physically homogeneous but spatially discrete shadow pixels. IHS, through physical clustering, significantly enhances Mamba's ability to model global shadow features.

[0065] Furthermore, the illumination histogram guidance module also plays an indispensable role in improving restoration accuracy. Removing the IG Block led to an increase in the RMSE metric, indicating that simple feature stitching cannot fully utilize the information from the illumination flow. The IGBlock, through global interaction in the histogram domain, can transform abstract illumination statistical priors into precise spatial control signals, playing a crucial role in correcting color deviations in shadow areas and eliminating boundary artifacts. In summary, the complete model achieved optimal performance across all metrics, fully demonstrating the rationality and complementarity of the two-stream architecture, hierarchical sorting mechanism, and histogram guidance module. These three elements work together to form an efficient maskless shadow removal framework.

[0066] Comparative experiment: To objectively and quantitatively verify the effectiveness of the illumination-guided two-stream Mamba architecture proposed in this invention, this embodiment conducted extensive quantitative comparative experiments on three mainstream shadow removal benchmark datasets (ISTD+, ISTD, and SRD). The benchmarks covered traditional physical model methods, classic mask-based deep learning methods (such as BMNet, DHAN, and DSC), and state-of-the-art mask-free methods (such as ShadowRefiner and ReHiT). Peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and root mean square error (RMSE) were used as the core evaluation metrics in the experiments.

[0067] Table 2. Comparison results with different models on the ISTD+ dataset.

[0068] On the ISTD+ dataset, which best reflects the ability to restore true illumination, our method demonstrates significant performance advantages. As shown in Table 2, our method achieves a PSNR of 31.84 dB, an SSIM of 0.989, and a RMSE of 3.03, achieving the best overall performance. Compared to the CVPR 2024 champion solution, ShadowRefiner (PSNR 31.03 dB, SSIM 0.928), and the latest ReHiT (PSNR 31.16 dB, SSIM 0.925), our method improves PSNR by 0.81 dB and 0.68 dB respectively, and achieves a qualitative leap in SSIM. This result fully demonstrates that introducing an illumination hierarchical sorting mechanism can capture illumination distribution more effectively than traditional Transformer or CNN, thereby achieving more accurate structure and texture restoration under maskless conditions.

[0069] Secondly, on the original ISTD dataset, despite some color inconsistencies, our method still demonstrates excellent robustness. As shown in Table 3, our method achieves a PSNR of 29.79 dB and an SSIM of 0.965. Compared to similar maskless methods like ReHiT (28.81 dB) and ShadowRefiner (28.75 dB), our method maintains a significant advantage of approximately 1 dB in PSNR, indicating that the illumination guidance strategy can more accurately locate and repair shadows without relying on explicit masks. Notably, our invention even surpasses the classic strongly supervised methods BMNet (0.959) and DSC (0.944) in the SSIM metric (0.965), further confirming the significant advantage of the two-stream Mamba architecture in maintaining image structural integrity and effectively avoiding structural distortion caused by incorrect estimation.

[0070] Table 3 shows the comparison results with different models on the ISTD dataset.

[0071] Finally, on the SRD dataset, which contains a large number of complex scenes, our method also demonstrates strong generalization ability. As shown in Table 4, although some strongly supervised methods (such as BMNet) perform strongly in terms of PSNR, our method achieves an RMSE value of 3.77, which is better than mainstream methods such as SSVA-Net (3.84), DMTN (4.06), and DSC (5.71). Since the RMSE index is more sensitive to large numerical errors, this leading index indicates that our method has a unique advantage in controlling pixel-level restoration errors and can effectively reduce severe color shifts or artifacts. Even under the difficult condition of lacking mask guidance, our method still achieves a PSNR of 26.38dB, which is better than methods such as TBRNet (24.53dB), strongly demonstrating the effectiveness and practical value of the implicit illumination guidance mechanism in processing unpaired data. Combining the quantitative experimental results of the three datasets, our method has achieved a level that surpasses the existing maskless state-of-the-art methods in all core indicators, especially in terms of structural fidelity and restoration accuracy.

[0072] Table 4 shows the comparison results with different models on the SRD dataset.

[0073] Visual analysis: Figure 3 and Figure 4The visualizations of IGDM's shadow removal performance on the ISTD+ and SRD datasets are presented respectively. It can be clearly observed that under various complex lighting conditions, this method demonstrates stable and superior performance compared to the contrasting methods in terms of brightness restoration, boundary transition, and color consistency in shadow areas.

[0074] In the ISTD+ dataset, from Figure 3 It can be seen that some comparative methods still exhibit varying degrees of brightness unevenness in the shadow restoration area, manifesting as the restored shadow area being significantly brighter or darker than the surrounding non-shadow areas, or exhibiting slight halo artifacts at the shadow boundaries. Our proposed method, however, can eliminate residual shadows while maintaining a natural and smooth brightness transition between shadow and non-shadow areas, resulting in better integration of the restored area into the overall lighting distribution. Furthermore, in scenes with complex shadow structures and large areas of soft shadows, our method provides more complete restoration of texture details, avoiding detail loss caused by over-enhancement. In the SRD dataset, from... Figure 4 The visualization results show that some comparison methods tend to exhibit significant color shifts or localized distortions in high-saturation areas or scenes with drastic color changes, and they struggle to achieve seamless integration with the background at shadow boundaries. In contrast, this method maintains good color consistency across various scenarios, effectively suppresses shadow boundary artifacts, and allows shadow areas to blend naturally with the surrounding environment. Furthermore, this method demonstrates a strong advantage in preserving structural information such as building edges and ground textures, resulting in visually more realistic and coherent restoration results.

[0075] comprehensive Figure 3 and Figure 4 The visualization results show that the proposed method can stably model complex shadow scenes in different datasets without explicit shadow masks, effectively achieving natural elimination of shadow areas and consistent restoration of the overall structure, further verifying the practicality and generalization ability of the method in real complex scenes.

[0076] This embodiment also provides a maskless image shadow removal system based on illumination-guided dual-stream Mamba, and the technical solution adopted is as follows: including: The image acquisition module is used to acquire shadow images; The encoding module is used to obtain a high-dimensional latent feature representation of the shadow image through a feature encoder, and then obtain illumination flow features and reflectivity flow features based on Retinex theory and through a feature decoupling mapping operator. The illumination stream processing module is used to obtain illumination prior features by passing the illumination stream features through the illumination prior Mamba module; The reflectivity stream processing module is used to input the reflectivity stream features and illumination prior features into the illumination-guided reflectivity Mamba module to obtain a reflectivity feature representation under illumination prior constraints. The decoding module is used to input the illumination prior features and reflectivity features into the decoder to obtain a shadowless image.

[0077] An illumination layer sorter is used to quantize pixels in the illumination stream features into multiple illumination layers to obtain illumination layer indices. The illumination layer indices are used to guide the sequence construction of illumination prior Mamba modules and illumination-guided reflectivity Mamba modules. During the quantization process, pixels with similar illumination intensities are grouped into the same illumination layer.

[0078] The illumination histogram guidance module is used to statistically model illumination prior features based on illumination hierarchy indexes, mapping illumination responses in the spatial domain to the statistical domain. The illumination histogram guidance module is located between the illumination prior Mamba module and the illumination-guided reflectivity Mamba module. Illumination prior features are processed by the illumination histogram guidance module to obtain illumination-guided features.

[0079] This embodiment also provides a computer device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the above-described method for maskless image shadow removal based on illumination-guided dual-stream Mamba.

[0080] This embodiment also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described method for maskless image shadow removal based on illumination-guided dual-stream Mamba.

[0081] This invention achieves accurate restoration of complex shadow regions without the need for prior shadow masking by explicitly decoupling illumination and reflectivity features and introducing the long-range modeling capabilities of a state-space model. Based on the Retinex physical model, this invention differentiates the illumination distribution and inherent object properties at the network structure level, effectively mitigating color ambiguity issues in maskless shadow removal tasks.

[0082] To overcome the limitations of discrete spatial distribution of shadow regions and the modeling capabilities of traditional sequential scanning strategies, this invention proposes an illumination layering and sorting mechanism. This mechanism dynamically layers and reorders features based on illumination intensity, enabling shadow regions with similar physical properties but spatial discontinuities to be continuously modeled in the sequential domain. This enhances the Mamba model's ability to perceive global illumination consistency. Building upon this, an illumination histogram guidance module is constructed. This module maps the spatial distribution of features in the illumination stream to the statistical domain for global modeling. Furthermore, refined illumination priors are injected layer by layer into the reflectivity modeling process, continuously guiding the evolution of reflectivity features through illumination information.

[0083] Through the synergistic effect of dual-stream decoupling modeling, illumination-aware sequence recombination, and statistical domain guidance mechanism, this invention can simultaneously characterize the overall illumination structure and local texture details of the shadow region. It achieves superior performance compared to existing maskless methods on multiple benchmark datasets, verifying the effectiveness and practical value of this technical solution in complex real-world scenarios.

[0084] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A maskless image shadow removal method based on illumination-guided dual-stream Mamba, characterized in that, include: S1: Obtain the shadow image; S2: The shadow image is processed by a feature encoder to obtain a high-dimensional latent feature representation, and then, based on Retinex theory, a feature decoupling mapping operator is used to obtain illumination flow features and reflectivity flow features. S3: The illumination flow features are obtained through the illumination prior Mamba module to obtain illumination prior features; S4: Input the reflectivity flow features and illumination prior features into the illumination-guided reflectivity Mamba module to obtain the reflectivity feature representation under illumination prior constraints; S5: Input the prior illumination features and reflectivity features into the decoder to obtain a shadowless image.

2. The maskless image shadow removal method based on illumination-guided dual-stream Mamba as described in claim 1, characterized in that, The pixels in the illumination flow feature are quantized into multiple illumination levels to obtain illumination level indexes. These illumination level indexes are used to guide the sequence construction of the illumination prior Mamba module and the illumination-guided reflectivity Mamba module. During the quantization process, pixels with similar illumination intensities are grouped into the same illumination level.

3. The maskless image shadow removal method based on illumination-guided dual-stream Mamba as described in claim 2, characterized in that, The sorting process for constructing the sequences of the illumination prior Mamba module and the illumination-guided reflectivity Mamba module both adopts a hierarchical constraint strategy based on lexicographical order, using the illumination level index as the primary sorting key and the scanning order of pixels in the original two-dimensional space as the secondary sorting key.

4. The maskless image shadow removal method based on illumination-guided dual-stream Mamba as described in claim 2, characterized in that, Illumination Hierarchy Index for: in, This indicates a round-down operation. To control the hyperparameters of stratification granularity, The spatial location of a single-channel illumination intensity feature map pixels, This represents the minimum value of the illumination component of a pixel in a single-channel illumination intensity feature map. This represents the maximum value of the illumination component of a pixel in a single-channel illumination intensity feature map.

5. The maskless image shadow removal method based on illumination-guided dual-stream Mamba as described in claim 1, characterized in that, In step S4, the prior features of illumination are passed through the illumination histogram guidance module and then input into the illumination-guided reflectance Mamba module. The illumination histogram guidance module is used to perform statistical modeling of illumination prior features based on illumination hierarchy index, and to map the illumination response in the spatial domain to the statistical domain.

6. The maskless image shadow removal method based on illumination-guided dual-stream Mamba as described in claim 5, characterized in that, The calculation formula for the illumination histogram guidance module is as follows: in, Lighting levels The corresponding histogram eigenvalues, Lighting levels The set of pixels, for The number of elements, Let be the feature value of the prior illumination features of the l-th layer at spatial location (x, y). The pixel at spatial location (x, y) corresponding to illumination level k.

7. The maskless image shadow removal method based on illumination-guided dual-stream Mamba as described in claim 1, characterized in that, The working process of the illumination prior Mamba module includes: The illumination flow features are sequentially passed through the first linear layer, the first DW-Conv module, and SiLU activation to obtain the first intermediate illumination features; The second illumination intermediate feature is obtained by transforming the first illumination intermediate feature through the first SSM and dimension transformation; The illumination flow features are sequentially passed through the second linear layer and activated by SiLU to obtain the third intermediate illumination feature; Multiplying the intermediate features of the second illumination by the intermediate features of the third illumination yields the prior illumination features; The operation of the light-guided reflectivity Mamba module includes: The reflectivity flow feature is passed sequentially through the third linear layer, the second DW-Conv module, and SiLU activation to obtain the first intermediate reflectivity feature; After multiplying the first intermediate reflectance feature with the prior illumination feature, and then performing a second SSM and dimensional transformation, the second intermediate reflectance feature is obtained. The reflectivity flow characteristics are sequentially passed through the fourth linear layer and SiLU activation to obtain the third intermediate reflectivity characteristics; Multiplying the second intermediate reflectance feature with the third intermediate reflectance feature yields the reflectance feature representation.

8. A maskless image shadow removal system based on illumination-guided dual-stream Mamba, characterized in that, A method for performing maskless image shadow removal based on illumination-guided dual-stream Mamba as described in any one of claims 1 to 7, comprising: The image acquisition module is used to acquire shadow images; The encoding module is used to obtain a high-dimensional latent feature representation of the shadow image through a feature encoder, and then obtain illumination flow features and reflectivity flow features based on Retinex theory and through a feature decoupling mapping operator. The illumination stream processing module is used to obtain illumination prior features by passing the illumination stream features through the illumination prior Mamba module; The reflectivity stream processing module is used to input the reflectivity stream features and illumination prior features into the illumination-guided reflectivity Mamba module to obtain the reflectivity feature representation under illumination prior constraints. The decoding module is used to input the illumination prior features and reflectivity features into the decoder to obtain a shadowless image.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements a maskless image shadow removal method based on illumination-guided dual-stream Mamba, as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements a maskless image shadow removal method based on illumination-guided dual-stream Mamba, as described in any one of claims 1 to 7.

Images