An image shadow removal method based on contrastive learning

Abstract

The application belongs to the technical field of image processing, and particularly relates to an image shadow removal method based on contrast learning, which comprises the following steps: separating direct light and ambient light of a shadow image and a non-shadow image by using a feature separation reconstruction network, and reconstructing a preliminary shadow removal result by using a joint decoder to perform contrast learning on the separated features; performing local region enhancement on an original input image, and pairing the shadow image and the non-shadow image to perform random pixel extraction to replace the original shadow input and non-shadow input; the application uses the idea of separating ambient light and direct light by using an asymmetric encoder, converts learning of a complex mapping into learning of a single mapping problem of ambient light and direct light, uses local region enhancement to improve the robustness of a model, optimizes the processing capacity of a boundary region, and simulates the uncertainty of a real scene, so that a high-quality non-shadow image is obtained.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and in particular to image shadow removal, specifically relating to an image shadow removal method based on contrastive learning. Background Technology

[0002] Shadows are dark areas formed when light is blocked or scattered by an object, and their appearance and perception are affected by brightness and color. Shadows in images or videos inevitably negatively impact the performance of downstream computer vision tasks, such as image decomposition, visual recognition, object tracking, and trajectory prediction. Therefore, shadow removal, as an important topic in image preprocessing, has been extensively studied, but it still faces many challenges in complex scenes.

[0003] Current shadow removal methods can be broadly categorized into two types: traditional methods based on image priors and deep learning-based methods. Early traditional methods relied on physical imaging models, using image gradients, image intensity, and color transfer to detect shadows and generate shadow-free images. In recent years, with the release of large-scale shadow removal datasets, deep learning-based shadow removal methods have outperformed traditional methods, achieving significant results. For example, DSC utilizes orientation-aware spatial contextual attention features for shadow detection and removal; Fu et al. treated shadow regions as underexposed areas, learning from multiple overexposed images and adaptively fusing them to achieve shadow removal; BMNet employs a reversible framework, using a reverse mapping of shadow generation for shadow removal; and SG-ShadowNet treats shadow removal as a style transfer problem, utilizing color priors from non-shadow regions to ensure overall color consistency in the shadow removal results.

[0004] However, these methods ignore the fundamental fact of shadow formation: pixel values ​​are determined by both illumination and reflectivity. They require a single network to simultaneously learn the complex mapping relationship between direct light and ambient light, but because the information from these two feature domains is intertwined, a single network struggles to recover both direct and ambient light. Although Le et al. used a simplified shadow illumination model combined with deep learning to illuminate shadow areas using parameter estimation, their illumination model was linear, while actual shadow formation is a non-linear process, and the network did not distinguish between direct and ambient light, resulting in limited shadow removal effectiveness.

[0005] Although some feature separation models have been proposed, existing models do not impose specific constraints on the separated features, resulting in incomplete feature separation. Shadow images inherently contain both shadow and ambient areas. Feature separation of shadow images should yield shadow features and non-shadow features. However, previous feature separation models did not constrain the separated features, leading to the presence of non-shadow features within shadow features and vice versa. This incomplete separation results in a visual color difference between the shadow-removed image and the true shadow-free image. Summary of the Invention

[0006] To address the aforementioned issues, this invention provides an image shadow removal method based on contrastive learning. By incorporating contrastive learning into feature separation, the method alleviates artifacts and color differences in the initial removal results to a certain extent, resulting in high-quality shadow-free images.

[0007] The specific plan includes the following steps:

[0008] S1. Construct an image shadow removal model, which includes a feature separation and reconstruction module, a non-shadow information guidance module, and a discriminator; collect multiple image groups to form training data, each image group including the initial shadow image to be removed, the initial shadow-free image, and the initial shadow mask image obtained in the same scene;

[0009] S2. Perform local region enhancement on each image group to obtain the corresponding preprocessed image group; a preprocessed image group includes the image I to be shadowed. SE Shadowless Image I SFE and shadow mask image M;

[0010] S3. For each preprocessed image group, select the image I to be de-enhanced by removing shadows. SE The first image pair is formed by combining the shadow mask image M and the shadow-free enhanced image I. SFE The shadow mask image M forms a second image pair;

[0011] S4. Separate the common input features of the first image pair and the second image pair to reconstruct the network, and obtain the reconstructed shadow image I′. S Reconstructing a shadowless image S ′ F and preliminary shadow removal image I C ; Calculate self-reconstruction loss and roughness loss

[0012] S5. Extract and reconstruct the shadow image I′ S Common feature E c_s and unique features E s_s Extract and reconstruct the shadowless image I S ′F Common feature E c_sf and unique features E sf_sf Based on these four features, comparative learning is performed, and the comparative learning loss is calculated.

[0013] S6. Dilate the shadow mask image M using morphological operations to obtain the shadow region boundary mask image M. b and the prior mask image M of the local non-shaded region p ;

[0014] S7. Remove initial shadows from image I C and shadow area boundary mask image M b Channel overlay yields a combined image, which is then input into a non-shadow information-guided network to obtain a refined shadowless image I. fine ; Calculate the refinement loss

[0015] S8. Transform the image without shadows into I fine and the image to be removed I S The first sample pair is formed by combining the shadowless image I. SF and the image to be removed I S Form a second sample pair; input the first and second sample pairs into the discriminator to obtain the discrimination result; calculate the adversarial loss.

[0016] S9. Self-reconstruction loss Roughness loss Comparative learning loss Detailed loss and generating adversarial losses The total loss is obtained by weighted summation, and the model parameters are trained by backpropagation until the parameters converge and the training is complete.

[0017] S10. Input the image to be processed into the trained image shadow removal model to obtain the shadow removal result.

[0018] The beneficial effects of this invention are:

[0019] This invention addresses the issue of shadow removal by separating direct light and ambient light, simplifying the complex mapping from shadows to shadowless images into a single mapping between direct and ambient light. In the network design, the feature separation and reconstruction network adaptively decomposes the image into direct and ambient light features during the separation and reconstruction phases. Furthermore, a contrastive learning mechanism is introduced based on feature separation, making the resulting color more closely resemble human perception of a shadowless image. Through contrastive learning, the model can better separate shadow and non-shadow features, reducing shadow residue caused by feature confusion. Local contrastive learning enhances the model's ability to reconstruct details at shadow boundaries, avoiding artifacts and color distortion. By constraining the feature space, contrastive learning allows the model to better adapt to lighting variations and material diversity in complex scenes.

[0020] Furthermore, this invention employs local region enhancement techniques to improve the model's generalization and robustness. Shadow boundaries are typically transitional regions where color and illumination changes, and traditional global optimization methods are prone to artifacts or blurring in these areas. Local region enhancement, through targeted training and attention mechanisms, can more accurately recover boundary details. Color recovery in shadow regions needs to be consistent with the surrounding non-shadow regions, but global methods may lead to color discrepancies due to neglecting local statistical characteristics. The contribution of local region enhancement lies in using local normalization or color propagation strategies to leverage local color priors from non-shadow regions to guide shadow region recovery and force the shadow removal results to align with the color distribution of the real shadowless image in the local region.

[0021] Through extensive experiments on public datasets, this invention demonstrates superior shadow removal performance compared to other state-of-the-art methods. Attached Figure Description

[0022] Figure 1 This is a flowchart of the method of the present invention;

[0023] Figure 2 This is a network model framework diagram of the present invention;

[0024] Figure 3 This is a schematic diagram of the feature separation and reconstruction module of the present invention;

[0025] Figure 4 This is a schematic diagram of the local region normalization module of the present invention;

[0026] Figure 5 This is a visualization comparing the performance of this invention with other methods on the ISTD+ and SRD test sets;

[0027] Figure 6 This figure shows the quantitative comparison results of the present invention with other methods on the SRD test set. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] This invention provides an image shadow removal method based on contrastive learning, such as... Figure 1-3 As shown, it includes the following steps:

[0030] S1. Construct an image shadow removal model, which includes a feature separation and reconstruction module, a non-shadow information guidance module, and a discriminator; collect multiple image groups to form training data, each image group including the initial shadow image to be removed, the initial shadow-free image, and the initial shadow mask image obtained in the same scene;

[0031] S2. Perform local region enhancement on each image group to obtain the corresponding preprocessed image group; a preprocessed image group includes the image I to be shadowed. SE Shadowless Image I SFE and shadow mask image M;

[0032] S3. For each preprocessed image group, select the image I to be de-enhanced by removing shadows. SE The first image pair is formed by combining the shadow mask image M and the shadow-free enhanced image I. SFE The shadow mask image M forms a second image pair;

[0033] S4. Separate the common input features of the first image pair and the second image pair to reconstruct the network, and obtain the reconstructed shadow image I′. S Reconstructing a shadowless image S ′ F and preliminary shadow removal image I C ; Calculate self-reconstruction loss and roughness loss

[0034] S5. Extract and reconstruct the shadow image I′ S Common feature E c_s and unique features E s_s Extract and reconstruct the shadowless image I S ′ F Common feature E c_sf and unique features E sf_sf Based on these four features, comparative learning is performed, and the comparative learning loss is calculated.

[0035] Specifically, contrastive learning is a self-supervised learning method. Its core idea is to construct a discriminative feature space by learning the similarities and differences between samples. Specifically, contrastive learning maximizes the similarity between positive sample pairs (similar samples) while minimizing the similarity between negative sample pairs (dissimilar samples), thus enabling the model to learn more discriminative feature representations. In shadow removal tasks, contrastive learning can be used to constrain the feature space, making the feature representations of shadowed and non-shadowed images more reasonable, thereby improving the shadow removal effect. The key to shadow removal is to decompose the image into two parts: direct light (non-shadowed areas) and ambient light (shadowed areas). Through contrastive learning, the following constraints can be designed:

[0036] Positive sample pairs: The common features (such as material, texture, etc.) of the shadowed image and the non-shadowed image should be as similar as possible.

[0037] Negative sample pairs: The shadow features of the shadowed image (i.e., the unique features of the shadowed image) should be as far away from the non-shadowed features of the non-shadowed image as possible.

[0038] The model minimizes the distance between positive sample pairs and maximizes the distance between negative sample pairs. This constraint allows the model to better separate shadow features from non-shadow features, thereby improving the accuracy of shadow removal.

[0039] Contrastive learning does not require a large amount of labeled data and can generate positive sample pairs through data augmentation, making it suitable for scenarios with scarce data. Through contrastive learning, the model can learn more discriminative feature representations, making it suitable for tasks such as classification and retrieval. Contrastive learning is also highly robust to noise and changes in data distribution.

[0040] S6. Dilate the shadow mask image M using morphological operations to obtain the shadow region boundary mask image M. b and the prior mask image M of the local non-shaded region p ;

[0041] S7. Remove initial shadows from image I C and shadow area boundary mask image M b Channel overlay yields a combined image, which is then input into a non-shadow information-guided network to obtain a refined shadowless image I. fine ; Calculate the refinement loss

[0042] S8. Transform the image without shadows into I fine and the image to be removed I S The first sample pair is formed by combining the shadowless image I. SF and the image to be removed I SForm a second sample pair; input the first and second sample pairs into the discriminator to obtain the discrimination result; calculate the adversarial loss.

[0043] S9. Self-reconstruction loss Roughness loss Comparative learning loss Detailed loss and generating adversarial losses The total loss is obtained by weighted summation, and the model parameters are trained by backpropagation until the parameters converge and the training is complete.

[0044] S10. Input the image to be processed into the trained image shadow removal model to obtain the shadow removal result.

[0045] Specifically, to enhance the model's attention to and learning of local features, and improve the detail recovery and visual consistency of shadow removal, this invention proposes a local region enhancement method in the data processing section. Step S2, which performs local region enhancement on each image group, specifically includes:

[0046] S21. Perform a Mask operation on the initial image to be shadowed to divide it into shadow and environment regions; select 6 first circular regions with a radius of 30px in the shadow region; for each first circular region, select 10% of the pixels and set their values ​​to 0; mark the first circular regions at the same positions in the initial image without shadows, and select the same pixels and set their values ​​to 0.

[0047] S22. Select 16 second circular regions with a radius of 40px in the environment region. For each second circular region, select 15% of the pixels and set their values ​​to 0. Mark the second circular regions at the same positions in the initial shadowless image and select the same pixels and set their values ​​to 0.

[0048] S23. Perform dilation and erosion operations on the initial image to be shadowed to divide it into boundary regions. Select six third circular regions with a radius of 20px in the boundary regions. For each third circular region, select 5% of the pixels and set their values ​​to 0. Mark the third circular regions at the same positions in the initial image without shadows and select the same pixels and set their values ​​to 0. At this point, the initial image to be shadowed to be shadowed and the initial image without shadows have been processed in the same way to make them correspond.

[0049] S24. The marked initial image to be removed, the marked initial image without shadow, and the initial shadow mask image are scaled to 286×286 and then randomly cropped to obtain a cropped image to be removed with a size of 256×256, a cropped image without shadow with a size of 256×256, and a cropped shadow mask image with a size of 256×256.

[0050] S25. Rotate and flip the image to be removed (without shadow), the image without shadow, and the image with the shadow mask, respectively, with a 50% probability, to obtain the image to be removed (I). SE Shadowless Image I SFE And the shadow mask image M.

[0051] Specifically, during the model training phase, an Adam optimizer with an initial learning rate of 0.0002 was used, and the ReduceLROnPlateau strategy was employed to decay the learning rate. Simultaneously, a total loss function was constructed, and the parameters of the SGGAN shadow removal network were updated using a backpropagation mechanism to complete the training of the SGGAN shadow removal network. The calculation formula:

[0052]

[0053] Wherein, λ1~λ5 are the hyperparameters of the loss function, and the above hyperparameters are set to: 10, 10, 10, 0.1, 0.1 respectively.

[0054] The various loss functions are described below:

[0055] 1) Self-reconstruction loss Self-reconstruction loss optimizes the L1 distance between the output of the self-reconstruction decoder and the two inputs, prompting different encoders to encode different features:

[0056]

[0057] ||·||1 represents the L1 distance.

[0058] 2) Roughness loss The coarsening loss is used to optimize the L1 distance between the initial removal result and the true shadow-free image, and can be expressed as:

[0059]

[0060] 3) Refine the loss The thinning loss is used to optimize the L1 distance between the thinned result and the true shadowless image, and can be expressed as:

[0061]

[0062] 4) Generate adversarial losses Image pairs are input into a discriminator for evaluation. Image pairs containing shadow removal results are classified as fake, while image pairs containing true shadow-free images are classified as real. During adversarial training, the generator is encouraged to produce more realistic shadow-free images. The adversarial loss can be expressed as:

[0063]

[0064] D stands for discriminator. This indicates that for variable I S I SF and I fine The expected value.

[0065] 5) Comparative learning loss Reconstruct the shadowless image I S ′ F Common feature E c_sf As a positive sample, the shadow image I′ will be reconstructed. S E's unique characteristics s_s As a negative sample, let the reconstructed shadow image I′ S Common feature E c_s Proximity to positive samples and distance from negative samples. The formula can be expressed as:

[0066]

[0067] α represents a hyperparameter used to adjust the relative importance of positive and negative sample pairs; m is a hyperparameter used to control the minimum distance between negative sample pairs, avoiding over-optimization of the model that could lead to feature collapse.

[0068] Specifically, the feature separation and reconstruction network includes encoder E S Encoder E C Encoder E SF Joint decoder J SR and joint decoder J S2SF ;in:

[0069] Encoder E S Encoder E C and encoder E SF The structures are identical, each consisting of 6 convolutional layers, with cascaded BatchNorm and LeakyReLU layers added between every two adjacent convolutional layers; the parameters of each convolutional layer are set to a kernel size of 4, a stride of 2, and padding of 1; the number of input channels of the 6 convolutional layers are 4, 64, 128, 256, 512, and 512 respectively, and the number of output channels are 64, 128, 256, 512, 512, and 512 respectively;

[0070] Joint Decoder J SR and joint decoder J S2SFThe structures are identical, each consisting of 6 deconvolutional layers. LeakyReLU and BatchNorm layers are added before and after the first 5 deconvolutional layers, respectively, and LeakyReLU and tanh layers are added before and after the 6th deconvolutional layer, respectively. The parameters of each deconvolutional layer are set to a kernel size of 4, a stride of 2, and padding of 1. The number of input channels of the 6 deconvolutional layers are 1024, 2048, 2048, 1024, 512, and 256, respectively, and the number of output channels are 1024, 1024, 512, 256, 128, and 3, respectively.

[0071] Specifically, in existing shadow removal methods, the network needs to learn complex mappings of ambient light and direct light simultaneously during the shadow removal process. Such networks have a bottleneck because they cannot simultaneously handle the recovery of both ambient and direct light. To address this issue, this invention constructs a feature separation and reconstruction network based on the shadow mapping relationship in the lighting model. Starting from the perspective of separating ambient light and direct light, it learns the features of ambient and direct light through the reconstruction process, transforming the complex process of learning the shadow-to-shadow-free mapping into two simple mapping problems: learning ambient light and direct light. This improves the shadow removal effect. Typically, in the lighting model, the relationship between a shadowed image and a shadow-free image can be expressed by the following formula:

[0072]

[0073] in, R represents the value of the i-th pixel in a shadowless image at wavelength λ. i (λ) represents the reflectance of the i-th pixel at wavelength λ. This represents the ambient light at wavelength λ for the i-th pixel. This represents the direct light at wavelength λ on the i-th pixel. This indicates that the value of the i-th pixel in the shaded image is obtained at wavelength λ. This represents the ambient light attenuation at wavelength λ for the i-th pixel. Based on the above analysis, the network is reconstructed by separating the common input features of the first image pair and the second image pair, as follows: Figure 3 As shown, it includes:

[0074] S41. Input the first image pair into encoder E respectively. S and encoder E C The characteristics of ambient light attenuation were obtained. and ambient light characteristics The second image pair is input into encoder E respectively. SF and encoder E C The characteristics of direct light were obtained. and ambient light characteristics

[0075] S42. Characteristics of ambient light attenuation and ambient light characteristics The first joint feature is obtained by combining the features of direct light. and ambient light characteristics The combination yields the second joint feature;

[0076] S43. Input the first joint feature into the joint decoder J respectively. SR and joint decoder J S2SF To obtain the reconstructed shadow image and preliminary shadow removal image I C ;

[0077] S44. Input the second joint feature into the joint decoder J SR Obtain the reconstructed shadowless image

[0078] Specifically, the non-shadow information guidance network includes an encoder and a decoder, wherein:

[0079] The encoder consists of six convolutional layers. After the first four convolutional layers, cascaded local region normalization (MRN) modules and LeakyReLU layers are added, respectively. After the fifth convolutional layer, cascaded BatchNorm and LeakyReLU layers are added. The parameters for each convolutional layer are set as follows: kernel size of 4, stride of 2, and padding of 1. The number of input channels for these six convolutional layers are 4, 64, 128, 256, 512, and 512 in sequence, and the corresponding number of output channels are 64, 128, 256, 512, 512, and 512 in sequence.

[0080] The decoder consists of six deconvolutional layers. The first deconvolutional layer is preceded and followed by a LeakyReLU layer and a BatchNorm layer, respectively. The second through fifth deconvolutional layers are preceded and followed by LeakyReLU layers and a local region normalization module, respectively. The sixth deconvolutional layer is preceded and followed by a LeakyReLU layer and a tanh layer, respectively. For each deconvolutional layer, the parameters are set as follows: kernel size of 4, stride of 2, and padding of 1. The number of input channels for these six deconvolutional layers are 512, 1024, 1024, 512, 256, and 128, respectively, and the number of output channels are 512, 512, 256, 128, 64, and 3, respectively.

[0081] Specifically, such as Figure 4 As shown, the processing steps of the local region normalization module include:

[0082] S15. Convert the intermediate feature image F output from the previous layer. inAs input to the local region normalization module;

[0083] S16. Transfer the intermediate feature image F in The input is fed into a 3×3 convolutional layer to obtain intermediate convolutional feature images; a shadow mask image M is used to guide the processing of the intermediate convolutional feature images to obtain shadow region features F. S and the feature of the shadowless area F SF ;

[0084] S17. Features of the shaded area F S and the feature of the shadowless area F SF Perform region normalization to obtain normalized feature F Norm ;

[0085] S18. Transfer the intermediate feature image F in Compared with the prior mask image M of the local non-shaded region p Multiplying them together yields the prior P for the locally unshaded region. color ;

[0086] S19. Prioritize the local non-shaded areas P color The first and second convolutional modules are input respectively to learn the affine parameters γ and β; the first and second convolutional modules have the same structure, each including three 1×1 convolutional layers and two LeakyReLU layers;

[0087] S20. Using affine parameters γ and β to normalize the feature F Norm Perform an affine transformation to obtain the output F of the local region normalization module. out .

[0088] Please see Figure 5 and Figure 6 The method of this invention has better shadow removal performance than other methods in various scenarios. The color difference inside and outside the shadow area of ​​the removal result is well controlled and the artifacts are effectively reduced.

[0089] In one embodiment, the model of this invention is quantitatively compared with the model in patent CN2024101883208 on the ISTD+ test set, involving a comparison of two metrics: CIEDE2000 and LPIPS. CIEDE2000, an improved version of the CIELAB color space, is a standard for quantifying color differences and can assess color differences perceived by the human eye. LPIPS is a deep learning-based image similarity measurement method. LPIPS extracts image features through a pre-trained deep neural network and calculates distances in the feature space, thus better reflecting the human eye's perception of image quality. The results are shown in Table 1. This invention can more effectively remove shadows, solving the problem that the shadow-removed image obtained by existing models has a color difference visually compared to the true shadow-free image.

[0090] Table 1 Comparison Results

[0091]

[0092] Using this invention, ambient light and direct light in shadowed and shadowless images can be effectively separated, improving the overall performance of the network. At the same time, due to the reasonable introduction of non-shadow priors, the colors of shadowed areas are accurately restored, thereby achieving better shadow removal performance.

[0093] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "setting," "connection," "fixing," "rotation," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal connection of two components or the interaction between two components. Unless otherwise explicitly limited, those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0094] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An image shadow removal method based on contrastive learning, characterized in that, Includes the following steps: S1. Construct an image shadow removal model, which includes a feature separation and reconstruction module, a non-shadow information guidance module, and a discriminator; collect multiple image groups to form training data, each image group including the initial shadow image to be removed, the initial shadow-free image, and the initial shadow mask image obtained in the same scene; S2. Perform local region enhancement on each image group to obtain the corresponding preprocessed image group; a preprocessed image group includes the image I to be shadowed. SE Shadowless Image I SFE and shadow mask image M; Step S2 specifically includes: S21. Perform a Mask operation on the initial image to be shadowed to divide it into shadow and environment regions; select 6 first circular regions with a radius of 30px in the shadow region; for each first circular region, select 10% of the pixels and set their values ​​to 0; mark the first circular regions at the same positions in the initial image without shadows, and select the same pixels and set their values ​​to 0. S22. Select 16 second circular regions with a radius of 40px in the environment region. For each second circular region, select 15% of the pixels and set their values ​​to 0. Mark the second circular regions at the same positions in the initial shadowless image and select the same pixels and set their values ​​to 0. S23. Dilate and erode the initial image to be shadowed to divide it into boundary regions. Select 6 third circular regions with a radius of 20px in the boundary regions. For each third circular region, select 5% of the pixels and set their values ​​to 0. Mark the third circular regions at the same positions in the initial image without shadows and select the same pixels and set their values ​​to 0. S24. The marked initial image to be removed, the marked initial image without shadow, and the initial shadow mask image are scaled to 286×286 and then randomly cropped to obtain a cropped image to be removed with a size of 256×256, a cropped image without shadow with a size of 256×256, and a cropped shadow mask image with a size of 256×256. S25. Rotate and flip the image to be removed (without shadow), the image without shadow, and the image with the shadow mask, respectively, with a 50% probability, to obtain the image to be removed (I). SE Shadowless Image I SFE and shadow mask image M; S3. For each preprocessed image group, select the image I to be de-enhanced by removing shadows. SE The first image pair is formed by combining the shadow mask image M and the shadow-free enhanced image I. SFE The shadow mask image M forms a second image pair; S4. Separate the common input features of the first image pair and the second image pair to reconstruct the network, and obtain the reconstructed shadow image. Reconstructing shadowless images and preliminary shadow removal image I C ; Calculate self-reconstruction loss and roughness loss ; S5. Extract and reconstruct the shadow image Common feature E c_s and unique features E s_s Extract and reconstruct shadowless images Common feature E c_sf and unique features E sf_sf Based on these four features, comparative learning is performed, and the comparative learning loss is calculated. ; S6. Dilate the shadow mask image M using morphological operations to obtain the shadow region boundary mask image M. b and the prior mask image M of the local non-shaded region p ; S7. Remove initial shadows from image I C and shadow area boundary mask image M b Channel overlay yields a combined image, which is then input into a non-shadow information-guided network to obtain a refined shadowless image I. fine ; Calculate the refinement loss ; S8. Transform the image without shadows into I fine and the image to be removed I S The first sample pair is formed by combining the shadowless image I. SF and the image to be removed I S Form a second sample pair; input the first and second sample pairs into the discriminator to obtain the discrimination result; calculate the adversarial loss. ; S9. Self-reconstruction loss Roughness loss Comparative learning loss Detailed loss analysis and generating adversarial losses The total loss is obtained by weighted summation, and the model parameters are trained by backpropagation until the parameters converge and the training is complete. S10. Input the image to be processed into the trained image shadow removal model to obtain the shadow removal result.

2. The image shadow removal method based on contrastive learning according to claim 1, characterized in that, The formula for calculating the total loss L is: Self-reconstruction loss The calculation formula: Roughness loss The calculation formula: Detailed loss The calculation formula: Generate adversarial loss The calculation formula: Comparative learning loss The calculation formula: in, Here are the hyperparameters of the loss function, ||·||1 represents the L1 distance, and D represents the discriminator. Indicates for variables , and The expected value of ||·||2 represents the L2 distance, α represents a hyperparameter used to adjust the relative importance of positive and negative sample pairs, and m is a hyperparameter.

3. The image shadow removal method based on contrastive learning according to claim 1, characterized in that, The feature separation and reconstruction network includes encoder E S Encoder E C Encoder E SF Joint decoder J SR and joint decoder J S2SF ; in: Encoder E S Encoder E C and encoder E SF The structures are identical, each consisting of 6 convolutional layers, with cascaded BatchNorm and LeakyReLU layers added between every two adjacent convolutional layers; the parameters of each convolutional layer are set to a kernel size of 4, a stride of 2, and padding of 1; the number of input channels of the 6 convolutional layers are 4, 64, 128, 256, 512, and 512 respectively, and the number of output channels are 64, 128, 256, 512, 512, and 512 respectively; Joint Decoder J SR and joint decoder J S2SF The structures are identical, each consisting of 6 deconvolutional layers. LeakyReLU and BatchNorm layers are added before and after the first 5 deconvolutional layers, respectively, and LeakyReLU and tanh layers are added before and after the 6th deconvolutional layer, respectively. The parameters of each deconvolutional layer are set to a kernel size of 4, a stride of 2, and padding of 1. The number of input channels of the 6 deconvolutional layers are 1024, 2048, 2048, 1024, 512, and 256, respectively, and the number of output channels are 1024, 1024, 512, 256, 128, and 3, respectively.

4. The image shadow removal method based on contrastive learning according to claim 3, characterized in that, By separating the common input features of the first image pair and the second image pair, a network is reconstructed to obtain the preliminary shadow removal image I. C ,include: S41. Input the first image pair into encoder E respectively. S and encoder E C The characteristics of ambient light attenuation were obtained. and ambient light characteristics The second image pair is input into encoder E respectively. SF and encoder E C The characteristics of direct light were obtained. and ambient light characteristics ; S42. Characteristics of ambient light attenuation and ambient light characteristics The first joint feature is obtained by combining the features of direct light. and ambient light characteristics The combination yields the second joint feature; S43. Input the first joint feature into the joint decoder J respectively. SR and joint decoder J S2SF To obtain the reconstructed shadow image and preliminary shadow removal image I C ; S44. Input the second joint feature into the joint decoder J SR Obtain the reconstructed shadowless image .

5. The image shadow removal method based on contrastive learning according to claim 1, characterized in that, The non-shadow information guidance network includes an encoder and a decoder, wherein: The encoder consists of 6 convolutional layers. After the 1st to 4th convolutional layers, cascaded local region normalization modules and LeakyReLU layers are added respectively. After the 5th convolutional layer, cascaded BatchNorm layers and LeakyReLU layers are added respectively. The parameters of each convolutional layer are set as follows: kernel size 4, stride 2, padding 1. The number of input channels of these 6 convolutional layers are 4, 64, 128, 256, 512, and 512 in sequence, and the corresponding number of output channels are 64, 128, 256, 512, 512, and 512 in sequence. The decoder consists of 6 deconvolutional layers. A LeakyReLU layer and a BatchNorm layer are added before and after the first deconvolutional layer, respectively. A LeakyReLU layer and a local region normalization module are added before and after the second to fifth deconvolutional layers, respectively. A LeakyReLU layer and a tanh layer are added before and after the sixth deconvolutional layer, respectively. The parameters for each deconvolutional layer are set to a kernel size of 4, a stride of 2, and padding of 1. The number of input channels for these 6 deconvolutional layers are 512, 1024, 1024, 512, 256, and 128 in sequence, and the number of output channels are 512, 512, 256, 128, 64, and 3 in sequence.

6. The image shadow removal method based on contrastive learning according to claim 5, characterized in that, The local region normalization module's processing steps include: S15. Convert the intermediate feature image F output from the previous layer. in As input to the local region normalization module; S16. Transfer the intermediate feature image F in The input is fed into a 3×3 convolutional layer to obtain intermediate convolutional feature images; a shadow mask image M is used to guide the processing of the intermediate convolutional feature images to obtain shadow region features F. S and the feature of the shadowless area F SF ; S17. Features of the shaded area F S and the feature of the shadowless area F SF Perform region normalization to obtain normalized feature F Norm ; S18. Transfer the intermediate feature image F in Compared with the prior mask image M of the local non-shaded region p Multiplying them together yields the prior P for the locally unshaded region. color ; S19. Prioritize the local non-shaded areas P color The first and second convolutional modules are input respectively to learn the affine parameters γ and β; the first and second convolutional modules have the same structure, each including three 1×1 convolutional layers and two LeakyReLU layers; S20. Using affine parameters γ and β to normalize the feature F Norm Perform an affine transformation to obtain the output F of the local region normalization module. out .

Images