Image shadow removal method based on comparative learning

Abstract

The invention belongs to the technical field of image processing, and particularly relates to an image shadow removal method based on comparative learning, which comprises the following steps of: separating direct light and ambient light of a shadow image and a shadow-free image by using a feature separation reconstruction network, performing comparative learning on separated features by using a joint decoder, and reconstructing to obtain a preliminary shadow removal result; the method comprises the following steps: performing local region enhancement on an original input image, and performing paired random pixel extraction on a shadow image and a shadow-free image to replace original shadow input and shadow-free input; according to the method, the idea that the asymmetric encoder separates the ambient light and the direct light is utilized, learning of complex mapping is converted into learning of single mapping of the ambient light and the direct light, the robustness of the model is enhanced and improved by using the local region, the boundary region processing capability is optimized, the uncertainty of a real scene is simulated, and the robustness of the model is improved. Therefore, a high-quality shadow-free image is obtained.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, in particular to image shadow removal, and specifically relates to an image shadow removal method based on contrast learning. Background Art

[0002] Shadows are dark areas formed when light is blocked or scattered when it encounters an object, and their appearance and perception are affected by brightness and color. Shadows in images or videos will inevitably have a negative impact on the performance of downstream computer vision tasks, such as image decomposition, visual recognition, object tracking, and trajectory prediction. Therefore, shadow removal, as an important topic in image preprocessing, has been widely studied, and it still faces many challenges in complex scenes.

[0003] Current shadow removal methods can be roughly divided into two categories: traditional methods based on image priors and methods based on deep learning. Early traditional methods relied on physical imaging models and performed shadow detection through image gradients, image intensity, and color transfer to generate shadow-free images. In recent years, with the release of large-scale shadow removal datasets, shadow removal methods based on deep learning have surpassed traditional methods in performance and achieved remarkable results. For example, DSC uses direction-aware spatial contextual attention features for shadow detection and removal; Fu et al. regard shadow areas as underexposed areas and achieve shadow removal by learning multiple overexposed images and adaptively fusing them; BMNet adopts a reversible framework to perform shadow removal through the reverse mapping of shadow generation; SG-ShadowNet regards shadow removal as a style transfer problem and uses the color prior of non-shadow areas to ensure the overall color coordination of the shadow removal results.

[0004] However, these methods ignore the basic fact of shadow formation: pixel values ​​are determined by both illumination and reflectivity. They require a single network to simultaneously learn the complex mapping relationship between direct light and ambient light, but since the information of these two feature domains is intertwined, it is difficult for a single network to take into account the restoration of both direct light and ambient light. Although Le et al. combined deep learning with a simplified shadow illumination model and used parameter estimation to illuminate the shadow area, their illumination model is linear, while actual shadow formation is a nonlinear process, and the network does not distinguish between direct light and ambient light, resulting in limited shadow removal effects.

[0005] Although there are methods to propose feature separation models, the existing feature separation models do not impose specific constraints on the separated features, resulting in incomplete feature separation. Shadow images themselves have shadow areas and ambient areas. Feature separation of shadow images should obtain shadow features and non-shadow features. However, the previous feature separation models did not constrain the separated features, resulting in the existence of non-shadow features in the separated shadow features, and shadow features in the non-shadow features. The separation is not complete, so that the image after removing the shadow has a visual color difference from the real shadow-free image. Summary of the invention

[0006] To solve the above problems, the present invention provides an image shadow removal method based on contrastive learning. Contrastive learning is added on the basis of feature separation, which alleviates the artifact and chromatic aberration problems existing in the preliminary removal results to a certain extent and obtains a high-quality shadow-free image.

[0007] The specific plan includes the following steps:

[0008] S1. Construct an image shadow removal model, which includes a feature separation and reconstruction module, a non-shadow information guidance module and a discriminator; collect multiple image groups to form training data, each image group includes an initial shadow image to be removed, an initial shadow-free image and an initial shadow mask image acquired in the same scene;

[0009] S2. Perform local area enhancement on each image group to obtain a corresponding pre-processed image group; a pre-processed image group includes an image I to be shadowed. SE , shadow-free image I SFE and shadow mask image M;

[0010] S3. For each pre-processed image group, the enhanced shadow image I to be removed SE and the shadow mask image M form the first image pair, and the shadow-free enhanced image I SFE and the shadow mask image M form a second image pair;

[0011] S4. Input the first image pair and the second image pair into the feature separation and reconstruction network to obtain the reconstructed shadow image I′ S , reconstruct the shadow-free image I S ' F and the preliminary shadow removal image I C ; Calculate the self-reconstruction loss and roughness loss

[0012] S5. Extract and reconstruct shadow image I′ S The common feature E c_s and unique feature E s_s , extract and reconstruct the shadow-free image I S 'F The common feature E c_sf and unique feature E sf_sf ; Perform contrastive learning based on these four features and calculate contrastive learning loss

[0013] S6. Use morphological operations to dilate the shadow mask image M to obtain the shadow area boundary mask image M b and the local non-shadow region prior mask image M p ;

[0014] S7. Remove the preliminary shadow image I C and shadow region boundary mask image M b The combined image is obtained by channel superposition, and the combined image is input into the non-shadow information guidance network to obtain the refined shadow-free image I fine ; Calculate refinement loss

[0015] S8. The shadow-free image I fine and the shadow image I to be removed S The first sample pair is composed of the shadow-free image I SF and the shadow image I to be removed S Form a second sample pair; input the first sample pair and the second sample pair into the discriminator to obtain the discrimination result; calculate the generated adversarial loss

[0016] S9. Will self-reconstruct the loss Roughness loss Contrastive Learning Loss Refinement loss and Generative Adversarial Loss Perform weighted summation to obtain the total loss, and back-propagate the training model parameters until the parameters converge to complete the training;

[0017] S10. Input the image to be processed into the trained image shadow removal model to obtain the shadow removal result.

[0018] Beneficial effects of the present invention:

[0019] Starting from the perspective of separating direct light and ambient light, the present invention simplifies the shadow removal task from the complex mapping problem of learning shadows to no shadows to a single mapping problem of learning direct light and ambient light. In the network design, the feature separation and reconstruction network adaptively decomposes the image into direct light and ambient light features in the separation and reconstruction stages, and introduces a contrast learning mechanism based on feature separation, so that the color of the removal result is closer to the human eye perception of the real shadow-free image. Through contrast learning, the model can better separate shadow features from non-shadow features and reduce shadow residues caused by feature confusion. Local contrast learning can enhance the model's ability to reconstruct details of shadow boundaries and avoid artifacts and color distortion. By constraining the feature space, contrast learning enables the model to better adapt to lighting changes and material diversity in complex scenes.

[0020] In addition, the present invention adopts local area enhancement technology to improve the generalization and robustness of the model. Shadow boundaries are usually transition areas where color and illumination change, and traditional global optimization methods are prone to produce artifacts or blurs here. Local area enhancement can restore boundary details more accurately through targeted training and attention mechanisms. The color restoration of the shadow area needs to be consistent with the surrounding non-shadow areas, but the global method may cause color difference due to ignoring local statistical characteristics. The contribution of local area enhancement is: through local normalization or color transfer strategies, the local color prior of the non-shadow area is used to guide the restoration of the shadow area and force the shadow removal result to align with the color distribution of the real shadow-free image in the local area.

[0021] Through extensive experiments on public datasets, the present invention demonstrates superior shadow removal performance compared with other advanced methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a flow chart of the method of the present invention;

[0023] Figure 2 It is a network model framework diagram of the present invention;

[0024] Figure 3 It is a structural schematic diagram of the feature separation and reconstruction module of the present invention;

[0025] Figure 4 It is a structural schematic diagram of a local area normalization module of the present invention;

[0026] Figure 5 This is a comparison chart of the visualization effect of the present invention and other methods on the ISTD+ and SRD test sets;

[0027] Figure 6 This is a quantitative comparison result diagram of the present invention and other methods on the SRD test set. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] The present invention provides an image shadow removal method based on contrast learning. Figure 1-3 As shown, the following steps are included:

[0030] S1. Construct an image shadow removal model, which includes a feature separation and reconstruction module, a non-shadow information guidance module and a discriminator; collect multiple image groups to form training data, each image group includes an initial shadow image to be removed, an initial shadow-free image and an initial shadow mask image acquired in the same scene;

[0031] S2. Perform local area enhancement on each image group to obtain a corresponding pre-processed image group; a pre-processed image group includes an image I to be shadowed. SE , shadow-free image I SFE and shadow mask image M;

[0032] S3. For each pre-processed image group, the enhanced shadow image I to be removed SE and the shadow mask image M form the first image pair, and the shadow-free enhanced image I SFE and the shadow mask image M form a second image pair;

[0033] S4. Input the first image pair and the second image pair into the feature separation and reconstruction network to obtain the reconstructed shadow image I′ S , reconstruct the shadow-free image I S ' F and the preliminary shadow removal image I C ; Calculate the self-reconstruction loss and roughness loss

[0034] S5. Extract and reconstruct shadow image I′ S The common feature E c_s and unique feature E s_s , extract and reconstruct the shadow-free image I S ' F The common feature E c_sf and unique feature E sf_sf ; Perform contrastive learning based on these four features and calculate contrastive learning loss

[0035] Specifically, contrastive learning is a self-supervised learning method, and its core idea is to construct a discriminative feature space by learning the similarities and differences between samples. Specifically, contrastive learning maximizes the similarity of positive sample pairs (similar samples) while minimizing the similarity of negative sample pairs (dissimilar samples), so that the model can learn more discriminative feature representations. In the shadow removal task, contrastive learning can be used to constrain the feature space so that the feature representation of shadow images and non-shadow images is more reasonable, thereby improving the effect of shadow removal. The key to shadow removal is to decompose the image into two parts: direct light (non-shadow area) and ambient light (shadow area). Through contrastive learning, the following constraints can be designed:

[0036] Positive sample pair: The common features (such as material, texture, etc.) of the shadow image and the non-shadow image should be as close as possible.

[0037] Negative sample pairs: The shadow features of the shadow image (i.e., the unique features of the shadow image) and the non-shadow features of the non-shadow image should be as far away as possible.

[0038] Minimize the distance between positive sample pairs and maximize the distance between negative sample pairs. Through this constraint, the model can better separate shadow features and non-shadow features, thereby improving the accuracy of shadow removal.

[0039] Contrastive learning does not require a large amount of labeled data, and can generate positive sample pairs through data enhancement, which is suitable for scenarios where data is scarce. Through contrastive learning, the model can learn more discriminative feature representations, which are suitable for tasks such as classification and retrieval. Contrastive learning is highly robust to noise and changes in data distribution.

[0040] S6. Use morphological operations to dilate the shadow mask image M to obtain the shadow area boundary mask image M b and the local non-shadow region prior mask image M p ;

[0041] S7. Remove the preliminary shadow image I C and shadow region boundary mask image M b The combined image is obtained by channel superposition, and the combined image is input into the non-shadow information guidance network to obtain the refined shadow-free image I fine ; Calculate refinement loss

[0042] S8. The shadow-free image I fine and the shadow image I to be removed S The first sample pair is composed of the shadow-free image I SF and the shadow image I to be removed SForm a second sample pair; input the first sample pair and the second sample pair into the discriminator to obtain the discrimination result; calculate the generated adversarial loss

[0043] S9. Will self-reconstruct losses Roughness loss Contrastive Learning Loss Refinement loss and Generative Adversarial Loss Perform weighted summation to obtain the total loss, and back-propagate the training model parameters until the parameters converge to complete the training;

[0044] S10. Input the image to be processed into the trained image shadow removal model to obtain the shadow removal result.

[0045] Specifically, in order to strengthen the model's attention to and learning of local features and improve the detail recovery ability and visual consistency of shadow removal, the present invention proposes a local area enhancement method in the data processing part. Step S2 performs local area enhancement on each image group, which specifically includes:

[0046] S21. Perform a mask operation on the initial image to be shadowed to be removed to divide the shadow area and the environment area; select 6 first circular areas with a radius of 30px in the shadow area; for each first circular area, select 10% of the pixels and set the value to 0; mark the first circular area at the same position of the initial image without shadow, and select the same pixel points and set the value to 0;

[0047] S22. Select 16 second circular areas with a radius of 40px in the environment area, and for each second circular area, select 15% of the pixels therein and set the values ​​to 0; mark the second circular area at the same position of the initial shadow-free image, and select the same pixels and set the values ​​to 0;

[0048] S23. Dilate and erode the initial image to be shadowed to be removed to divide the boundary area, select 6 third circular areas with a radius of 20px in the boundary area; for each third circular area, select 5% of the pixels and set the value to 0; mark the third circular area at the same position of the initial image without shadows, and select the same pixels and set the value to 0; so far, the initial image to be shadowed to be removed and the initial image without shadows are processed in the same way so that the two correspond.

[0049] S24. Scale the marked initial shadow-to-be-removed image, the marked initial shadow-free image, and the initial shadow mask image to 286×286 and randomly crop them to obtain a cropped shadow-to-be-removed image with a size of 256×256, a cropped shadow-free image with a size of 256×256, and a cropped shadow mask image with a size of 256×256;

[0050] S25. The cropped shadow-removed image, the cropped shadow-free image, and the cropped shadow mask image are rotated and flipped with a probability of 50% to obtain the shadow-removed image I SE , shadow-free image I SFE and the shadow mask image M.

[0051] Specifically, during the model training phase, the Adam optimizer with an initial learning rate of 0.0002 was used, and the ReduceLROnPlateau strategy was used to decay the learning rate. At the same time, a total loss function was constructed, and the parameters of the SGGAN shadow removal network were updated through the back-propagation mechanism to complete the training of the SGGAN shadow removal network. The total loss function constructed The calculation formula is:

[0052]

[0053] Among them, λ 1 ~λ 5 are the loss function hyperparameters, and the above hyperparameters are set to 10, 10, 10, 0.1, 0.1 respectively.

[0054] The various loss functions are described as follows:

[0055] 1) Self-reconstruction loss The self-reconstruction loss encourages different encoders to encode different features by optimizing the L1 distance between the self-reconstruction decoder output and the two inputs:

[0056]

[0057] ||·|| 1 Indicates L 1 distance.

[0058] 2) Roughness loss The rough loss is used to optimize the L1 distance between the preliminary removal result and the true shadow-free image, which can be expressed as:

[0059]

[0060] 3) Refinement loss The refinement loss is used to optimize the L1 distance between the refinement result and the true shadow-free image, which can be expressed as:

[0061]

[0062] 4) Generate adversarial loss The image pairs are input into the discriminator for identification. The image pairs containing shadow removal results are judged as false, and the image pairs containing real shadow-free images are judged as true. In the process of adversarial training, the generator is prompted to generate more realistic shadow-free images. The adversarial loss can be expressed as:

[0063]

[0064] D represents the discriminator, For variable I S ,I SF and I fine expected value.

[0065] 5) Contrastive Learning Loss The shadow-free image I will be reconstructed S ' F The common feature E c_sf As a positive sample, the shadow image I′ will be reconstructed S Unique features of E s_s As a negative sample, let the reconstructed shadow image I′ S The common feature E c_s Close to positive samples and away from negative samples. The formula can be expressed as:

[0066]

[0067] α represents a hyperparameter used to adjust the relative importance of positive sample pairs and negative sample pairs; m is a hyperparameter used to control the minimum distance between negative sample pairs to avoid feature collapse caused by over-optimization of the model.

[0068] Specifically, the feature separation and reconstruction network includes an encoder E S 、Encoder E C 、Encoder E SF , Joint Decoder J SR and joint decoder J S2SF ;in:

[0069] Encoder E S 、Encoder E C and encoder E SF The structures are the same, all consisting of 6 convolutional layers, and a cascaded BatchNorm layer and a LeakyRelu layer are added between every two adjacent convolutional layers; the parameters of each convolutional layer are set to a convolution kernel size of 4, a stride of 2, and a padding of 1; the number of input channels of the 6 convolutional layers is 4, 64, 128, 256, 512, and 512, respectively, and the number of output channels is 64, 128, 256, 512, 512, and 512, respectively;

[0070] Joint Decoder J SR and joint decoder JS2SF The structures are the same, consisting of 6 deconvolution layers. LeakyRelu layers and BatchNorm layers are added before and after the 1st to 5th deconvolution layers, respectively, and LeakyRelu layers and tanh layers are added before and after the 6th deconvolution layer, respectively. The parameters of each deconvolution layer are set to a convolution kernel size of 4, a stride of 2, and a padding of 1. The number of input channels of the 6 deconvolution layers are 1024, 2048, 2048, 1024, 512, and 256, respectively, and the number of output channels are 1024, 1024, 512, 256, 128, and 3, respectively.

[0071] Specifically, in the existing shadow removal methods, the network needs to learn the complex mapping of ambient light and direct light at the same time during the process of removing shadows. Such a network has a bottleneck because the network cannot take into account the restoration of ambient light and direct light at the same time. To solve this problem, the present invention constructs a feature separation and reconstruction network based on the shadow mapping relationship in the illumination model. From the perspective of separating ambient light and direct light, the characteristics of ambient light and direct light are learned through the reconstruction process, and the complex mapping process of learning shadows to no shadows is converted into two simple mapping problems of learning ambient light and direct light, thereby improving the shadow removal effect. Among them, usually in the illumination model, the relationship between the shadowed image and the shadowless image can be expressed by the following formula:

[0072]

[0073] in, Indicates the value of the i-th pixel of the shadow-free image at wavelength λ, R i (λ) represents the reflectivity of the i-th pixel at wavelength λ, represents the ambient light of the i-th pixel at wavelength λ, represents the direct light of the i-th pixel at wavelength λ, It means that the i-th pixel point obtains the i-th pixel value of the shadow image under the wavelength λ. represents the ambient light attenuation of the i-th pixel under wavelength λ; Based on the above analysis, the first image pair and the second image pair are input into the feature separation and reconstruction network, such as Figure 3 As shown, including:

[0074] S41. Input the first image pair into the encoder E respectively S and encoder E C , get the ambient light attenuation characteristics and ambient light characteristics The second image pair is input into the encoder E SF and encoder E C , get the direct light characteristics and ambient light characteristics

[0075] S42. The ambient light attenuation characteristic and ambient light characteristics Combine the first joint feature and convert the direct light feature and ambient light characteristics Combining to obtain a second joint feature;

[0076] S43. Input the first joint features into the joint decoder J SR and joint decoder J S2SF , and obtain the reconstructed shadow image and the preliminary shadow removal image I C ;

[0077] S44. Input the second joint feature into the joint decoder J SR Get the reconstructed shadow-free image

[0078] Specifically, the non-shadow information guided network includes an encoder and a decoder, wherein:

[0079] The encoder consists of 6 convolutional layers, in which the 1st to 4th convolutional layers are followed by cascaded local area normalization modules and LeakyRelu layers, respectively, and the 5th convolutional layer is followed by cascaded BatchNorm and LeakyRelu layers. For each convolutional layer, the parameters are set as follows: the convolution kernel size is 4, the stride is 2, and the padding is 1. The number of input channels of these 6 convolutional layers is 4, 64, 128, 256, 512, and 512 in order, and the corresponding number of output channels is 64, 128, 256, 512, 512, and 512 in order.

[0080] The decoder consists of 6 deconvolution layers, wherein the first deconvolution layer is preceded and followed by a LeakyRelu layer and a BatchNorm layer, the second to fifth deconvolution layers are preceded and followed by a LeakyRelu layer and a local area normalization module, and the sixth deconvolution layer is preceded and followed by a LeakyRelu layer and a tanh layer. For each deconvolution layer, the parameters are set as follows: the convolution kernel size is 4, the stride is 2, and the padding is 1. The number of input channels of the 6 deconvolution layers is 512, 1024, 1024, 512, 256, and 128, respectively, and the number of output channels is 512, 512, 256, 128, 64, and 3, respectively.

[0081] Specifically, Figure 4 As shown in FIG. 1 , the processing process of the local area normalization module includes:

[0082] S15. The intermediate feature image F output by the previous layer inAs the input of the local area normalization module;

[0083] S16. The intermediate feature image F in Input into a 3×3 convolution layer to obtain the intermediate convolution feature image; use the shadow mask image M to guide the intermediate convolution feature image to obtain the shadow area feature F S and the unshaded area feature F SF ;

[0084] S17. Shadow area feature F S and the unshaded area feature F SF Perform regional normalization to obtain the normalized feature F Norm ;

[0085] S18. The intermediate feature image F in and the local non-shadow area prior mask image M p Multiply them together to get the local non-shadow area prior P color ;

[0086] S19. The local non-shadow area prior P color The first convolution module and the second convolution module are input respectively to learn the affine parameters γ and β; the first convolution module and the second convolution module have the same structure, both including 3 1×1 convolution layers and 2 LeakyRelu layers;

[0087] S20. Use affine parameters γ and β to normalize the feature F Norm Perform affine transformation to obtain the output F of the local area normalization module out .

[0088] See also Figure 5 and Figure 6 Compared with other methods, the method of the present invention has better shadow removal performance in various scenes, the color difference inside and outside the shadow area of ​​the removal result is well controlled and the artifacts are effectively reduced.

[0089] In one embodiment, the model of the present invention is compared with the model in patent CN2024101883208 in terms of visual perception on the ISTD+ test set, which involves a comparison of two indicators, CIEDE2000 and LPIPS. CIEDE2000, as an improved version of the CIELAB color space, is a standard for quantifying color differences and can evaluate color differences perceived by the human eye. LPIPS is an image similarity measurement method based on deep learning. LPIPS extracts image features through a pre-trained deep neural network and calculates the distance in the feature space, thereby better reflecting the human eye's perception of image quality. The results are shown in Table 1. The present invention can remove shadows more effectively and solve the problem that the shadow-removed image obtained by the existing model has visual color difference from the real shadow-free image.

[0090] Table 1 Comparison results

[0091]

[0092] By utilizing the present invention, the ambient light and direct light of shadow images and shadow-free images can be effectively separated, thereby improving the overall performance of the network. At the same time, due to the reasonable introduction of non-shadow priors, the color of the shadow area is accurately restored, thereby achieving better shadow removal performance.

[0093] In the present invention, unless otherwise clearly stipulated and limited, the terms such as "installation", "setting", "connection", "fixation" and "rotation" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral one; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. Unless otherwise clearly defined, ordinary technicians in this field can understand the specific meanings of the above terms in the present invention according to the specific circumstances.

[0094] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for removing image shadows based on contrastive learning, characterized in that: The following steps are involved: S1. Construct an image shadow removal model, which includes a feature separation and reconstruction module, a non-shadow information guidance module and a discriminator; collect multiple image groups to form training data, each image group includes an initial shadow image to be removed, an initial shadow-free image and an initial shadow mask image acquired in the same scene; S2. Perform local area enhancement on each image group to obtain a corresponding pre-processed image group; a pre-processed image group includes an image I to be shadowed. SE , shadow-free image I SFE and shadow mask image M; S3. For each pre-processed image group, the enhanced shadow image I to be removed SE and the shadow mask image M form the first image pair, and the shadow-free enhanced image I SFE and the shadow mask image M form a second image pair; S4. Input the first image pair and the second image pair into the feature separation and reconstruction network to obtain the reconstructed shadow image I′ S , reconstruct the shadow-free image I S ' F and the preliminary shadow removal image I C ; Calculate the self-reconstruction loss and roughness loss S5. Extract and reconstruct shadow image I′ S The common feature E c_s and unique feature E s_s , extract and reconstruct the shadow-free image I S ' F The common feature E c_sf and unique feature E sf_sf ; Perform contrastive learning based on these four features and calculate contrastive learning loss S6. Use morphological operations to dilate the shadow mask image M to obtain the shadow area boundary mask image M b and the local non-shadow region prior mask image M p ; S7. Remove the preliminary shadow image I C and shadow region boundary mask image M b The combined image is obtained by channel superposition, and the combined image is input into the non-shadow information guidance network to obtain the refined shadow-free image I fine ; Calculate refinement loss S8. The shadow-free image I fine and the shadow image I to be removed S The first sample pair is composed of the shadow-free image I SF and the shadow image I to be removed S Form a second sample pair; input the first sample pair and the second sample pair into the discriminator to obtain the discrimination result; calculate the generated adversarial loss S9. Will self-reconstruct the loss Roughness loss Contrastive Learning Loss Refinement loss and Generative Adversarial Loss Perform weighted summation to obtain the total loss, and back-propagate the training model parameters until the parameters converge to complete the training; S10. Input the image to be processed into the trained image shadow removal model to obtain the shadow removal result.

2. The image shadow removal method based on contrast learning according to claim 1, characterized in that: Step S2 specifically includes: S21. Perform a mask operation on the initial image to be shadowed to be removed to divide the shadow area and the environment area; select 6 first circular areas with a radius of 30px in the shadow area; for each first circular area, select 10% of the pixels and set the value to 0; mark the first circular area at the same position of the initial image without shadow, and select the same pixel points and set the value to 0; S22. Select 16 second circular areas with a radius of 40px in the environment area, and for each second circular area, select 15% of the pixels therein and set the values ​​to 0; mark the second circular area at the same position of the initial shadow-free image, and select the same pixels and set the values ​​to 0; S23. Dilate and erode the initial image to be shadow-removed to divide the boundary area, select 6 third circular areas with a radius of 20px in the boundary area; for each third circular area, select 5% of the pixels and set the value to 0; mark the third circular area at the same position of the initial shadow-free image, and select the same pixel points and set the value to 0; S24. Scale the marked initial shadow-to-be-removed image, the marked initial shadow-free image, and the initial shadow mask image to 286×286 and randomly crop them to obtain a cropped shadow-to-be-removed image with a size of 256×256, a cropped shadow-free image with a size of 256×256, and a cropped shadow mask image with a size of 256×256; S25. The cropped shadow-removed image, the cropped shadow-free image, and the cropped shadow mask image are rotated and flipped with a probability of 50% to obtain the shadow-removed image I SE , shadow-free image I SFE and the shadow mask image M.

3. The image shadow removal method based on contrast learning according to claim 1, characterized in that: Total loss The calculation formula is: Self-reconstruction loss The calculation formula is: Roughness loss The calculation formula is: Refinement loss The calculation formula is: Generative Adversarial Loss The calculation formula is: Contrastive Learning Loss The calculation formula is: Among them, λ1~λ5 are loss function hyperparameters, ||·||1 represents L1 distance, D represents the discriminator, For variable I S ,I SF and I fine , ||·||2 represents the L2 distance, α represents a hyperparameter used to adjust the relative importance of positive sample pairs and negative sample pairs, and m is a hyperparameter.

4. The image shadow removal method based on contrast learning according to claim 1, characterized in that: The feature separation and reconstruction network includes an encoder E S 、Encoder E C 、Encoder E SF , Joint Decoder J SR and joint decoder J S2SF ; in: Encoder E S 、Encoder E C and encoder E SF The structures are the same, all consisting of 6 convolutional layers, and a cascaded BatchNorm layer and a LeakyRelu layer are added between every two adjacent convolutional layers; the parameters of each convolutional layer are set to a convolution kernel size of 4, a stride of 2, and a padding of 1; the number of input channels of the 6 convolutional layers is 4, 64, 128, 256, 512, and 512, respectively, and the number of output channels is 64, 128, 256, 512, 512, and 512, respectively; Joint Decoder J SR and joint decoder J S2SF The structures are the same, consisting of 6 deconvolution layers. LeakyRelu layers and BatchNorm layers are added before and after the 1st to 5th deconvolution layers, respectively, and LeakyRelu layers and tanh layers are added before and after the 6th deconvolution layer, respectively. The parameters of each deconvolution layer are set to a convolution kernel size of 4, a stride of 2, and a padding of 1. The number of input channels of the 6 deconvolution layers are 1024, 2048, 2048, 1024, 512, and 256, respectively, and the number of output channels are 1024, 1024, 512, 256, 128, and 3, respectively.

5. The image shadow removal method based on contrast learning according to claim 4 is characterized in that: The first image pair and the second image pair are input into the feature separation and reconstruction network to obtain the preliminary shadow removal image I C ,include: S41. Input the first image pair into the encoder E respectively S and encoder E C , get the ambient light attenuation characteristics and ambient light characteristics The second image pair is input into the encoder E SF and encoder E C , get the direct light characteristics and ambient light characteristics S42. The ambient light attenuation characteristic and ambient light characteristics Combine the first joint feature and convert the direct light feature and ambient light characteristics Combining to obtain a second joint feature; S43. Input the first joint features into the joint decoder J SR and joint decoder J S2SF , and obtain the reconstructed shadow image and the preliminary shadow removal image I C ; S44. Input the second joint feature into the joint decoder J SR Get the reconstructed shadow-free image 6. The image shadow removal method based on contrast learning according to claim 1, characterized in that: The non-shadow information guided network includes an encoder and a decoder, wherein: The encoder consists of 6 convolutional layers, wherein the cascaded local area normalization module and the LeakyRelu layer are added after the 1st to 4th convolutional layers, respectively, and the cascaded BatchNorm layer and the LeakyRelu layer are added after the 5th convolutional layer; the parameters of each convolutional layer are set to a convolution kernel size of 4, a stride of 2, and a padding of 1; the number of input channels of the 6 convolutional layers is 4, 64, 128, 256, 512, and 512 in order, and the corresponding number of output channels is 64, 128, 256, 512, 512, and 512 in order; The decoder consists of 6 deconvolution layers, wherein a LeakyRelu layer and a BatchNorm layer are added before and after the first deconvolution layer, respectively; a LeakyRelu layer and a local area normalization module are added before and after the second to fifth deconvolution layers, respectively; a LeakyRelu layer and a tanh layer are added before and after the sixth deconvolution layer, respectively; the parameters of each deconvolution layer are set to a convolution kernel size of 4, a stride of 2, and a padding of 1; the number of input channels of the 6 deconvolution layers is 512, 1024, 1024, 512, 256, and 128, respectively, and the number of output channels is 512, 512, 256, 128, 64, and 3, respectively.

7. The image shadow removal method based on contrast learning according to claim 6 is characterized in that: The processing of the local area normalization module includes: S15. The intermediate feature image F output by the previous layer in As the input of the local area normalization module; S16. The intermediate feature image F in Input into a 3×3 convolution layer to obtain the intermediate convolution feature image; use the shadow mask image M to guide the intermediate convolution feature image to obtain the shadow area feature F S and the unshaded area feature F SF ; S17. Shadow area feature F S and the unshaded area feature F SF Perform regional normalization to obtain the normalized feature F Norm ; S18. The intermediate feature image F in and the local non-shadow area prior mask image M p Multiply them together to get the local non-shadow area prior P color ; S19. The local non-shadow area prior P color The first convolution module and the second convolution module are input respectively to learn the affine parameters γ and β; the first convolution module and the second convolution module have the same structure, both including 3 1×1 convolution layers and 2 LeakyRelu layers; S20. Use affine parameters γ and β to normalize the feature F Norm Perform affine transformation to obtain the output F of the local area normalization module out .

Images