A shadow removal method based on deep learning and reflectivity

Abstract

The present invention discloses a shadow removal method based on deep learning and reflectivity. The steps include: 1) converting the selected colored shadow image into a grayscale image and a color image respectively and performing shadow annotation; setting the supervised image as the ratio image between the grayscale image of the shadow image and the grayscale image of the corresponding shadowless image; 2) training a U-shaped deep neural network using training data; 3) for an image A to be processed, calculating the reflectivity of each pixel in the shadow area of image A; 4) calculating the color presented by each pixel in the shadow area of image A under white light illumination according to the reflectivity, and using the obtained color as the color of the corresponding pixel after shadow removal to generate a color image B after shadow removal of image A; 5) inputting the color image of image A into the U-shaped deep neural network to generate a ratio image P A ; then according to image P A generate the grayscale image of image B, and then combine with image B to generate the corresponding shadow removal image of image A.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and relates to a method capable of removing shadow regions in color digital images. In particular, it relates to a shadow removal method based on deep learning and reflectivity. Background Art

[0002] With the improvement of computer performance and the popularization of digital imaging devices, as well as the rapid development of information technology in recent years, the two major fields of computer vision and graphic image processing for digital images have received increasing attention from all sectors of society. A shadow is generated when a part of the light source in a scene fails to irradiate a certain area due to mutual occlusion between objects or between different parts of an object itself. As a common phenomenon in nature, shadow regions often appear in digital images. The existence of shadow regions has both advantages and disadvantages for computer vision tasks and image processing tasks. The advantages lie in that the features of the shadow regions in the image can provide important clues for estimating the geometric features of the scene, the position of the occluded light source relative to the object, object motion, and camera calibration. The disadvantages are that shadow regions have a greater impact on some common computer vision tasks and image processing tasks. Since shadow regions damage the clarity and integrity of the target object in the image, it may lead to the target object not being completely segmented or misrecognized, reducing the performance of tasks including image segmentation and object detection; for computer vision tasks that use image sequences for calculation such as object tracking, the impact of shadow regions is even more obvious. If the appearance of the target object changes between different image frames due to shadow regions, it will lead to situations such as target loss. Therefore, designing an accurate and highly robust shadow removal method has very broad application value in many practical applications.

[0003] The existing shadow removal methods are mainly divided into two categories, namely the traditional shadow removal method based on region pairing and the deep shadow removal method based on deep neural network. The basic idea of the traditional shadow removal method based on region pairing is as follows: First, the input image is segmented according to color and local texture to ensure the consistency of materials in each segmented region as much as possible; then, a method is designed to find non-shadow regions with the same material as the shadow region in the image; finally, the brightness and average color of the non-shadow region are used to update the corresponding information of the shadow region to remove the shadow region. Since it is difficult to find paired regions for all shadow regions, and in some cases, there are no paired regions for some shadow regions in the image, the deep shadow removal method has occupied the mainstream in recent years. The shadow removal method based on deep neural network usually uses a U-shaped network as the basic architecture to generate shadow removal results. In order to improve the ability of the neural network to retain details in the original image as much as possible, a cross-layer connection structure is usually used. The training strategies of the deep neural network can be divided into three categories, namely the training strategy based on shadow-free images, the training strategy based on discriminant networks, and the training strategy based on intermediate results. The training strategy based on shadow-free images first uses the deep neural network to generate a result image according to the input image, and then corrects the deep neural network through gradient backpropagation using the difference between the generated result image and the standard shadow-free image. The training strategy based on discriminant networks uses the training strategy of the Generative Adversarial Network (GAN), designs a deep discriminant network to judge whether there are still shadow regions in the result image generated by the shadow removal network, and then updates the parameters of the main network according to the results of the discriminant network. This training method does not require a shadow-free image paired with the input image as supervised data. After simply labeling the training image as to whether there are shadow regions, the discriminant network is first trained, and then the discriminant network and the main network are combined for end-to-end training. The training strategy based on intermediate results calculates the shadow-free image according to the formula and needs to use the deep neural network to regress relevant parameters. In order to obtain better results, usually after calculating the shadow-free image according to the parameter values obtained by regression, a U-shaped network is added for result refinement, and the U-shaped network and the parameter regression network are jointly trained end-to-end.

[0004] Although existing shadow removal methods have obtained high evaluation parameters on the evaluation dataset, they still face many problems, which are mainly reflected in the following aspects. First, it is difficult to obtain training data. The existing methods for obtaining shadow-free training data mostly adopt the following scheme: fix the position, shooting direction and exposure parameters of the camera, and point to a scene without shadow areas. First, use artificial occlusion to create a shadow area in the shooting area and capture images with shadow areas; then remove the occluder and capture shadow-free images. Since the subsequent white balance algorithm of digital cameras is related to the captured scene, the disappearance of the shadow area in the captured scene may cause the image signal processor (ISP) of the camera to generate different white balance parameters from those when capturing shadow images, resulting in inconsistent colors in the non-shadow areas of the two images. Second, the non-shadow areas in the image are easily changed. Existing shadow removal methods mostly use U-shaped deep neural networks to generate the final shadow removal results. This network structure can directly generate an image with the same size as the input image. However, since all pixels of the newly generated image are regenerated, there is a high possibility that the pixels in the non-shadow areas that should not change in the shadow removal task will change.

[0005] In addition, existing shadow removal methods mainly focus on learning statistical laws from training data, and rarely study the physical characteristics of the shadow areas themselves, resulting in these methods being relatively dependent on training data. However, in practical applications, the sources of images are diverse, and it is difficult to ensure the consistency of the data distribution between training data and test data. Therefore, improving the robustness of shadow removal methods to data from different sources is a research hotspot for practical applications. Summary of the Invention

[0006] Aiming at the shadow areas in a single color digital image, the present invention proposes a shadow removal method based on deep learning and reflectivity. Since directly using a deep neural network to generate the shadow removal result easily causes changes in the non-shadow areas of the image, the present invention uses a deep neural network to generate intermediate results required for calculating the shadow-free image. At the same time, in order to reduce the influence of training data on the calculation results, according to the illumination model of the image, the illumination conditions of the scene and the spectral reflectivity of the shadow area are introduced to calculate the color of the shadow pixels after shadow removal.

[0007] The technical solution of the present invention is as follows:

[0008] A shadow removal method based on deep learning and reflectivity, the steps of which include:

[0009] 1) Convert the colored shadow image into two images, namely a grayscale image and a color image.

[0010] 2) Set the supervised image in the training data as the ratio image between the grayscale image of the shadow image and the grayscale image of the shadowless image.

[0011] 3) Randomly rotate, symmetrically transform, and change the aspect ratio of the grayscale image of the shadow image, the corresponding shadow region annotation image, and the supervised image to augment the training data.

[0012] 4) Use a U-shaped deep neural network with cross-layer connection structure and train it with the augmented training data to learn how to generate a ratio image using the grayscale image and the shadow region annotation image.

[0013] 5) By improving the existing methods for calculating the scene illumination condition and the object spectral reflectance, calculate the illumination condition of the shadow region based on the color image of the test data, and then calculate the reflectance of each pixel in the shadow region according to the illumination condition.

[0014] 6) Calculate the color that the shadow pixel presents under standard white light illumination based on the reflectance of the shadow region, and use this color as the color of the shadow pixel after shadow removal to generate the color image of the image after shadow removal.

[0015] 7) Use the ratio image generated by the deep neural network trained in step 4) to calculate the grayscale image of the test image after shadow removal, and then combine the grayscale image and the color image to generate the final image after shadow removal.

[0016] Furthermore, in step 1), the input color image with a shadow region is converted into a grayscale image, and then a color image is generated by means of brightness normalization.

[0017] Furthermore, in step 2), the output of the deep neural network is set as the ratio image generated by dividing the grayscale image of the image with a shadow region by the grayscale image of the standard shadowless image. Since the existing training data set only contains the shadowless image corresponding to the shadow image, it is necessary to recalculate the supervised data in the training data set according to this calculation method before training the deep neural network.

[0018] Furthermore, in step 3), for the grayscale image, the shadow region annotation image, and the ratio image obtained in step 2) corresponding to each image in the training data, assign a random angle to rotate it, then randomly select half of the images in the training data set for vertical symmetry, and finally randomly select half of the images to change their aspect ratio. After processing, crop the resolution of the image to be the same as the original image, delete the extra part, and set the pixel value of the missing part to zero. In this way, the diversity of the training data is increased, and the performance of the deep neural network is improved.

[0019] Further, in step 4), the currently commonly used U-shaped deep neural network is used as the basic network architecture, a cross-layer connection structure is added to the basic network structure, and the deep neural network is trained by combining the training data augmented in step 3). The input of this network is the grayscale image of the shaded area image and the position identification map of the shaded area, and the supervised data for supervising the generation result is the ratio image calculated in step 2).

[0020] Further, in step 5), for the test data, first decompose it into a grayscale image and a color image according to step 1). By improving the existing method for calculating the scene illumination condition and spectral reflectance (reference: Wang Xiao, Yao Siyuan, Dai Pengwen, Wang Rui, Cao Xiaochun, "Calculation Method of Illumination Condition and Spectral Reflectance Based on Color Difference between Shadow and Peripheral Region," Acta Electronica Sinica, accepted for publication), calculate the illumination condition of the shaded area in this image according to the color image and the position identification image of the shaded area, and then estimate the spectral reflectance of all shadow pixels by combining the colors of the shadow area pixels.

[0021] Further, in step 6), calculate the color presented by the shadow pixels under the illumination of the standard white light source D65 defined in the sRGB color space according to the spectral reflectance of the shadow pixels, and use the calculated color after brightness normalization as the color of the shadow pixels after shadow removal. The colors of the non-shadow areas remain unchanged, and a color image after shadow removal is generated.

[0022] Further, in step 7), input the grayscale image of the test data and the position identification image of the shaded area into the deep neural network trained in step 4) to generate a ratio image. Then divide the grayscale image by the ratio image to obtain the grayscale image after shadow removal, and multiply the grayscale image and the color image generated in step 6) to obtain the shadow removal image.

[0023] The network structure of the entire algorithm of the present invention is as Figure 1 shown.

[0024] Advantages of the present invention:

[0025] 1. The present invention is dedicated to removing the shaded area in a single color digital image, which has an adverse impact on many practical applications, thus making the present invention have practical application value.

[0026] 2. The present invention uses a deep neural network to generate the intermediate results required for calculating the shadow-free image, which can effectively reduce the situation where the non-shadow area is changed by the deep neural network, thereby improving the effect of shadow removal.

[0027] 3. The present invention combines a learning-based deep neural network with reflectivity, one of the regional physical properties, which can reduce the dependence of the shadow removal method on training data and improve the robustness of the shadow removal method; and the calculation of regional reflectivity does not depend on training data and can be embedded in existing shadow removal methods to improve performance.

[0028] In summary, the shadow removal method based on deep learning and reflectivity proposed by the present invention can effectively remove the shadow area in a color digital image on the premise of making relatively small changes to the non-shadow area of the image. At the same time, by using the reflectivity of the region to calculate color information, the influence of training data is reduced and the robustness of the method is improved. Description of the Drawings

[0029] Figure 1 It is a flowchart of a shadow removal method based on deep learning and reflectivity. Detailed Embodiments

[0030] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts fall within the protection scope of the present invention.

[0031] A specific implementation manner for implementing the present invention is as follows. A shadow removal method based on deep learning and reflectivity, and its steps are:

[0032] 1) Convert the color input image into two images, a grayscale image and a color image.

[0033] 2) Set the supervised image in the training data as the ratio image between the grayscale image of the shadow image and the grayscale image of the non-shadow image.

[0034] 3) Randomly rotate, symmetrically transform, and change the aspect ratio of the grayscale image of the shadow image, the corresponding shadow area annotation image, and the supervised image to augment the training data.

[0035] 4) Use a U-shaped deep neural network with a cross-layer connection structure and train it in combination with the augmented training data to learn how to generate a ratio image using the grayscale image and the shadow area annotation image.

[0036] 5) By improving the existing method for calculating the scene illumination condition and the object spectral reflectivity, calculate the illumination condition of the shadow area according to the color image of the test data, and then calculate the reflectivity of each pixel in the shadow area according to the illumination condition.

[0037] 6) Calculate the color presented by the shadow pixels under standard white light illumination based on the reflectance of the shadow area, and use this color as the color of the shadow pixels after shadow removal to generate the color image of the image after shadow removal.

[0038] 7) Calculate the grayscale image of the test image after shadow removal using the ratio image generated by the deep neural network trained in step 4), and then combine the grayscale image and the color image to generate the final shadow-removed image.

[0039] In an embodiment of the present invention, the color input image with a shadow area is converted into a grayscale image L, and then the color image X of the input image is generated by means of brightness normalization.

[0040] In an embodiment of the present invention, in order to reduce the possibility of changes in the non-shadow area in the generated shadow-free image, the present invention uses a deep neural network to use the grayscale image L of the shadow image and combines the position annotation image of the shadow area in the shadow image to generate a ratio image. The grayscale image of the shadow-free image is calculated by dividing the grayscale image L of the shadow image by the ratio image. Since the grayscale value of the pixels in the shadow-free image is not less than the grayscale value of the corresponding pixels in the shadow area image, this calculation method can ensure that the values of all pixels in the generated ratio image are not greater than 1, thereby improving the accuracy of using the deep neural network. For this reason, the supervised data in the training data is changed, and the color shadow-free image is converted into a grayscale image L in the same method as step 1). g , and then calculate the new supervised data M according to the following formula α :

[0041] M α = L / L g (1)

[0042] In an embodiment of the present invention, aiming at the problem that the styles of the shadow areas in the existing shadow removal data set are relatively single, the grayscale image of the shadow image, the position annotation image of the shadow area, and the newly calculated supervised image M α are randomly rotated, then randomly select half of the images for vertical symmetry, and finally randomly select half of the images to change their aspect ratios. After processing, the resolution of the images is cropped to be the same as that of the original images, and the extra border parts are deleted, and the pixel response values of the missing parts are defined as zero.

[0043] In an embodiment of the present invention, a deep neural network is trained using augmented training data. The deep neural network uses a U-shaped architecture with skip connection structures (reference: Le H., Samaras D., Physics-based Shadow Image Decomposition for Shadow Removal. IEEE Transactions on Pattern Analysis and Machine Intelligence, early access, 2021. https: / / doi.org / 10.1109 / TPAMI.2021.3124934). The network consists of a total of fifteen convolutional layers.

[0044] The loss function of the network consists of two sub-loss functions. The first sub-loss function is the reconstruction loss function L rec , which calculates the 1-norm difference between the scaled image output by the deep neural network and the supervised image. The second sub-loss function is the penumbra region loss function L penumbra . This loss function is designed to enhance the description ability of the deep neural network for the penumbra region near the shadow region boundary. The penumbra region loss function calculates the 1-norm difference between the scaled image of the deep neural network and the supervised image in the region near the shadow region boundary. Specifically in terms of operation, it calculates the region where the distance from the shadow region boundary is less than fifteen pixels, and this region is obtained through morphological dilation and erosion operations.

[0045] The overall loss function of the deep neural network is expressed by the formula:

[0046] L final = L rec + μL penumbra (2)

[0047] In an embodiment of the present invention, by improving the existing method for calculating the scene illumination condition and spectral reflectance (reference: Wang Xiao, Yao Siyuan, Dai Pengwen, Wang Rui, Cao Xiaochun, "Calculation Method of Illumination Condition and Spectral Reflectance Based on Color Difference between Shadow and Peripheral Region," Acta Electronica Sinica, accepted), the illumination condition of the ambient light irradiating the shadow region is calculated, and then the spectral reflectance of each shadow pixel is calculated according to the illumination condition of the ambient light.

[0048] When calculating the lighting conditions using the reference method, it is necessary to separate the effects of direct light and ambient light based on the brightness information of the shadow image, and then calculate the lighting conditions of direct light and ambient light. Such separation calculations will affect the accuracy of the calculation results of the ambient light lighting conditions. The present invention does not require the use of the lighting conditions of direct light. Therefore, the part of calculating the lighting conditions in the reference method is improved, and the color image X of the shadow image and the position annotation image of the shadow area are used to calculate the lighting conditions of the ambient light. Then, the method of calculating the spectral reflectance in the reference method is used to calculate the spectral reflectance of the shadow pixels according to the lighting conditions of the ambient light.

[0049] The lighting conditions of an area in the scene are described by Planck's formula, and the form of Planck's formula is:

[0050]

[0051] In this formula, I represents the brightness of the light source, c1 and c2 are two constants, λ represents the wavelength of light, T represents the thermodynamic temperature of the luminous body, and the unit is Kelvin (K). According to Planck's formula, the lighting conditions of an area are only related to the color temperature T without considering the brightness.

[0052] To calculate the light source color temperature T of the shadow area y , the specific steps are as follows:

[0053] (i) Use the SLIC superpixel segmentation algorithm to segment the color image into superpixels, and calculate the average color of each superpixel according to the segmentation result. Then, according to the position annotation image of the shadow area, find the shadow superpixels near the boundary of the shadow area. Finally, find a non-shadow superpixel closest to each shadow superpixel as a paired superpixel.

[0054] (ii) For each pair of superpixels, express the average color of the shadow superpixel with the formula:

[0055]

[0056] Express the average color of the non-shadow superpixel with the formula:

[0057]

[0058] In these two formulas, 400 and 700 represent the visible light band, with the unit of nm; E represents the light source spectrum calculated according to Planck's formula; λ refers to the wavelength of light; 6500K is the color temperature of the standard white light source; R(λ) refers to the spectral reflectance of the area; Q(λ) is the color matching function in the sRGB color space, and i refers to the color channel of the image.

[0059] (iii) Transform the optimized formula for calculating the lighting conditions in the reference method into:

[0060]

[0061] In this formula, C s and C are the colors calculated according to formulas (4) and (5), and are the average colors of the shadow superpixels and non-shadow superpixels calculated according to the superpixel segmentation results. According to this formula and the optimization algorithm of the reference method, the light source color temperature T of the shadow area is calculated for each pair of superpixels y . The calculation result of the pair of superpixels with the smallest optimization error value D is taken as the final calculation result, that is, the calculation result of the illumination conditions of the pair of superpixels corresponding to the minimum error value is taken as the illumination conditions of the image shadow area.

[0062] After obtaining the light source color temperature T of the illuminated shadow area y , using the method of calculating the spectral reflectance in the reference method, according to the light source color temperature T of the shadow area y calculate the spectral reflectance R(λ) of all shadow pixels.

[0063] In an embodiment of the present invention, for the shadow area pixels with spectral reflectance R(λ), calculate the color presented under the illumination of the standard white light source D65, that is, the light source with a color temperature of 6500K, according to formula (5), and replace the color of the corresponding pixels in the color image X with the calculated result to generate the color image X of the image after shadow removal N .

[0064] In an embodiment of the present invention, for the grayscale image L of the input image, use the trained deep neural network to generate the corresponding ratio image M α , and then calculate the grayscale image L of the image after shadow removal according to the following formula N :

[0065] L N = L / M α (7)

[0066] Combined with the grayscale image L N and the color image X N , generate the shadow removal image I according to the following formula N :

[0067] I N = L N × X N (8)

[0068] The shadow removal method based on deep learning and reflectance proposed by the present invention has the following test environment and experimental results:

[0069] (1) Test environment:

[0070] System environment: Windows 10, Anaconda;

[0071] Hardware environment: Memory: 16GB; GPU: NVIDIA RTX 3060, video memory 6GB; CPU: 2.30GHz, Intel i7-11800H, hard disk: 1TB;

[0072] (2) Experimental data:

[0073] The deep neural network of the present invention is trained on a synthetic dataset composed of the ISTD dataset and the SRD dataset. The ISTD dataset contains 1,870 shadow images taken in 180 scenarios. Each shadow image has a corresponding shadow area identifier and a shadowless image. Among them, 1,330 are training data, and the remaining 540 are test data. The SRD dataset contains 3,088 images taken in 120 scenarios. Each image has a corresponding shadowless image. Among them, 2,680 are training data, and the remaining 408 are test data. It should be noted that due to technical reasons during the shooting process of the standard shadowless images in the ISTD dataset, the error between some images and the areas corresponding to the non-shadow areas in the shadow images is relatively large. The literature (Le H., Samaras D., Physics-based Shadow Image Decomposition for Shadow Removal. IEEE Transactions on Pattern Analysis and Machine Intelligence, early access, 2021. https: / / doi.org / 10.1109 / TPAMI. 2021.3124934) improved it and named it the ISTD+ dataset.

[0074] (3) Optimization method:

[0075] The Adam operator optimization method is adopted, and the size of the minibatch is set to 8 during training.

[0076] (4) Experimental results:

[0077] 1) Performance comparison:

[0078] When compared with other methods, these methods include two traditional methods, Guo and Gong. Guo is a classic traditional method. Although Gong does not require training, it requires users to perform simple manual annotation. In addition, there are some shadow removal methods based on deep learning, including DeshadowNet, ST-CGAN, DSC, Mask-ShadowGAN, which directly use deep neural networks to generate shadow-free images, and two deep methods, SP+M-Net and Exposure, which use deep neural networks to generate intermediate results for shadow removal to reduce changes in non-shadow regions.

[0079] The performance comparison results are shown in Table 1 below. In the input column, S represents the shadow image, M represents the position annotation image of the shadow region, and P represents the shadow-free image paired with the shadow image. The experimental results show that the method of the present invention can better preserve the non-shadow regions in the input image and also achieve good results in the overall image performance.

[0080] Table 1 is the performance comparison

[0081]

[0082]

[0083] 2) Robustness experiment:

[0084] This experiment explores the influence of training data on the shadow removal results. The datasets used are still the SRD and ISTD+ datasets. The comparison methods are SP+M-Net and Exposure. When training the model, first use the training data of the two datasets to train the deep neural network used in the present invention, SP+M-Net and Exposure as the control group, test the test data of the two datasets and calculate the evaluation parameters. Then initialize all parameters, use the training data of the SRD dataset to train these three deep neural networks, test the test data of the two datasets and calculate the evaluation parameters. In the third experiment, use the training data of ISTD+ to train the deep model, test the test data of the two datasets and calculate the evaluation parameters. As shown in the robustness experiment results in Table 2 below, although the shadow removal method based on deep learning and reflectivity proposed in the present invention still has a certain dependence on training data, the evaluation parameters of the shadow region decrease less compared with the existing shadow removal methods; at the same time, the evaluation parameters of the non-shadow regions of the existing shadow removal methods also decrease significantly, while the evaluation parameters of the non-shadow regions of the present invention change very little, indicating that the robustness of the present invention has been greatly improved and meets the design intention.

[0085] Table 2 is the robustness experiment

[0086]

[0087] As can be clearly seen from the above experiments, both the grayscale ratio map calculation based on deep learning and the color calculation based on reflectivity involved in the present invention are effective. Using both for the shadow removal task can achieve good performance and robustness.

[0088] The above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit them. Those of ordinary skill in the art can modify or equivalently replace the technical solutions of the present invention without departing from the scope of the present invention. The protection scope of the present invention shall be subject to what is described in the claims.

Claims

1. A shadow removal method based on deep learning and reflectivity, the steps of which include: 1) Convert the selected colored shadow image into a grayscale image and a color image respectively and perform shadow annotation on them; Set the supervised image as the ratio image between the grayscale image of the shadow image and the grayscale image of the shadow-free image corresponding to the shadow image; wherein, convert the colored shadow image into a grayscale image, then perform brightness normalization on the grayscale image to generate the color image; divide the grayscale image of the shadow image by the grayscale image of the shadow-free image corresponding to the shadow image to generate the ratio image; 2) Use the training data obtained in step 1) to train a U-shaped deep neural network, and the U-shaped deep neural network generates a ratio image according to the supervised image and the corresponding grayscale image and shadow region annotation image; 3) For an image A to be processed, first generate the color image of the image A to be processed and calculate the illumination condition of its shadow region, and then calculate the reflectivity of each pixel in the shadow region of the image A to be processed according to the illumination condition; 4) Calculate the color presented by each pixel in the shadow region of the image A to be processed under standard white light illumination according to the reflectivity obtained in step 3), and use the obtained color as the color of the corresponding pixel after shadow removal to generate the color image B after shadow removal of the image A to be processed; 5) Input the color image of the image A to be processed into the U-shaped deep neural network trained in step 2) to generate a scaled image P A ; then, based on the scaled image P A generate the grayscale image of the color image B obtained in step 4), and then combine the grayscale image with the color image B obtained in step 4) to generate the shadow removal image corresponding to the image A to be processed.

2. The method according to claim 1, wherein The method for training the U-shaped deep neural network is: input the grayscale image L of the colored shadow image sample and the shadow region annotation image corresponding to the grayscale image L into the U-shaped deep neural network to generate a ratio image P; then calculate the loss function value according to the ratio image P and the supervised image corresponding to the colored shadow image sample, and optimize the U-shaped deep neural network.

3. The method according to claim 2, characterized in that, Through the loss function L final = L rec + μL penumbra Calculate the value of the loss function; where, L rec is the reconstruction loss function, used to calculate the 1-norm difference between the scaled image I output by the U-shaped neural network and the corresponding supervised image; L penumbra is the penumbra region loss function, used to calculate the 1-norm difference between the scaled image P output by the U-shaped neural network and the corresponding supervised image in a set region near the boundary of the shadow region, and μ is the set weight coefficient.

4. The method according to claim 3, wherein The set region near the shadow region boundary is the region with a distance less than fifteen pixels from the shadow region boundary.

5. The method according to claim 3 or 4, characterized in that, The set region near the shadow region boundary is obtained by calculating through morphological dilation and erosion operations.

6. The method according to claim 1, characterized in that, According to the formula a minimum error value D is determined, and a pair of superpixel illumination conditions corresponding to the minimum error value D is used as the illumination condition of the image shadow region; wherein, is the average color of the shadow superpixel in a pair of superpixels calculated according to the superpixel segmentation result of the color image, is the average color of the non-shadow superpixel in a pair of superpixels calculated according to the superpixel segmentation result of the color image, T y is the light source color temperature of the shadow region in the color image, R(λ) is the spectral reflectance of the shadow region in the color image, and λ represents the wavelength of the light source.

7. The method according to claim 1, characterized in that, Randomly rotate, symmetrically transform and change the aspect ratio of the grayscale image of the shadow image, the corresponding shadow region annotation image and the supervised image to augment the training data.

8. A server, characterized in that, It includes a memory and a processor, the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program includes instructions for executing the steps in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method in any one of claims 1 to 7 are implemented.

Images