Image shadow detection and removal method and system based on variational characteristic decomposition

Abstract

The invention discloses an image shadow detection and removal method and system based on variational feature decomposition, and relates to the technical field of image processing. Comprises: acquiring a to-be-processed image; performing variational feature decomposition on the to-be-processed image to obtain an intensity feature and a color feature of light; performing shadow detection based on the intensity feature and the color feature of the light; and performing shadow removal on the to-be-processed image according to the shadow detection result. According to the method, the features of the image light are decomposed into the intensity and color features, so that the interference on shadow detection is effectively reduced, and the detection accuracy is improved; shadow detection is carried out in combination with a saliency detection method, so that the interpretability and visual effect of detection are effectively enhanced; and finally, the shadow region is repaired by using the global feature extraction capability of a Fourier transform module and Fourier convolution and the local detail repair capability of ResBlock, so that the shadow is thoroughly removed, the image details and textures are reserved, and the image quality and the visual consistency are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image shadow detection and removal method and system based on variational feature decomposition. Background Art

[0002] At present, there are still some drawbacks and deficiencies in image shadow detection and removal, mainly including the following aspects:

[0003] Slow processing speed: Shadow detection and removal tasks are often carried out separately, resulting in a slow final processing speed and low efficiency of shadow removal.

[0004] Poor shadow removal effect: In the shadow removal method, the characteristics of shadows are not considered from the root cause, resulting in poor detection effect of shadow areas and finally poor shadow removal effect.

[0005] Limited model generalization ability: Existing models are trained on specific data sets and have limited generalization ability for remote sensing image data. The method of shadow removal has not been applied to remote sensing data sets.

[0006] Limited application scope: At present, the method of shadow removal is only applied to image shadow removal and has not been applied in other fields. Summary of the Invention

[0007] In view of the problems existing in the prior art, the present invention is proposed.

[0008] Therefore, the problem to be solved by the present invention is to propose a variational feature decomposition model for decomposing the characteristics of light, fundamentally solving the inaccurate detection of shadow areas and significantly improving the effect of shadow removal.

[0009] To solve the above technical problems, the present invention provides the following technical solutions:

[0010] In a first aspect, an embodiment of the present invention provides an image shadow detection and removal method based on variational feature decomposition, which includes obtaining an image to be processed;

[0011] Performing variational feature decomposition on the image to be processed to obtain the intensity feature and color feature of light;

[0012] Performing shadow detection based on the intensity feature and color feature of light;

[0013] Performing shadow removal processing on the image to be processed according to the shadow detection result.

[0014] As a preferred solution of the image shadow detection and removal method based on variational feature decomposition of the present invention, wherein: the variational feature decomposition includes:

[0015] Decompose the illumination information in the image to be processed into two dimensions: the intensity feature and the color feature of light;

[0016] The intensity feature of light is used to characterize the brightness change in the image, and the color feature is used to characterize the color distribution in the image.

[0017] As a preferred embodiment of the image shadow detection and removal method based on variational feature decomposition according to the present invention, wherein: the shadow detection includes:

[0018] Extract local and global information through two rounds of progressive feature processing;

[0019] Analyze the intensity feature and color feature of light based on the saliency detection method;

[0020] Generate a shadow detection map to characterize the shadow area.

[0021] As a preferred embodiment of the image shadow detection and removal method based on variational feature decomposition according to the present invention, wherein: the shadow removal includes:

[0022] Perform frequency domain conversion using Fourier transform;

[0023] Extract global features by combining convolution operations;

[0024] Repair local details through a residual structure;

[0025] Generate a shadow-free image.

[0026] As a preferred embodiment of the image shadow detection and removal method based on variational feature decomposition according to the present invention, wherein: the shadow removal process is adaptable to optical image data, remote sensing image data, and target detection image data, and the same variational feature decomposition model and shadow processing flow are used for different types of image data.

[0027] As a preferred embodiment of the image shadow detection and removal method based on variational feature decomposition according to the present invention, wherein: the variational feature decomposition model includes the following steps:

[0028] Establish a variational feature decomposition energy model:

[0029]

[0030] Wherein, I and C represent the intensity feature and color feature of light, α and β are weight parameters used to balance the intensity feature and color feature of light, and S and T are implicit variables;

[0031] Transform it into an unconstrained optimization problem:

[0032]

[0033] Among them, γ and δ are penalty factors;

[0034] The variables are separated to obtain an iterative update format:

[0035]

[0036] Among them, n represents the number of iterations. The values of γ and δ increase continuously during the iteration. For this iterative model, the partial derivatives of the sum in the above formula are directly calculated to obtain the form of its closed solution:

[0037]

[0038] Among them, I and C represent the intensity feature and color feature of light, n represents the number of iterations, F is the currently extracted feature, and T is the target feature value.

[0039] As a preferred solution of the image shadow detection and removal method based on variational feature decomposition according to the present invention, wherein: the implicit variables are solved by combining the L2 prior regularization term and the Total Variation spatial prior regularization term:

[0040]

[0041] Among them, the L2 prior term is used to ensure feature smoothness, and Total Variation is used to suppress noise;

[0042] Fourier transform is used for shadow removal. The features are transformed into the frequency domain and the low-frequency features are enhanced, and the image is reconstructed through inverse transformation. The residual structure is used to repair local details.

[0043] In a second aspect, an embodiment of the present invention provides an image shadow detection and removal system based on variational feature decomposition, which includes a variational feature decomposition module, a saliency detection and Fourier transform module, a remote sensing image shadow removal module, and an application module of shadow removal in target detection;

[0044] The variational feature decomposition module is used to decompose the features of light into the intensity feature and color feature of light;

[0045] The saliency detection and Fourier transform module presents the shadow detection result better and removes the shadow through Fourier transform;

[0046] The remote sensing image shadow removal module creates a dataset XWLD from the remote sensing images collected by the unmanned aerial vehicle, and applies the shadow removal algorithm to the remote sensing images for the first time;

[0047] The application module of shadow removal in target detection applies the shadow removal method in target detection, and also applies it to the target detection of vehicles in remote sensing images.

[0048] In a third aspect, an embodiment of the present invention provides a computer device, including a memory and a processor, where the memory stores a computer program, and: when the computer program instructions are executed by the processor, the steps of the image shadow detection and removal method based on variational feature decomposition as described in the first aspect of the present invention are implemented.

[0049] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, and: when the computer program instructions are executed by the processor, the steps of the image shadow detection and removal method based on variational feature decomposition as described in the first aspect of the present invention are implemented.

[0050] The beneficial effects of the present invention are as follows: By obtaining the image to be processed and performing variational feature decomposition, the features of light are decomposed into intensity and color features, thereby providing a more accurate feature representation for shadow detection, avoiding the interference of light intensity and color changes on detection, and significantly improving the accuracy and robustness of shadow detection. Based on the decomposed features for shadow detection, the shadow area can be more accurately identified, and at the same time, the saliency detection method enhances the interpretability and visual effect of the detection. Finally, combined with the shadow detection result for shadow removal processing, by combining the global feature extraction ability of Fourier transform module and Fourier convolution and the local detail restoration ability of ResBlock, the shadow area is restored, not only completely removing the shadow, but also retaining the details and textures of the image, significantly improving the image quality and visual consistency, and greatly improving the detection accuracy in applications such as object detection, verifying its efficiency and practicality in the field of image processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following described drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0052] Figure 1 It is a flowchart of an image shadow detection and removal method based on variational feature decomposition;

[0053] Figure 2 It is a computer device diagram of an image shadow detection and removal method based on variational feature decomposition;

[0054] Figure 3 It is a schematic diagram of the model structure of an image shadow detection and removal method based on variational feature decomposition.

[0055] Figure 4Schematic diagram of Fourier transform for shadow removal in the image shadow detection and removal method based on variational feature decomposition.

[0056] Figure 5 Schematic diagram of the feature decomposition model in the image shadow detection and removal method based on variational feature decomposition.

[0057] Figure 6 Schematic diagram of the comparison of quantitative results between the proposed algorithm in the image shadow detection and removal method based on variational feature decomposition and the LAB-Net algorithm on ISTD.

[0058] Figure 7 Schematic diagram of the comparison of quantitative results between the proposed algorithm in the image shadow detection and removal method based on variational feature decomposition and the LAB-Net algorithm on SRD.

[0059] Figure 8 Schematic diagram of the comparison of quantitative results between the proposed algorithm in the image shadow detection and removal method based on variational feature decomposition and the SpA-Former algorithm on DeepGlobe.

[0060] Figure 9 Schematic diagram of the comparison of quantitative results between the proposed algorithm in the image shadow detection and removal method based on variational feature decomposition and the SpA-Former algorithm on XWLD.

[0061] Figure 10 Schematic diagram of the comparison of the number of parameters on the ISTD dataset in the image shadow detection and removal method based on variational feature decomposition.

[0062] Figure 11 Schematic diagram of the comparison of floating-point operation counts on the ISTD dataset in the image shadow detection and removal method based on variational feature decomposition.

[0063] Figure 12 Schematic diagram of the qualitative comparison on the ISTD dataset in the image shadow detection and removal method based on variational feature decomposition.

[0064] Figure 13 Schematic diagram of the qualitative comparison on the SRD dataset in the image shadow detection and removal method based on variational feature decomposition.

[0065] Figure 14 Schematic diagram of the qualitative comparison on the DeepGlobe dataset in the image shadow detection and removal method based on variational feature decomposition.

[0066] Figure 15 Schematic diagram of the qualitative comparison on the XWLD dataset in the image shadow detection and removal method based on variational feature decomposition.

[0067] Figure 16 Schematic diagram for qualitative comparison of shadow detection in the image shadow detection and removal method based on variational feature decomposition.

[0068] Figure 17 Schematic diagram of the UAV remote sensing image before and after shadow removal in the image shadow detection and removal method based on variational feature decomposition.

[0069] Figure 18 Schematic diagram for parameter analysis of the image shadow detection and removal method based on variational feature decomposition on the ISTD dataset.

[0070] Figure 19 Schematic diagram for parameter analysis of the image shadow detection and removal method based on variational feature decomposition on the SRD dataset.

[0071] Figure 20 Schematic diagram for parameter analysis of the image shadow detection and removal method based on variational feature decomposition on the DeepGlobe dataset.

[0072] Figure 21 Schematic diagram for parameter analysis of the image shadow detection and removal method based on variational feature decomposition on the XWLD dataset.

[0073] Figure 22 Schematic diagram for comparison results of different samples in object detection of the image shadow detection and removal method based on variational feature decomposition.

[0074] Figure 23 Schematic diagram for comparison of confidence scores of different samples in object detection of the image shadow detection and removal method based on variational feature decomposition.

[0075] Figure 24 Schematic diagram of the UAV remote sensing image before and after shadow removal in vehicle object detection of the image shadow detection and removal method based on variational feature decomposition. Detailed implementation manners

[0076] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following will describe the detailed implementation manners of the present invention with reference to the accompanying drawings of the specification.

[0077] In the following description, many specific details are set forth to fully understand the present invention. However, the present invention can also be implemented in other ways different from this description. Those skilled in the art can make similar extensions without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0078] Secondly, the so-called "one embodiment" or "embodiment" herein refers to specific features, structures, or characteristics that may be included in at least one implementation manner of the present invention. The phrase "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it an individual or alternative embodiment that is mutually exclusive with other embodiments.

[0079] Embodiment 1

[0080] Referring to Figure 1 , which is the first embodiment of the present invention. This embodiment provides an image shadow detection and removal method based on variational feature decomposition, including:

[0081] S1: Obtain the image to be processed;

[0082] It should be noted that the "image to be processed" refers to the original image that needs to be subjected to shadow detection and removal processing. This image may be sourced from different types of image data, including but not limited to:

[0083] Optical image: For example, an image obtained by an ordinary camera device. Such images are usually affected by natural light changes and may have shadow areas that affect the image quality.

[0084] Remote sensing image: A ground image obtained by devices such as satellites or drones. These images often have shadows or reflections due to different lighting conditions and atmospheric effects, which affect the accurate identification of ground objects.

[0085] Target detection image: Image data used for automated identification of specific objects. These images may have interference in shadow areas, affecting the recognition accuracy of the target.

[0086] The common feature of these images to be processed is that they all contain shadow areas that may affect the image quality or subsequent analysis tasks. The objective of this invention is to remove or reduce these shadows through image processing, improving the usability and accuracy of the image.

[0087] S2: Perform variational feature decomposition on the image to be processed to obtain the intensity feature and color feature of light;

[0088] It should be noted that variational feature decomposition is based on the lighting information of the image. Through mathematical modeling and optimization methods, the pixel information of the image is disassembled into two independent parts that can respectively describe the brightness (light intensity) and color (color) in the image.

[0089] The intensity feature of light reflects the brightness changes of each pixel in the image, that is, the strength of the light illumination. It describes the brightness information of different regions in the image and is usually related to factors such as the light source position in the image, the reflection characteristics of the object surface, and the ambient illumination. The light intensity feature is an important basis for shadow detection and removal because shadow regions usually exhibit changes or decreases in brightness. By decomposing the light intensity feature, the system can more clearly identify which parts are the regions darkened due to the presence of shadows, thus providing a basis for subsequent shadow detection.

[0090] The color feature describes the color information of each pixel in the image, that is, the hue, saturation, brightness, etc. of each pixel. This feature mainly reflects the color distribution and color changes in the image and is usually related to factors such as the object category in the image and the environment. The color feature is also very crucial for shadow removal because shadow regions not only affect the brightness but may also change the color distribution in that region. By decomposing the color feature, the system can better understand the color changes in the image and ensure that the color of the image remains natural and real after shadow removal.

[0091] S3: Perform shadow detection based on the intensity feature and color feature of the light;

[0092] It should be noted that shadow detection refers to the process of using the light intensity feature and color feature extracted from the image to be processed to identify which parts of the image belong to the shadow region. Shadows usually manifest as a decrease in light intensity or a change in color in the image. Therefore, by analyzing these two features, the shadow region and the non - shadow region can be effectively distinguished.

[0093] S4: Perform shadow removal processing on the image to be processed according to the shadow detection result

[0094] It should be noted that shadow removal processing refers to, after completing shadow detection, using the detected shadow region information and through certain algorithms and technical means to repair or replace the image content in the shadow region, so that the shadow effect in the image disappears or is significantly weakened, and the natural light illumination and color of the image are restored. The purpose of shadow removal is to eliminate the impact of the shadow region on the image quality. In particular, shadows may cause brightness and color distortion in the image, affecting the visual effect of the image and subsequent processing (such as object detection, object recognition, etc.). The image after shadow removal should restore a more natural light illumination and color, making the content of the image clearer and more real.

[0095] In summary, the method for detecting and removing image shadows based on variational feature decomposition provided by the present invention can accurately extract the light intensity and color features of the image, effectively distinguish between illumination changes and color distributions, and provide clear input data for subsequent shadow detection and removal. Secondly, the shadow detection method using saliency detection and progressive feature extraction can accurately locate shadow regions under complex illumination conditions, especially in images with high contrast or uneven illumination. Finally, through shadow removal processing, the brightness and clarity of the image are maintained, while details are restored, avoiding over-smoothing or loss of details.

[0096] Embodiment 2

[0097] Referring to Figures 1 - 5 , this is the second embodiment of the present invention. This embodiment provides a method for detecting and removing image shadows based on variational feature decomposition, including:

[0098] S1: Obtain the image to be processed;

[0099] In the embodiments of the present application, the image to be processed includes, but is not limited to: optical images, remote sensing images, and object detection images.

[0100] S2: Perform variational feature decomposition on the image to be processed to obtain the intensity feature and color feature of light;

[0101] In the embodiments of the present application, the variational feature decomposition includes:

[0102] Decompose the illumination information in the image to be processed into two dimensions: the intensity feature and color feature of light;

[0103] The intensity feature of light is used to characterize the brightness changes in the image, and the color feature is used to characterize the color distribution in the image.

[0104] The variational feature decomposition model includes the following steps:

[0105] Establish a variational feature decomposition energy model:

[0106]

[0107] Among them, I and C represent the intensity feature and color feature of light, α and β are weight parameters used to balance the intensity feature and color feature of light, and S and T are implicit variables;

[0108] Convert it into an unconstrained optimization problem:

[0109]

[0110] Among them, γ and δ are penalty factors;

[0111] Separate the variables to obtain an iterative update format:

[0112]

[0113] Among them, n represents the number of iterations. The values of γ and δ increase continuously during the iteration. For this iterative model, by directly taking the partial derivatives of the sum in the above formula, the form of its closed solution is obtained:

[0114]

[0115] Among them, I and C represent the intensity feature and color feature of light, n represents the number of iterations, F is the currently extracted feature, and T is the target feature value.

[0116] Essentially, S and T are implicit prior knowledge of the light intensity feature and color feature, and are used to describe the inherent attributes and features of light in feature decomposition. Traditional optimization methods solve such problems by using gradient information to search for the optimal solution in the process of energy descent, but it is easy to fall into local optima. This paper uses a new type of implicit prior definition. The L1 prior term regularization prompts the model to pay more attention to those key regions and reduces the redundancy of the model. The L2 prior term regularization focuses on global information, ensures that the output of the model is smoother, and helps the model learn more smooth and balanced features. Since both the light intensity feature and color feature are crucial for feature decomposition, the L2 prior regularization term is more suitable for solving implicit variables.

[0117] Solve the implicit variables S and T by combining the L2 prior regularization term and the Total Variation spatial prior regularization term:

[0118]

[0119] Among them, the L2 prior term is used to ensure feature smoothness, and Total Variation is used to suppress noise.

[0120] S3: Perform shadow detection based on the intensity feature and color feature of the light;

[0121] In the embodiments of this application, the shadow detection includes:

[0122] Extract local and global information through two rounds of progressive feature processing;

[0123] Analyze the intensity feature and color feature of the light based on the saliency detection method;

[0124] Generate a shadow detection map to represent the shadow area.

[0125] It should be noted that the first-round RNN aims to generate a feature map that summarizes the context information of the position points from the input image. This part focuses on capturing local features and extracting the initial feature relationships between the shadow and non-shadow regions. Based on the feature map generated in the first round, the second-round RNN further collects non-local context information through deeper image information to obtain a feature map with global perception ability, as shown in Figure 4 shown. Through two rounds of progressive feature processing, both the local details of the image are retained and the model's understanding ability of global information is enhanced, providing a high-quality feature representation for the subsequent shadow detection task.

[0126] After two rounds of RNN feature extraction and variational feature decomposition module processing, in order to more accurately extract shadow feature information, a saliency map is introduced in the research. The saliency map can effectively integrate local and global context information to generate a more accurate shadow feature representation.

[0127] Preferably, compared with the traditional heat map form, the saliency map shows more significant advantages in the shadow detection task, can avoid the interference caused by light intensity features and color features, and improve the quality of the detection effect. The method of saliency detection focuses on mining the basis of the model's decision-making, quantifies the sensitivity of the model to each part of the input image by calculating the gradient, and highlights the key regions that really affect the shadow detection result.

[0128] When presenting the shadow area, the saliency map retains the spatial structure and detail information of the area, clearly shows the boundary between the shadow and non-shadow areas, and further improves the interpretability and visual expressiveness of the detection effect.

[0129] In the "end-to-end" image restoration architecture, ResBlock is a common design module. It can reconstruct the shadow-free corresponding image from the shadowed image by modeling the high-frequency and low-frequency differences between the shadow image and the non-shadow image. Traditional ResBlocks are mainly good at capturing the high-frequency components in the image and have weak modeling ability for low-frequency information, but the low-frequency information in the image contains more image structure information.

[0130] S4: Perform shadow removal processing on the image to be processed according to the shadow detection result

[0131] In the embodiments of the present application, shadow removal includes:

[0132] Perform frequency domain conversion using Fourier transform;

[0133] Extract global features by combining convolution operations;

[0134] Repair local details through the residual structure;

[0135] Generate a shadow-free image.

[0136] The shadow removal process is adaptable to optical image data, remote sensing image data, and object detection image data, and the same variational feature decomposition model and shadow processing flow are adopted for different types of image data.

[0137] Shadow removal by Fourier transform is achieved by performing a frequency domain transform on the features and enhancing the low-frequency features, reconstructing the image through an inverse transform, and using a residual structure to repair local details.

[0138] Preferably, the Fourier transform shadow removal module can improve the overall effect of shadow removal by fully combining the global feature extraction ability of Fourier convolution and the local detail repair ability of ResBlock.

[0139] The Fourier convolution residual block enhances the expression of low-frequency features through frequency domain conversion and retains the high-frequency details in the image. Subsequently, the Saliency Map generated by the saliency detection module is used to guide the weighted optimization of Fourier features, highlighting the regions significantly affected by shadows through the enhancement of the salient regions. These processing results are defined as the negative residuals of the image and need to be removed from the original image. On this basis, the negative residual features are locally corrected by the ResBlock module and added to the input original image to generate a shadow-free image. The specific architecture diagram of Fourier transform shadow removal is as shown in the appendix Figure 5 as follows.

[0140] Furthermore, this embodiment also provides an image shadow detection and removal system based on variational feature decomposition, including a variational feature decomposition module, a saliency detection and Fourier transform module, a remote sensing image shadow removal module, and an application module of shadow removal in object detection;

[0141] The variational feature decomposition module is used to decompose the features of light into light intensity features and color features;

[0142] The saliency detection and Fourier transform module better presents the shadow detection results and removes the shadows through Fourier transform;

[0143] The remote sensing image shadow removal module creates a dataset XWLD from the remote sensing images collected by the drone and applies the shadow removal algorithm to the remote sensing images for the first time;

[0144] The application module of shadow removal in object detection applies the shadow removal method in object detection and also applies it to the object detection of vehicles in remote sensing images.

[0145] This embodiment also provides a computer device, which is applicable to the case of the image shadow detection and removal method based on variational feature decomposition, including a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the image shadow detection and removal method based on variational feature decomposition as proposed in the above embodiment.

[0146] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a carrier network, NFC (Near Field Communication), or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, a touchpad, or a mouse, etc.

[0147] This embodiment also provides a storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the image shadow detection and removal method based on variational feature decomposition as proposed in the above embodiment.

[0148] In summary, through Fourier transform for frequency domain conversion, the present invention can enhance the low-frequency features of an image and effectively eliminate the influence of shadows on the image. By combining convolution operations to extract global features, the overall structure of the image can be more accurately restored. The application of the residual structure helps to repair the details that may be lost during the shadow removal process, enabling the image to retain more local details while removing shadows. This multi-level and multi-dimensional shadow removal method significantly improves the removal effect, and is particularly applicable to different types of image data such as optical images, remote sensing images, and target detection images, with strong adaptability.

[0149] Embodiment 3

[0150] Referring to Figures 1 - 24 , this is the third embodiment of the present invention. This embodiment provides a method for detecting and removing image shadows based on variational feature decomposition. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0151] The main algorithm extracts encoded features through the TransFormer module and then detects and removes shadows from images through the GAN network. The entire GAN network consists of two modules. One is the TwoRNN Feature Decomposition (TRFD) module for shadow detection, and the other is the ResidualFourier Change Block (RFCB) module for shadow removal. The final shadow-removed image is obtained through CNN convolution and sent to the discriminator for judgment. The specific details of the entire network are as shown in the appendix Figure 3 as follows.

[0152] This example is a series of experiments we conducted on small-scale optical image datasets ISTD and SRD, and large-scale remote sensing datasets DeepGlobe and XWLD.

[0153] Table 1 shows the comparative experiments of current mainstream image shadow processing algorithms on the ISTD and SRD datasets. The results show that the algorithm in this paper performs outstandingly in the shadow area and the whole image, and also maintains high competitiveness in the non-shadow area. On the ISTD dataset, compared with the LAB-Net algorithm, the RMSE of the shadow area, non-shadow area, and the whole image decreased by 24%, 10%, and 11% respectively, and the PSNR increased by 2.01dB, 1.28dB, and 0.63dB respectively. The specific improvement results are as follows Figure 6 as follows. On the SRD dataset, compared with the LAB-Net algorithm, the RMSE of the shadow area, non-shadow area, and the whole image decreased by 18%, 6%, and 10% respectively, and the PSNR increased by 2.15dB, 0.22dB, and 1.45dB respectively. The specific improvement results are as follows Figure 7 as follows. The scenes involved in the SRD dataset are slightly more complex than those in the ISTD, and the accuracy index values are lower than those in the ISTD. However, the method in this paper still has significant improvements, demonstrating its excellent generalization ability and robustness in different scenarios.

[0154] Table 1

[0155]

[0156]

[0157] To conduct research on removing shadows from remote sensing datasets, the experiment introduced mainstream shadow removal methods on the ISTD and SRD datasets into remote sensing image processing, and trained and tested them on the publicly available remote sensing dataset DeepGlobe and the private remote sensing dataset XWLD constructed in this paper. Table 3 shows the quantitative comparison results of each algorithm on the DeepGlobe and XWLD datasets. The results show that the algorithm in this paper performs outstandingly in the shadow area, non-shadow area, and the entire image. On the DeepGlobe dataset, compared with the SpA-Former algorithm, the RMSE of the shadow area, non-shadow area, and the entire image decreased by 19%, 17%, and 25% respectively, and the PSNR increased by 2.78dB, 1.49dB, and 1.75dB respectively. The specific improvement results are as follows Figure 8 shown. On the XWLD dataset, compared with the SpA-Former algorithm, the RMSE of the shadow area, non-shadow area, and the entire image decreased by 25%, 26%, and 34% respectively, and the PSNR increased by 2.88dB, 1.98dB, and 2.03dB respectively. The specific improvement results are as follows Figure 9 shown. Due to the complexity of the remote sensing scene, the dataset contains various types of ground objects such as large-scale terrain, buildings, and vegetation, as well as more complex irregular shadow distributions. The overall result of the quantitative accuracy has decreased compared with the single-image dataset. The method proposed in this paper has a greater improvement compared with other methods on the remote sensing dataset, demonstrating stronger algorithm robustness.

[0158] Table 2

[0159]

[0160]

[0161] The algorithm in this paper uses a lightweight generative adversarial network to simultaneously generate images for shadow detection and removal. While maintaining the image quality, it can efficiently remove shadows. Table 3 shows the efficiency comparison of each algorithm on the ISTD dataset, including the comparison of the number of parameters and floating-point calculations in the network. The SpA-Former method can also simultaneously generate images for shadow detection and removal, but it does not consider the conditions for shadow formation during shadow detection. It performs shadow detection through the form of a heat map of the attention mechanism, resulting in poor shadow detection and finally poor shadow removal effect. On the ISTD dataset, the method in this paper is slightly less efficient than the SpA-Former method, but the detection and removal effect is greatly improved. The comparison of the number of network parameters and floating-point calculations of different methods is as follows Figure 10 and Figure 11 shown.

[0162] Table 3

[0163]

[0164]

[0165] In this paper, the proposed algorithm is applied to the test sets of image datasets ISTD and SRD, as well as remote sensing datasets DeepGlobe and XWLD, respectively, to generate corresponding shadow removal results. Figures 12 - 15 In, the first column is the input data, the second to sixth columns show the accuracies of different methods, and the last column is the true shadow-free image.

[0166] Experiments on the image datasets ISTD and SRD show that in the shadow detection stage, the combination strategy of illumination variational feature decomposition and saliency detection can effectively overcome the problems of insufficient detection accuracy and edge blurring brought by traditional heat map methods. Through in-depth feature decomposition, the method in this paper can more accurately separate the shadow area and the non-shadow area, significantly improving the accuracy and robustness of shadow detection. In the shadow removal stage, the method in this paper further optimizes the processing of the shadow area, making the shadow in the removed image disappear more thoroughly and the texture details better restored, and finally generating an image that is closer to the real shadow-free scene. In the SRD dataset, the method in this paper still shows highly stable performance in the face of complex and variable shadow forms and diverse lighting conditions. In the qualitative comparison effect diagrams of shadow detection and removal, it can be seen that the method in this paper can better process the shadow edge area compared with other methods, avoiding artifacts and blurring problems in the image, while retaining the details of the non-shadow area and improving the overall image quality.

[0167] Experiments on remote sensing datasets DeepGlobe and XWLD show that compared with conventional single images, the scenes in remote sensing images are more complex, containing various textures and lighting conditions, and it is generally difficult to achieve the same level of precision as conventional images in terms of detection results. Due to the significantly higher complexity of the scenes involved in remote sensing images than in conventional images, if we do not conduct in-depth analysis starting from the lighting characteristics and the nature of shadows, simply using traditional shadow detection and removal methods often fails to achieve satisfactory results. Methods such as DSC, Auto-Exp, DHAN, and SpA-Former all have deficiencies in dealing with remote sensing images. Their ability to handle complex shadows is limited, especially in areas with large lighting variations and rich scene depths. In remote sensing images, SpA-Former has a better shadow removal effect than the DSC, Auto-Exp, and DHAN algorithms. However, its heatmap-based shadow detection method is limited by the attention mechanism in remote sensing scenes, resulting in a decrease in detection accuracy. The generated heatmaps have low quality, the identification of shadow areas is inaccurate, and there are obvious problems of color dullness in the images after shadow removal. For remote sensing image datasets, simply relying on traditional shadow detection and removal methods is difficult to adapt to the complexity of remote sensing scenes. The method in this paper, by fundamentally analyzing the characteristics of light in detail and combining variational feature decomposition with saliency detection strategies, overcomes the limitations of traditional methods in complex scenes and maintains the texture details and color consistency of the original image while removing shadows.

[0168] In addition, this paper not only compares the effects of shadow removal but also the performance of intermediate shadow detection. Currently, the main methods for presenting shadow detection maps mainly include two types: one is in the form of heatmaps, and the other is based on saliency detection methods. The SpA-former method presents shadow detection results in the form of heatmaps. However, since heatmaps are easily affected by the light intensity characteristics and color characteristics in the input images when showing the attention distribution of the model, the final presented detection effect maps are not ideal. Attention heatmaps mainly reflect the attention distribution of the model by showing the image areas that the model focuses on in a specific task. In dealing with shadow detection tasks, this method is easily significantly interfered by the lighting changes in the input images, resulting in the model generating incorrect high-attention distributions in some shadow areas and reducing the detection accuracy. The method using saliency detection in this paper can avoid the influence of light intensity characteristics and color characteristics and improve the quality of the detection effect. The qualitative comparison of the shadow detection between the method in this paper and the SpA-Former method is as follows Figure 16 shown.

[0169] In addition, Figure 17The visual effects of UAV remote sensing images before and after shadow removal are shown. By applying the method in this paper to UAV remote sensing images, the influence of shadows on image quality can be effectively eliminated, the visual consistency and brightness uniformity of the images can be significantly improved, and the ground object information in the images can be made clearer and more distinguishable. It provides more reliable basic data for subsequent remote sensing image interpretation, target recognition and other tasks, and further enhances the application value of remote sensing data. In the parameter analysis, observe how parameter changes affect the overall shadow removal effect of the image. By adjusting α and β to adjust the weights of the light intensity feature and the color feature, observe the change trend of the overall model performance. In the parameter analysis of this paper, the indicators (PSNR, SSIM, RMSE) of the shadow area, non-shadow area and the whole image are compared to evaluate the method performance. The specific parameter analysis results are as follows Figure 18 、 Figure 19 、 Figure 20 and Figure 21 as shown. Experiments on four different datasets show that when the weight α of the light intensity takes a value of 0.6, the three different evaluation indicators of RMSE, SSIM, and PSNR all achieve the best results in the shadow area, non-shadow area, and the whole image. In most shadow scenes, the weakening of light intensity is the most significant feature of shadows. The occlusion of the light source will cause a significant reduction in the brightness of the shadow area, and this intensity change is easier to detect than the color change. Giving a higher weight to the light intensity feature can ensure that the model performs well when dealing with these common scenes. In addition, although the light intensity feature dominates, the color feature still provides valuable supplementary information. In some multi-light source or complex lighting conditions, the color change may be more obvious than the intensity change. The light intensity feature as the main feature (α = 0.6) effectively captures the core features of shadows, while the color feature (β = 0.4) provides auxiliary information, further improving the performance of the model in some complex lighting scenes, making the model have better robustness and generalization ability in various scenes.

[0170] Shadows in the image will obscure the important visual features of the target, resulting in a decrease in local contrast of the image, blurred edges, and loss of texture information, seriously affecting the interpretation ability of the visual model for the target. In UAV remote sensing images, shadows are usually generated by the combined action of variable lighting conditions, complex terrain structures, and diverse object forms, further exacerbating the difficulty of visual detection.

[0171] Currently, object detection tasks in images mostly rely on enhancing the scale of the dataset or improving the detection algorithm to enhance the robustness of the model. For the interference caused by shadows, no in-depth exploration has been carried out. The vast majority of detection methods do not fully consider the impact of shadows and simply treat them as background noise or uncontrollable factors, resulting in limited detection performance of the model in shadow areas. Although some studies have tried to weaken the impact of shadows through data preprocessing, such as adjusting brightness or contrast, these methods cannot fundamentally solve the problem of feature loss caused by shadow occlusion.

[0172] Figure 22 Shown is the object detection accuracy of YOLO after removing shadows using different algorithms. The confidence scores of the objects detected using the YOLO model are marked below the figure. It can be seen from the figure that by applying the shadow removal algorithm in this paper, the recognition accuracy of the YOLO model has increased by 19.5%, and the difference in accuracy from using the YOLO model on real images is only 2.8%. The improved accuracy verifies the importance of removing shadows for improving object detection performance and also verifies that the shadow removal algorithm used in this paper has a better effect on handling shadows compared to other algorithms. The specific confidence comparison of different samples is as follows Figure 23 shown.

[0173] In addition, the method of shadow removal and vehicle object detection in this paper are applied to drone remote sensing images. It can be seen from the pictures that using the method in this paper to remove shadows makes the vehicle object detection in remote sensing images more accurate. Removing shadows is important for improving object detection performance. The specific vehicle object detection in drone remote sensing images before and after shadow removal is as follows Figure 24 shown.

[0174] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. An image shadow detection and removal method based on variational feature decomposition, characterized in that: including, obtaining an image to be processed; performing variational feature decomposition on the image to be processed to obtain the intensity feature and color feature of light; performing shadow detection based on the intensity feature and color feature of light; performing shadow removal processing on the image to be processed according to the shadow detection result.

2. The method for image shadow detection and removal based on variational feature decomposition according to claim 1, characterized in that: The variational feature decomposition includes: decomposing the illumination information in the image to be processed into two dimensions, namely the intensity feature and color feature of light; the intensity feature of light is used to characterize the light and dark changes in the image, and the color feature is used to characterize the color distribution in the image.

3. The method for image shadow detection and removal based on variational feature decomposition according to claim 2, wherein: The shadow detection includes: extracting local and global information through two rounds of progressive feature processing; analyzing the intensity feature and color feature of light based on a saliency detection method; generating a shadow detection map to characterize the shadow area.

4. The method for image shadow detection and removal based on variational feature decomposition according to claim 3, wherein: The shadow removal includes: performing frequency domain conversion using Fourier transform; extracting global features by combining convolution operations; repairing local details through a residual structure; generating a shadow-free image.

5. The method for image shadow detection and removal based on variational feature decomposition according to claim 4, characterized in that: The shadow removal processing is adaptable to optical image data, remote sensing image data, and target detection image data, and the same variational feature decomposition model and shadow processing flow are adopted for different types of image data.

6. The method for image shadow detection and removal based on variational feature decomposition according to claim 5, wherein: The variational feature decomposition model includes the following steps: establishing a variational feature decomposition energy model: where I and C represent the intensity feature and color feature of light, α and β are weight parameters used to balance the intensity feature and color feature of light, and S and T are implicit variables; transforming it into an unconstrained optimization problem: where γ and δ are penalty factors; separating variables to obtain an iterative update format: where n represents the number of iterations, and the values of γ and δ increase continuously during the iteration. Solving this iterative model, directly taking the partial derivatives of and in the above formula to obtain its closed-form solution: where I and C represent the intensity feature and color feature of light, n represents the number of iterations, F is the currently extracted feature, and T is the target feature value.

7. The method for image shadow detection and removal based on variational feature decomposition according to claim 6, characterized in that: solving the implicit variable by combining the L2 prior regular term and the Total Variation spatial prior regular term: where the L2 prior term is used to ensure feature smoothness, and Total Variation is used to suppress noise; performing shadow removal using Fourier transform, performing frequency domain transformation on the feature and enhancing the low-frequency feature, reconstructing the image through inverse transformation, and using a residual structure to repair local details.

8. An image shadow detection and removal system based on variational feature decomposition, based on the image shadow detection and removal method based on variational feature decomposition according to any one of claims 1 to 7, characterized in that: It also includes a variational feature decomposition module, a saliency detection and Fourier transform module, a remote sensing image shadow removal module, and an application module of shadow removal in target detection; the variational feature decomposition module is used to decompose the feature of light into the intensity feature and color feature of light; the saliency detection and Fourier transform module better presents the shadow detection result and removes the shadow through Fourier transform; the remote sensing image shadow removal module creates a dataset XWLD from the remote sensing images collected by the drone and applies the shadow removal algorithm to the remote sensing images for the first time; the application module of shadow removal in target detection applies the shadow removal method in target detection and also applies it to the target detection of vehicles in remote sensing images.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that: When the processor executes the computer program, the steps of the method for image shadow detection and removal based on variational feature decomposition according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, the steps of the method for image shadow detection and removal based on variational feature decomposition according to any one of claims 1 to 7 are implemented.

Images