Image de-shadowing method balancing determinism recovery and perceptual quality

Abstract

The application discloses a kind of image shadow removal methods of balancing deterministic recovery and perceived quality, comprising:1, first stage global regression analysis is carried out to shadow image, global illumination and color consistency of image are repaired using global regression network based on no residual, and deterministic illumination anchor point is established;2, the output of global regression network is input into the diffusion model of second stage as conditional space prior, and local perceptual refinement is carried out using conditional diffusion model based on block, and fine texture details are recovered by generative prior;3, in the reverse denoising sampling process of second stage, introduce random block overlap mechanism, calculate the mean of all covering block prediction noise as updating basis for the purpose of eliminating the boundary artifact when full resolution inference.The application can significantly improve the visual fidelity of the de-shadowed image while ensuring pixel-level accuracy, and can efficiently process high-resolution images under limited video memory constraints.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically a method for image shadow removal that balances deterministic restoration and perceived quality. Background Technology

[0002] Image shadow removal, a fundamental and crucial image restoration task in computer vision, aims to eliminate shadowed areas from degraded images occluded by single or multiple light sources and accurately restore the background texture and color beneath the shadows. This technology has broad application needs and significant prospects in practical scenarios such as autonomous driving, drone aerial photography, satellite remote sensing, and post-processing of everyday mobile phone photos.

[0003] In recent years, with the rapid development of deep learning technology, end-to-end image shadow removal methods based on convolutional neural networks and visual Transformers have achieved remarkable results. These methods typically utilize large amounts of paired data for training, achieving shadow removal by establishing a non-linear mapping from degraded features to shadow-free features. However, existing image shadow removal methods still suffer from the following two significant shortcomings and technical bottlenecks when dealing with complex real-world scenes, especially high-resolution images: First, global illumination and color consistency are difficult to guarantee, easily leading to color deviations. Shadow removal is essentially a non-additive illumination restoration process. Many existing network architectures, due to improper handling of the balance between global information and local features, result in obvious discontinuities in hue, brightness, and saturation between the shadow-removed region and the original shadowless region. Especially when facing large-area shadows with complex ambient lighting, the model is prone to producing unnatural color shifts or overexposure artifacts, failing to maintain large-scale global illumination consistency.

[0004] Second, there is a serious conflict between high perceived quality and computational resource allocation. In pursuit of higher realism and finer textures, some methods have begun to introduce conditional generative models with strong generative priors. However, high-resolution images contain massive amounts of pixels and fine features. Directly inputting high-resolution images into such complex generative networks causes the size of the feature maps in the network to grow exponentially. This leads to a sharp increase in GPU memory throughput during inference and execution, often triggering memory overflow errors due to exceeding hardware physical constraints. To avoid memory collapse, if high-resolution large images are processed using the traditional method of simple slicing and then stitching, the lack of a co-sampling mechanism between blocks introduces severe, discontinuous block artifacts and physical gap artifacts at the edges of the final stitched image, greatly damaging the visual realism of the entire image. Summary of the Invention

[0005] This invention addresses the shortcomings of existing technologies by proposing an image shadow removal method that balances deterministic restoration and perceived quality. It aims to effectively combine the architectural advantages of global regression networks and conditional diffusion models, and fully leverage the characteristics of deterministic illumination anchor point representation and local perceived detail representation. This enables collaborative sampling of high-resolution images with ultra-low memory consumption and effective removal of large-scale realistic shadows in images.

[0006] To achieve the above-mentioned objectives, the present invention adopts the following technical solution: The present invention provides an image shadow removal method that balances deterministic restoration and perceived quality, characterized by the following steps: Step 1: Construct a paired dataset of degraded images and clean, shadow-free images. and restored dataset ,Will Any sample pair in the middle is denoted as ,in, This represents a degraded image with shadows. This represents a clean image without shadows. This represents an image to be restored, which is currently shaded. Indicates the height of the image. Indicates the width of the image. This represents the number of channels in the image, and ; Step 2: Construct a global regression network It includes: an initial feature extraction layer, a backbone network, and an output projection layer, and will Enter to The forward matrix evolution is performed to output a preliminary deshaded image with temporary illumination anchors. Based on and Constructing a multi-joint loss function Used for backpropagation gradient optimization The parameters, until The training converges, and the optimal global regression model is obtained. ; Step 3, Input into the trained optimal global regression model The forward prediction is performed, and the corresponding coarsely deshaded image is output. ; Step 4: Set the size of the sliding window. The sliding step size is Used for and By performing overlapping block sampling at the same coordinates, a set of regression prior blocks can be obtained. and clean block set ; and will Any prior block in the middle is denoted as ,Will Any clean block in the middle is denoted as ;in, This represents the side length of the sliding window. ; Step 5: Construct a conditional diffusion probability model This includes: conditional noise estimation network and diffusion sampler; in a given first Next, random noise features are introduced. Used for Perform forward diffusion noise addition processing to obtain the first... The noisy latent variable feature block corresponding to the step ;Will and After channel splicing, it is used as the input for rigid control conditions of external strong space. The process is performed to construct the conditional diffusion objective loss function. Used for Perform gradient updates until Convergence is achieved, yielding the optimal conditional diffusion probability model after training. ; Step 6, Input into the trained optimal global regression model The forward prediction is performed, and the corresponding coarsely deshaded image is output. Using the sliding window to The data is trimmed into several test regression prior blocks, and initialized with any test regression prior block. Initial latent feature noise blocks of uniform size ,Will and After channel concatenation, the concatenation is used as an external guiding condition and input into the trained optimal conditional diffusion probability model. The noise level of the overlapping region of the multi-contribution block is reconstructed through inverse denoising to obtain denoised and reconstructed feature blocks. Then, all the denoised and reconstructed feature blocks are spatially seamlessly stitched together to finally output the shadow-removed image. ,in, This indicates the total number of steps.

[0007] The image shadow removal method that balances deterministic restoration and perceived quality as described in this invention is also characterized in that step 2 includes: Step 2.1: Utilize the initial feature extraction layer... Linear channel amplification is performed to output the initial feature map. ,in, express The preset number of standard feature channels; Step 2.2, constructing the backbone network, is... Layer encoder, It consists of a layer decoder and a cross-layer gated jump connection structure. Each layer encoder and decoder includes: a spatial interaction module and a channel evolution module; Step 2.2.1: Initialize the current layer sequence number The global initial feature map As the first Joint input feature map of layers ; Step 2.2.2, will Enter the first In the layer encoder, by the first The spatial interaction module of the layer Perform multi-scale spatial local feature extraction and spatial nonlinear gated modulation to output the first... Spatial interaction feature matrix of layers ,in, , and No. The height, width, and number of channels of the layer feature map; Step 2.2.3, the The channel evolution module of the layer first performs... The channels are then subjected to a high-dimensional linear expansion, followed by the... After performing channel-level matrix multiplication gated weight quantization based on full-graph spatial statistics, the extended features of the layer are obtained. High-dimensional channel evolution characteristics of layers Then, through convolution operations... Perform cross-channel information fusion and output the first... Channel evolution characteristics of the layer Finally, by using downsampling convolution... Perform cross-channel information fusion and nonlinear feature smoothing integration to output the first... Joint input feature map of layers Order No. The space of layer input is ,in , , Therefore, ; Step 2.2.4, Assign to Then, return to step 2.2.2 and repeat the process until... This allows us to obtain the deepest channel evolution characteristics. ; Step 2.2.5, Initialization ,Will As the first Decoding spatial feature map of the layer ; Step 2.2.6: Utilize the upsampling layer to... After performing upsampling mapping that doubles the spatial resolution and halves the number of channels, the input is the first... In the layer decoder, the signal is processed sequentially through a spatial interaction module and a channel evolution module to output the first... Layer decoding spatial feature map ; Step 2.2.7: Calculate the first step using equation (4). Layer-related fusion feature matrix : (4) In equation (4), Indicates the first Cascaded control Convolution parameter matrix, This represents the matrix convolution operation. Sigmoid This represents a sigmoid nonlinear activation function. This represents the element-wise multiplication operation between corresponding matrices; Step 2.2.8, will and By splicing along the channel dimension, we obtain the first... Multi-scale fusion feature map ; Step 2.2.9, will Assign to Then, return to step 2.2.5 and execute sequentially until... Until then, thus obtaining the first Multi-scale fusion feature map , and record as ; Step 2.3: Use the output projection layer to... Matrix dimensionality reduction projection and convolution are performed to obtain a preliminary unshaded image. ; Step 2.4, based on and Construct a multi-joint loss function This includes: Charbonnier pixel reconstruction loss and global color consistency loss Used for The gradient of all convolutional weights within the model is optimized by backpropagation to obtain the trained global regression model.

[0008] Furthermore, step 2.2.2 includes: Step a1, for After layer normalization, depth-separable convolutional layers with progressively smaller degenerate kernel sizes are then applied to the first... The normalized feature maps of the first layer are processed to obtain the first layer. The first intermediate feature matrix of the layer ,in, , and No. The height, width, and number of channels of the layer feature map; Step a2, along the channel dimension Perform tensor partitioning to obtain the first... The first feature branch of the layer and the The second feature branch of the layer ; Step a3: Use equation (1) to obtain the first... Spatial interaction feature matrix of layers : (1) In equation (1), Indicates that the convolution kernel is The weight matrix of the local spatial convolution operation. This represents the matrix convolution operation. This represents element-wise multiplication between corresponding matrices.

[0009] Furthermore, step 2.2.3 includes: Step b1, for Perform convolution kernel as The convolution operation yields the first... Extended feature matrix of the layer : Step b2, along the channel axis dimension Perform tensor partitioning to obtain the first... The first high-dimensional feature of the layer and the The second high-dimensional feature of the layer ; Step b3, using equation (2) Process and calculate the first... The channel bias descriptor vector of the layer with global full-graph spatial statistical properties : (2) In equation (2), express The Middle Line 1 The channel feature vector corresponding to the spatial pixel of the column; Step b4, using equation (3) and Perform channel-by-channel matrix multiplication and weighting to obtain the first... High-dimensional channel evolution characteristics of layers : (3) In equation (3), This represents the channel mapping weight matrix without bias terms. This represents the matrix convolution operation. This represents element-wise multiplication between corresponding matrices; Step b5: Use convolution kernels for Convolution operations on Perform cross-channel information fusion to smoothly integrate nonlinear features while maintaining the same number of feature channels, thereby obtaining the first... Channel evolution characteristics of the layer ; Step b6: Use downsampling convolutional layers to... Sampling is performed to output the first... Joint input feature map of layers .

[0010] Furthermore, step 2.4 includes: Step 2.4.1: Construct the Charbonnier pixel reconstruction loss using equation (5). : (5) In equation (5), express The total number of pixels, for The Middle RGB three-channel prediction vector at each pixel for The Middle The target vector at each pixel. As a smoothing adjustment factor, Represents Euclidean norm operations; Step 2.4.2: Construct the global color consistency loss using equation (6). : (6) In equation (6), The standard deviation is expressed as The low-pass Gaussian filter matrix, Represents matrix convolution. express Norm operations; Step 2.4.3: Construct the multiple joint loss function using equation (7). : + (7) In equation (7), This represents a hyperparameter used to balance the weights of pixel-level fidelity and color consistency.

[0011] Furthermore, step 5 includes: Step 5.1, from The sequence number of the current step is obtained by random sampling. ; Step 5.2, for Proceed to the first The forward diffusion noise processing step generates the first step. The noisy latent variable feature block corresponding to the step ; Step 5.3: Construct the first equation using equation (8). The corresponding conditional diffusion target loss function for each step : (8) In equation (8), Represents the mathematical expectation operator. Represents Euclidean norm operations; This represents standard Gaussian noise.

[0012] Step 5.4, with Minimize as the optimization objective, and use the backpropagation algorithm to... The learning weights are iteratively updated until the maximum number of iterations is reached, thus obtaining the first... The optimized conditional noise estimation network; Step 5.5, return to step 5.1 and execute sequentially until... The optimal conditional diffusion probability model is obtained after training until the preset convergence condition is met. .

[0013] Furthermore, step 6 includes: Step 6.1, Initialization Using the physical coordinates of the sliding window, construct a... Full-image latent feature state matrix with completely uniform size and will Initialize to an initial state tensor filled with standard Gaussian noise; Step 6.2, targeting Any pixel located at the intersection of the physical coverage areas of multiple sliding windows In real time, it locates and collects overlapping data points that include pixels. All sliding block subsets of the position ; Step 6.3, Calculate using equation (9) medium pixel j Multi-block fusion noise estimate at the location : (9) In equation (9), Indicates overlap and coverage of pixels. j The total number of sliding blocks on the top, and They represent the first d The sliding block in the first... The potential feature noise block of the step and the corresponding test regression prior block; Step 6.4, Obtain The multiple noise estimates at each pixel location are combined to form the first... Full-map noise estimation matrix of the step Thus, using equation (10) to Update and calculate the first... Full-graph latent feature state of the step : (10) In equation (10), It is independent and identically distributed random Gaussian characteristic noise. For the first Forward diffusion control parameters, From the initial step to the th The cumulative multiplication and diffusion scheduling parameters of the step. For the reverse process, the first Control variance of the step; Step 6.5, Assign to Then, return to step 6.2 and execute sequentially until... The loop stops when the time is right, thus obtaining the noiseless full graph latent state matrix. Convert to a precisely deshadowed image .

[0014] The present invention provides an electronic device, including a memory and a processor, characterized in that the memory is used to store a program supporting the processor to execute the image deshading method balancing deterministic restoration and perceived quality, and the processor is configured to execute the program stored in the memory.

[0015] The present invention discloses a computer-readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, performs the steps of the image deshading method that balances deterministic restoration and perceived quality.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. By employing a first-stage global regression network, this invention effectively solves the problems of inconsistent global illumination and color deviation that are prone to occur in complex scenes in existing image shadow removal methods, and significantly improves the color stability of the shadow removal area.

[0017] 2. By introducing a block-based conditional diffusion model in the second stage, this invention successfully resolves the inherent conflict between the sensory reconstruction quality and hardware memory consumption in high-resolution image processing using generative priors, and achieves high-quality detail restoration under extremely low computational resource constraints.

[0018] 3. By introducing overlapping block collaborative sampling and pixel-level mean fusion mechanism in the second-stage reverse denoising sampling process, this invention completely eliminates the block effect and boundary discontinuity artifacts caused by block independent reasoning, and greatly enhances the overall sensory visual realism of high-resolution large images.

[0019] 4. This invention organically cascades the two stages of "deterministic regression" and "random generation," taking into account both the accuracy of image restoration in pixel-level metrics and the realism in visual perception, thus opening up a new paradigm for processing high-resolution complex degraded images. Attached Figure Description

[0020] Figure 1 This is a network framework diagram of the first stage of the present invention; Figure 2 This is a structural diagram of the internal gating and attention blocks used in the first stage of the present invention; Figure 3 This is a schematic diagram of the implementation framework of the present invention in the second stage; Figure 4 This is a schematic diagram illustrating the guided sampling used in the second stage of the present invention. Detailed Implementation

[0021] In this embodiment, to accurately lock the global color and illumination of a high-resolution image while perfectly reconstructing local high-sensory-quality details without border artifacts, under limited hardware computing resources and video memory constraints, an image shadow removal method balancing deterministic restoration and perceptual quality is proposed. This method includes: a first-stage global regression model and a second-stage conditional diffusion model. The first-stage global regression model takes the degraded image as input and infers a coarsely shadow-removed restored image. The second-stage conditional diffusion model refines the coarsely shadow-removed restored image to obtain the final perceptually optimized result. Specifically, the method includes the following steps: Step 1: Construct a paired dataset of degraded images and clean, shadow-free images. and restored dataset ,Will Any sample pair in the middle is denoted as ,in, This represents a degraded image with shadows. This represents a clean image without shadows. This represents an image to be restored, which is currently shaded. Indicates the height of the image. Indicates the width of the image. This represents the number of channels in the image, and .

[0022] Step 2, see Figure 1 Construct a global regression network It includes: an initial feature extraction layer, a backbone network, and an output projection layer, and will Enter to The forward matrix evolution is performed to output a preliminary deshaded image with temporary illumination anchors. Based on and Constructing a multi-joint loss function Used for backpropagation gradient optimization The parameters, until The training converges, and the optimal global regression model is obtained. The backbone network comprises multiple feature extraction modules, each of which is constructed by sequentially connecting spatial interaction modules and channel evolution modules. The spatial interaction modules and channel evolution modules internally contain a simplified gating module and a pooling attention module: the simplified gating module performs element-wise multiplication and segmentation of the feature matrix along the channel dimension to achieve non-linear feature mapping without activation functions; the pooling attention module extracts global contextual information through global average pooling and max pooling to generate adaptive attention weight coefficients for channel features.

[0023] Step 2.1: Utilize the initial feature extraction layer... Linear channel amplification is performed to output the initial feature map. ,in, express The preset number of standard feature channels; Step 2.2, constructing the backbone network, is... Layer encoder, It consists of a layer decoder and a cross-layer gated jump connection structure. Each layer encoder and decoder includes: a spatial interaction module and a channel evolution module; Step 2.2.1: Initialize the current layer sequence number The global initial feature map As the first Joint input feature map of layers ; Step 2.2.2, will Enter the first In the layer encoder, by the first The spatial interaction module of the layer Perform multi-scale spatial local feature extraction and spatial nonlinear gated modulation to output the first... Spatial interaction feature matrix of layers ,in, , and No. The height, width, and number of channels of the layer feature map; combined with Figure 2 The upper section provides an instanced explanation of the internal implementation and algorithm flow of the simplified gating module. Instead of using traditional nonlinear activation functions with complex threshold or exponential calculations (such as ReLU, GELU, etc.), the simplified gating module directly utilizes the underlying basic arithmetic dot product unit to perform element-wise dot product of the two sub-feature matrices. This activation function-free nonlinear feature mapping mechanism not only introduces high-performance gating activation characteristics in linear space but also avoids the huge floating-point operation overhead and hardware table lookup time associated with nonlinear activation functions on large-scale, high-resolution feature maps.

[0024] Step a1, for After layer normalization, depth-separable convolutional layers with progressively smaller degenerate kernel sizes are then applied to the first... The normalized feature maps of the first layer are processed to obtain the first layer. The first intermediate feature matrix of the layer ,in, , and No. The height, width, and number of channels of the layer feature map; Step a2, along the channel dimension Perform tensor partitioning to obtain the first... The first feature branch of the layer and the The second feature branch of the layer ; Step a3: Use equation (1) to obtain the first... Spatial interaction feature matrix of layers : (1) In equation (1), Indicates that the convolution kernel is The weight matrix of the local spatial convolution operation. This represents the matrix convolution operation. This represents element-wise multiplication between corresponding matrices.

[0025] Step 2.2.3, the The channel evolution module of the layer first performs... The channels are then subjected to a high-dimensional linear expansion, followed by the... After performing channel-level matrix multiplication gated weight quantization based on full-graph spatial statistics, the extended features of the layer are obtained. High-dimensional channel evolution characteristics of layers Then, through convolution operations... Perform cross-channel information fusion and output the first... Channel evolution characteristics of the layer Finally, by using downsampling convolution... Perform cross-channel information fusion and nonlinear feature smoothing integration to output the first... Joint input feature map of layers Order No. The space of layer input is ,in , , Therefore, To accurately capture the global large-scale illumination contrast difference between shadowed areas and the non-shadowed normal background, the extraction flow of the pooling attention module is as follows: Figure 2 As shown in the lower section. The goal of this module is to capture large-scale global illumination contrast information, allowing the network to adaptively adjust the sensitivity of each channel to shadow and non-shadow areas, thus eliminating color banding.

[0026] Step b1, for Perform convolution kernel as The convolution operation yields the first... Extended feature matrix of the layer : Step b2, along the channel axis dimension Perform tensor partitioning to obtain the first... The first high-dimensional feature of the layer and the The second high-dimensional feature of the layer ; Step b3, using equation (2) Process and calculate the first... The channel bias descriptor vector of the layer with global full-graph spatial statistical properties : (2) In equation (2), express The Middle Line 1 The channel feature vector corresponding to the spatial pixel of the column; Step b4, using equation (3) and Perform channel-by-channel matrix multiplication and weighting to obtain the first... High-dimensional channel evolution characteristics of layers : (3) In equation (3), This represents the channel mapping weight matrix without bias terms. This represents the matrix convolution operation. This represents element-wise multiplication between corresponding matrices; Step b5: Use convolution kernels for Convolution operations on Perform cross-channel information fusion to smoothly integrate nonlinear features while maintaining the same number of feature channels, thereby obtaining the first... Channel evolution characteristics of the layer ; Step b6: Use downsampling convolutional layers to... Sampling is performed to output the first... Joint input feature map of layers .

[0027] Step 2.2.4, Assign to Then, return to step 2.2.2 and repeat the process until... This allows us to obtain the deepest channel evolution characteristics. ; Step 2.2.5, Initialization ,Will As the first Decoding spatial feature map of the layer ; Step 2.2.6: Utilize the upsampling layer to... After performing upsampling mapping that doubles the spatial resolution and halves the number of channels, the input is the first... In the layer decoder, the signal is processed sequentially through a spatial interaction module and a channel evolution module to output the first... Layer decoding spatial feature map ; Step 2.2.7: Calculate the first step using equation (4). Layer-related fusion feature matrix : (4) In equation (4), Indicates the first Cascaded control Convolution parameter matrix, This represents the matrix convolution operation. Sigmoid This represents a sigmoid nonlinear activation function. This represents the element-wise multiplication operation between corresponding matrices; Step 2.2.8, will and By splicing along the channel dimension, we obtain the first... Multi-scale fusion feature map ; Step 2.2.9, will Assign to Then, return to step 2.2.5 and execute sequentially until... Until then, thus obtaining the first Multi-scale fusion feature map , and record as ; Step 2.3: Use the output projection layer to... Matrix dimensionality reduction projection and convolution are performed to obtain a preliminary unshaded image. ; Step 2.4, based on and Construct a multi-joint loss function This includes: Charbonnier pixel reconstruction loss and global color consistency loss Used for The gradient of all convolutional weights within the model is optimized by backpropagation to obtain the trained global regression model.

[0028] Step 2.4.1: Construct the Charbonnier pixel reconstruction loss using equation (5). : (5) In equation (5), express The total number of pixels, for The Middle RGB three-channel prediction vector at each pixel for The Middle The target vector at each pixel. As a smoothing adjustment factor, This represents the Euclidean norm operation.

[0029] Step 2.4.2: Construct the global color consistency loss using equation (6). : (6) In equation (6), The standard deviation is expressed as The low-pass Gaussian filter matrix, Represents matrix convolution. express Norm operations.

[0030] Step 2.4.3: Construct the multiple joint loss function using equation (7). : + (7) In equation (7), This represents a hyperparameter used to balance the weights of pixel-level fidelity and color consistency.

[0031] Step 3, Input into the trained optimal global regression model The forward prediction is performed, and the corresponding coarsely deshaded image is output. .

[0032] Step 4: Set the size of the sliding window. The sliding step size is Used for and By performing overlapping block sampling at the same coordinates, a set of regression prior blocks can be obtained. and clean block set ; and will Any prior block in the middle is denoted as ,Will Any clean block in the middle is denoted as ;in, This represents the side length of the sliding window. .

[0033] Step 5, see Figure 3 Construct a conditional diffusion probability model This includes: conditional noise estimation network and diffusion sampler; in a given first Next, random noise features are introduced. Used for Perform forward diffusion noise addition processing to obtain the first... The noisy latent variable feature block corresponding to the step ;Will and After channel splicing, it is used as the input for rigid control conditions of external strong space. The process is performed to construct the conditional diffusion objective loss function. Used for Perform gradient updates until Convergence is achieved, yielding the optimal conditional diffusion probability model after training. .

[0034] Step 5.1, from The sequence number of the current step is obtained by random sampling. ; Step 5.2, for Proceed to the first The forward diffusion noise processing step generates the first step. The noisy latent variable feature block corresponding to the step ; Step 5.3: Construct the first equation using equation (8). The corresponding conditional diffusion target loss function for each step : (8) In equation (8), Represents the mathematical expectation operator. Represents Euclidean norm operations; Indicates standard Gaussian noise; Step 5.4, with Minimize as the optimization objective, and use the backpropagation algorithm to... The learning weights are iteratively updated until the maximum number of iterations is reached, thus obtaining the first... The optimized conditional noise estimation network.

[0035] Step 5.5, return to step 5.1 and execute sequentially until... The optimal conditional diffusion probability model is obtained after training until the preset convergence condition is met. .

[0036] Step 6, Input into the trained optimal global regression model The forward prediction is performed, and the corresponding coarsely deshaded image is output. Using the sliding window to The data is trimmed into several test regression prior blocks, and initialized with any test regression prior block. Initial latent feature noise blocks of uniform size ,Will and After channel concatenation, the concatenation is used as an external guiding condition and input into the trained optimal conditional diffusion probability model. The noise level of the overlapping region of the multi-contribution block is reconstructed through inverse denoising to obtain denoised and reconstructed feature blocks. Then, all the denoised and reconstructed feature blocks are spatially seamlessly stitched together to finally output the shadow-removed image. ,in, This indicates the total number of steps.

[0037] Step 6.1, Initialization Using the physical coordinates of the sliding window, construct a... Full-image latent feature state matrix with completely uniform size and will Initialize to an initial state tensor filled with standard Gaussian noise; this step is performed at each discrete sampling time step. The internal function is dynamically invoked. This is due to the initial full-map latent feature state before inference begins. It is initialized directly with a pure Gaussian white noise matrix at full image resolution, therefore, in each subsequent loop step, the same sliding grid step size is used. and size From the current full map status Cut out the first The sliding block in the first... Potential characteristic noise block of the step It can establish a completely locked spatial coordinate mapping bus between the micro-level segmented latent variables and the macro-level full-image matrix canvas, thus eliminating spatial texture misalignment and image distortion that are prone to occur in the long-cycle denoising iteration of the generative model from the source of the algorithm.

[0038] Step 6.2, targeting Any pixel located at the intersection of the physical coverage areas of multiple sliding windows In real time, it locates and collects overlapping data points that include pixels. All sliding block subsets of the position ; Step 6.3, Calculate using equation (9) medium pixel j Multi-block fusion noise estimate at the location : (9) In equation (9), Indicates overlap and coverage of pixels. j The total number of sliding blocks on the top, and They represent the first d The sliding block in the first... The potential feature noise block of the step and the corresponding test regression prior block.

[0039] Combination Figure 4 The collaborative sampling noise level fusion mechanism shown, in specific implementation, involves noisy latent variable blocks. With spatial regression prior block In the input conditional denoising network First, a cascaded splicing process is performed along the channel axis, expanding and merging the 3-channel feature tensor into a 6-channel combined feature tensor, thus laying the foundation for globally deterministic illumination background constraints for the conditional denoising network. Next, the mean of the predicted noise features for all covered blocks is calculated using an arithmetic mean unit, and then directly assembled and reconstructed into a full-image noise matrix. This can forcibly align the originally isolated reverse evolution trajectories of each block with the transition surfaces of the continuous manifold, making their distribution adaptively converge, thereby fundamentally eliminating the physical block effect and splicing gap artifacts that are inevitably caused by traditional block-based reasoning.

[0040] Step 6.4, Obtain The multiple noise estimates at each pixel location are combined to form the first... Full-map noise estimation matrix of the step Thus, using equation (10) to Update and calculate the first... Full-graph latent feature state of the step : (10) In equation (10), It is independent and identically distributed random Gaussian characteristic noise. For the first Forward diffusion control parameters, From the initial step to the th The cumulative multiplication and diffusion scheduling parameters of the step. For the reverse process, the first Control variance of the step.

[0041] Step 6.5, Assign to Then, return to step 6.2 and execute sequentially until... The loop stops when the time is right, thus obtaining the noiseless full graph latent state matrix. Convert to a precisely deshadowed image .

[0042] In this embodiment, an electronic device includes a memory and a processor. The memory stores a program that supports the processor in executing the above-described method, and the processor is configured to execute the program stored in the memory.

[0043] In this embodiment, a computer-readable storage medium stores a computer program, which is executed by a processor to perform the steps of the above method.

[0044] To verify the effectiveness of this method, the commonly used high-resolution shadow public dataset WSRD is used. The WSRD dataset contains 1000 training images, 100 validation images, and 75 test images, each a high-resolution image of 1440x1920. Table 1 evaluates the quantitative results of the proposed method and the comparison methods on the high-resolution shadow removal task. Methods that have performed well in the past are used in the table: Mask-ShadowNet represents an image shadow removal method based on a mask-embedded adversarial network; SwinIR represents a general image restoration method based on a moving window Transformer; ShuffleFormer represents an image shadow removal method enhanced by a channel shuffling mechanism; ShadowFormer represents a context-guided image shadow removal method based on a dual-space Transformer; ShadowDiffusion represents an image shadow removal method based on a single-stage diffusion model; Refusion represents an image restoration method based on a diffusion model of coarse-to-fine reconstruction; and MDFormer represents an image shadow removal method based on a multi-channel decoupled architecture. PSNR represents the peak signal-to-noise ratio, SSIM represents the structural similarity index, LPIPS represents the learned perceptual image proximity, and MOS represents the mean opinion score.

[0045] Table 1. Quantitative results of the method of the present invention and the comparative method on high-resolution shadow removal tasks.

[0046] As can be clearly seen from Table 1, compared with other shadow removal methods, the method proposed in this invention effectively improves subjective perception and outperforms previous supervised learning and generative model methods in terms of objective indicators.

Claims

1. An image shadow removal method that balances deterministic restoration and perceived quality, characterized in that, Includes the following steps: Step 1: Construct a paired dataset of degraded images and clean, shadow-free images. and restored dataset ,Will Any sample pair in the middle is denoted as ,in, This represents a degraded image with shadows. This represents a clean image without shadows. This represents an image to be restored, which is currently shaded. Indicates the height of the image. Indicates the width of the image. This represents the number of channels in the image, and ; Step 2: Construct a global regression network It includes: an initial feature extraction layer, a backbone network, and an output projection layer, and will Enter to The forward matrix evolution is performed to output a preliminary deshaded image with temporary illumination anchors. Based on and Constructing a multi-joint loss function Used for backpropagation gradient optimization The parameters, until The training converges, and the optimal global regression model is obtained. ; Step 3, Input into the trained optimal global regression model The forward prediction is performed, and the corresponding coarsely deshaded image is output. ; Step 4: Set the size of the sliding window. The sliding step size is Used for and By performing overlapping block sampling at the same coordinates, a set of regression prior blocks can be obtained. and clean block set ; and will Any prior block in the middle is denoted as ,Will Any clean block in the middle is denoted as ;in, This represents the side length of the sliding window. ; Step 5: Construct a conditional diffusion probability model This includes: conditional noise estimation network and diffusion sampler; in a given first Next, random noise features are introduced. Used for Perform forward diffusion noise addition processing to obtain the first... The noisy latent variable feature block corresponding to the step ;Will and After channel splicing, it is used as the input for rigid control conditions of external strong space. The process is performed to construct the conditional diffusion objective loss function. Used for Perform gradient updates until Convergence is achieved, yielding the optimal conditional diffusion probability model after training. ; Step 6, Input into the trained optimal global regression model The forward prediction is performed, and the corresponding coarsely deshaded image is output. Using the sliding window to The data is trimmed into several test regression prior blocks, and initialized with any test regression prior block. Initial latent feature noise blocks of uniform size ,Will and After channel concatenation, the concatenation is used as an external guiding condition and input into the trained optimal conditional diffusion probability model. The noise level of the overlapping region of the multi-contribution block is reconstructed through inverse denoising to obtain denoised and reconstructed feature blocks. Then, all the denoised and reconstructed feature blocks are spatially seamlessly stitched together to finally output the shadow-removed image. ,in, This indicates the total number of steps.

2. The image shadow removal method balancing deterministic restoration and perceived quality according to claim 1, characterized in that, Step 2 includes: Step 2.1: Utilize the initial feature extraction layer... Linear channel amplification is performed to output the initial feature map. ,in, express The preset number of standard feature channels; Step 2.2, constructing the backbone network, is... Layer encoder, It consists of a layer decoder and a cross-layer gated jump connection structure. Each layer encoder and decoder includes: a spatial interaction module and a channel evolution module; Step 2.2.1: Initialize the current layer sequence number The global initial feature map As the first Joint input feature map of layers ; Step 2.2.2, will Enter the first In the layer encoder, by the first The spatial interaction module of the layer Perform multi-scale spatial local feature extraction and spatial nonlinear gated modulation to output the first... Spatial interaction feature matrix of layers ,in, , and No. The height, width, and number of channels of the layer feature map; Step 2.2.3, the The channel evolution module of the layer first performs... The channels are then subjected to a high-dimensional linear expansion, followed by the... After performing channel-level matrix multiplication gated weight quantization based on full-graph spatial statistics, the extended features of the layer are obtained. High-dimensional channel evolution characteristics of layers Then, through convolution operations... Perform cross-channel information fusion and output the first... Channel evolution characteristics of the layer Finally, by using downsampling convolution... Perform cross-channel information fusion and nonlinear feature smoothing integration to output the first... Joint input feature map of layers Order No. The space of layer input is ,in , , Therefore, ; Step 2.2.4, Assign to Then, return to step 2.2.2 and repeat the process until... This allows us to obtain the deepest channel evolution characteristics. ; Step 2.2.5, Initialization ,Will As the first Decoding spatial feature map of the layer ; Step 2.2.6: Utilize the upsampling layer to... After performing upsampling mapping that doubles the spatial resolution and halves the number of channels, the input is the first... In the layer decoder, the signal is processed sequentially through a spatial interaction module and a channel evolution module to output the first... Layer decoding spatial feature map ; Step 2.2.7: Calculate the first step using equation (4). Layer-related fusion feature matrix : (4) In equation (4), Indicates the first Cascaded control Convolution parameter matrix, This represents the matrix convolution operation. Sigmoid This represents a sigmoid nonlinear activation function. This represents the element-wise multiplication operation between corresponding matrices; Step 2.2.8, will and By splicing along the channel dimension, we obtain the first... Multi-scale fusion feature map ; Step 2.2.9, will Assign to Then, return to step 2.2.5 and execute sequentially until... Until then, thus obtaining the first Multi-scale fusion feature map , and record as ; Step 2.3: Use the output projection layer to... Matrix dimensionality reduction projection and convolution are performed to obtain a preliminary unshaded image. ; Step 2.4, based on and Construct a multi-joint loss function This includes: Charbonnier pixel reconstruction loss and global color consistency loss Used for The gradient of all convolutional weights within the model is optimized by backpropagation to obtain the trained global regression model.

3. The image shadow removal method balancing deterministic restoration and perceived quality according to claim 2, characterized in that, Step 2.2.2 includes: Step a1, for After layer normalization, depth-separable convolutional layers with progressively smaller degenerate kernel sizes are then applied to the first... The normalized feature maps of the first layer are processed to obtain the first layer. The first intermediate feature matrix of the layer ,in, , and No. The height, width, and number of channels of the layer feature map; Step a2, along the channel dimension Perform tensor partitioning to obtain the first... The first feature branch of the layer and the The second feature branch of the layer ; Step a3: Use equation (1) to obtain the first... Spatial interaction feature matrix of layers : (1) In equation (1), Indicates that the convolution kernel is The weight matrix of the local spatial convolution operation. This represents the matrix convolution operation. This represents element-wise multiplication between corresponding matrices.

4. The image shadow removal method balancing deterministic restoration and perceived quality according to claim 2, characterized in that, Step 2.2.3 includes: Step b1, for Perform convolution kernel as The convolution operation yields the first... Extended feature matrix of the layer : Step b2, along the channel axis dimension Perform tensor partitioning to obtain the first... The first high-dimensional feature of the layer and the The second high-dimensional feature of the layer ; Step b3, using equation (2) Process and calculate the first... The channel bias descriptor vector of the layer with global full-graph spatial statistical properties : (2) In equation (2), express The Middle Line 1 The channel feature vector corresponding to the spatial pixel of the column; Step b4, using equation (3) and Perform channel-by-channel matrix multiplication and weighting to obtain the first... High-dimensional channel evolution characteristics of layers : (3) In equation (3), This represents the channel mapping weight matrix without bias terms. This represents the matrix convolution operation. This represents element-wise multiplication between corresponding matrices; Step b5: Use convolution kernels for Convolution operations on Perform cross-channel information fusion to smoothly integrate nonlinear features while maintaining the same number of feature channels, thereby obtaining the first... Channel evolution characteristics of the layer ; Step b6: Use downsampling convolutional layers to... Sampling is performed to output the first... Joint input feature map of layers .

5. The image shadow removal method balancing deterministic restoration and perceived quality according to claim 2, characterized in that, Step 2.4 includes: Step 2.4.1: Construct the Charbonnier pixel reconstruction loss using equation (5). : (5) In equation (5), express The total number of pixels, for The Middle RGB three-channel prediction vector at each pixel for The Middle The target vector at each pixel. As a smoothing adjustment factor, Represents Euclidean norm operations; Step 2.4.2: Construct the global color consistency loss using equation (6). : (6) In equation (6), The standard deviation is expressed as The low-pass Gaussian filter matrix, Represents matrix convolution. express Norm operations; Step 2.4.3: Construct the multiple joint loss function using equation (7). : + (7) In equation (7), This represents a hyperparameter used to balance the weights of pixel-level fidelity and color consistency.

6. The image shadow removal method balancing deterministic restoration and perceived quality according to claim 1, characterized in that, Step 5 includes: Step 5.1, from The sequence number of the current step is obtained by random sampling. ; Step 5.2, for Proceed to the first The forward diffusion noise processing step generates the first step. The noisy latent variable feature block corresponding to the step ; Step 5.3: Construct the first equation using equation (8). The corresponding conditional diffusion target loss function for each step : (8) In equation (8), Represents the mathematical expectation operator. Represents Euclidean norm operations; Indicates standard Gaussian noise; Step 5.4, with Minimize as the optimization objective, and use the backpropagation algorithm to... The learning weights are iteratively updated until the maximum number of iterations is reached, thus obtaining the first... The optimized conditional noise estimation network; Step 5.5, return to step 5.1 and execute sequentially until... The optimal conditional diffusion probability model is obtained after training until the preset convergence condition is met. .

7. The image shadow removal method balancing deterministic restoration and perceived quality according to claim 1, characterized in that, Step 6 includes: Step 6.1, Initialization Using the physical coordinates of the sliding window, construct a... Full-image latent feature state matrix with completely uniform size and will Initialize to an initial state tensor filled with standard Gaussian noise; Step 6.2, targeting Any pixel located at the intersection of the physical coverage areas of multiple sliding windows In real time, it locates and collects overlapping data points that include pixels. All sliding block subsets of the position ; Step 6.3, Calculate using equation (9) medium pixel j Multi-block fusion noise estimate at the location : (9) In equation (9), Indicates overlap and coverage of pixels. j The total number of sliding blocks on the top, and They represent the first d The sliding block in the first... The potential feature noise block of the step and the corresponding test regression prior block; Step 6.4, Obtain The multiple noise estimates at each pixel location are combined to form the first... Full-map noise estimation matrix of the step Thus, using equation (10) to Update and calculate the first... Full-graph latent feature state of the step : (10) In equation (10), It is independent and identically distributed random Gaussian characteristic noise. For the first Forward diffusion control parameters, From the initial step to the th The cumulative multiplication and diffusion scheduling parameters of the step. For the reverse process, the first Control variance of the step; Step 6.5, Assign to Then, return to step 6.2 and execute sequentially until... The loop stops when the time is right, thus obtaining the noiseless full graph latent state matrix. Convert to a precisely deshadowed image .

8. An electronic device, comprising a memory and a processor, characterized in that, The memory is used to store a program that supports a processor in executing the image deshading method for balancing deterministic restoration and perceived quality as described in any one of claims 1-7, and the processor is configured to execute the program stored in the memory.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is run by the processor, it performs the steps of the image deshading method that balances deterministic restoration and perceived quality as described in any one of claims 1-7.

Images