Document image shadow elimination method based on pixel adaptive shadow illumination residual model

Abstract

The invention discloses a document image shadow elimination method based on a pixel adaptive shadow illumination residual model, relates to the technical field of computer vision and image processing, and provides a novel shadow elimination framework based on combined guidance of a background image and a difference image. And a shadow-illumination residual error model is innovatively constructed. The model collaboratively optimizes a shadow removal process through a double-branch architecture: an illumination correction branch explicitly decouples a shadow illumination effect, and a residual learning branch adaptively separates a content residual, so that fine-grained textures and text details are effectively reserved while the shadow is eliminated. In a public data set test, PSNR (peak signal-to-noise ratio), SSIM (structural similarity) and RMSE (root-mean-square) indexes are superior to those of an existing method.

Description

Technical Field

[0001] This invention relates to the field of computer vision and image processing technology, and more specifically, to a method for eliminating shadows in document images based on a pixel-adaptive shadow and illumination residual model. Background Technology

[0002] In daily work and life, various documents such as textbooks, newspapers, brochures, and receipts often need to be digitized for electronic archiving or online sharing. With the widespread use and portability of smartphones, mobile devices have become the primary tool for capturing document images. However, when light is obstructed during the shooting process, shadowed areas often appear in the resulting images. These low-brightness areas significantly reduce image clarity and affect text recognition accuracy. Eliminating document shadows is crucial for improving text readability and accessibility, especially in applications such as document analysis, optical character recognition (OCR), and assistive technologies for the visually impaired.

[0003] Existing shadow removal methods are mainly divided into two categories: methods based on traditional optimization algorithms and methods based on deep learning. Traditional optimization methods are usually based on physically based lighting models to build algorithms, but because they make overly idealistic assumptions about lighting conditions, they are difficult to accurately depict the complex shadow changes in real scenes, and have obvious limitations in practical applications.

[0004] The emergence of deep learning methods overcomes the dependence on physical model assumptions in traditional methods, achieving shadow removal through end-to-end feature learning. Existing deep learning methods primarily utilize document background color as guiding information for shadow removal, but this approach is prone to artifacts, text blurring, and distortion of background texture and text. The root cause lies in the bias in the estimation of shadowless backgrounds by existing methods, which directly affects the final shadow removal effect. Some recent research attempts to generate background images using filtering as guiding information. While this can more accurately reflect background information and shadow features, colored shadows in the background may still be misidentified by the network as background color, leading to incorrect inferences.

[0005] Early studies typically modeled the relationship between shadowed and non-shadowed images as a linear transformation. However, this model requires globally uniform illumination in the shadowed region, making it difficult to adapt to scenes with non-uniform shadows or multiple light source interference. Although subsequent studies proposed pixel-adaptive shadow relighting models that predict relighting coefficients using deep networks, directly predicting illumination adjustment parameters is a severely ill-posed problem. This can lead to the optimization process failing to converge to a reasonable solution and can easily destroy high-frequency structural details such as text strokes.

[0006] In view of this, the present invention proposes a document image shadow removal method based on a pixel adaptive shadow lighting residual model. Summary of the Invention

[0007] The above-mentioned technical objective of the present invention is achieved through the following technical solution: The first aspect of the present invention provides a document image shadow removal method based on a pixel adaptive shadow lighting residual model, comprising the following steps: acquiring an original image with shadows, using a filtering operator to generate a background image and a difference map between the shadow image and the background, performing multi-scale extraction on the original image, background image and difference map to obtain three view features, and fusing the three view features into a decoder to generate a shadow mask probability map. The shadow probability map is processed by feature encoding to obtain the encoded features; the shadow image and the difference map are input into the feature fusion model enhanced by fast Fourier transform, and the FFT-processed features are output; the encoded features and the FFT-processed features are fused to obtain the enhanced feature representation, and the enhanced feature representation is input into the decoder to output the illumination residual coefficients. Obtain the trained shadow and illumination residual model, input the shadow mask probability map and illumination residual coefficients into the trained shadow and illumination residual model, and output a shadowless image.

[0008] In conjunction with the first aspect, the present invention is further configured such that the shadow lighting residual model is: ; In the formula, Image without shadows This is a probability map of the shadow mask. The shadow image of the masked area. and It is the pixel in each shadow area The corresponding spatial variation in illumination coefficient.

[0009] In conjunction with the first aspect, the present invention is further configured such that the coefficient and The value is automatically adjusted through the following mechanism: ; ; ; in, and This is the residual correction amount. This is the brightness adjustment amount. For pixels The original brightness adjustment prediction value is unconstrained.

[0010] In conjunction with the first aspect, the present invention is further configured such that the multi-scale extraction employs ResNet-18.

[0011] In conjunction with the first aspect, the present invention is further configured such that: the trained shadow and illumination residual model dynamically selects training images using a random sampling strategy, and terminates the training process when the loss function tends to stabilize and no longer decreases significantly.

[0012] In conjunction with the first aspect, the present invention is further configured such that the loss function is: ; in, , This results in a loss of appearance consistency. To counter the loss function, For gradient smoothing loss, For the identity loss in the non-shaded region, For shadow consistency loss, For regional similarity loss, This is due to cross-channel consistency loss.

[0013] In conjunction with the first aspect, the present invention is further configured such that: the step of inputting the shadow image and the difference map into the feature fusion model enhanced by Fast Fourier Transform to obtain the enhanced feature representation includes processing the shadow image and the difference map by FFT and outputting the FFT-processed features; and fusing the encoded features and the FFT-processed features to obtain the enhanced feature representation.

[0014] A second aspect of the present invention also provides an apparatus / device / system for document image shadow removal based on a pixel-adaptive shadow lighting residual model, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above method.

[0015] A third aspect of the present invention also provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0016] A fourth aspect of the present invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0017] In summary, the present invention has the following beneficial effects: This invention first proposes a lighting and shadow residual model. The residual learning method effectively avoids excessive modification of background information by accurately decoupling shadows from background content (rather than directly predicting a clean image). This method can accurately separate shadow transparency changes (such as the brightness attenuation coefficient of shadow areas) while preserving the original structure of the strokes in the printed text. Secondly, it is found that the difference map between the shadow image and the background image can effectively eliminate most of the shadow information while retaining the main text content. By using both the difference map and the background image as guiding inputs to the network, the system can accurately identify shadow areas and extract text features from them, thus achieving high-precision shadow removal and text preservation. This method demonstrates significant effectiveness and good generalization ability in document image shadow processing in real-world scenarios. Attached Figure Description

[0018] Figure 1 This is a flowchart of a document image shadow removal method based on a pixel adaptive shadow lighting residual model in Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of a feature fusion model enhanced by fast Fourier transform in Embodiment 1 of the present invention; Figure 3 This is a visual comparison result between the present invention and the prior art in Embodiment 1 of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Example 1: Existing research has established a fundamental shadow removal paradigm that uses a linear illumination adjustment model to remove shadows from the masked area. Convert to a shadowless image : in, This represents the pixel coordinates within the shaded area. and These represent global illumination correction parameters, which remain constant across all shadow pixels. However, this employs a constant relighting coefficient. and The linear model can only handle uniformly lit shadows and performs poorly when faced with non-uniform shadows. To overcome this limitation, Self-ShadowGAN proposes a pixel-adaptive shadow relighting model: in, and It is the pixel in each shadow area The corresponding spatial variation relighting coefficient is designed to accommodate complex shadows with non-uniform lighting characteristics. To ensure the integrity of shadowless areas during relighting, the original shadowless pixels... and The values ​​are fixed at 1 and 0 respectively.

[0021] Accurate detection of shadow areas remains a technical challenge, and imperfect detection can significantly affect shadow removal results. The coefficient for non-shadow areas is fixed at [value missing]. and There are three main problems: First, shadow boundaries in real-world scenes often have gradual transitions rather than sharp edges, which makes binary classification inherently inaccurate; second, the lighting conditions in non-shadow areas may still need to be finely adjusted to maintain luminosity consistency with the processed shadow areas; third, detection errors inevitably propagate during relighting, leading to incomplete shadow removal or overcorrection of non-shadow areas.

[0022] To address this, the present invention proposes a unified framework for jointly optimizing shadow detection and illumination correction (shadow-illumination residual model): in, This represents a probability map of the shadow mask, rather than a binary mask. This flexible modeling has three major advantages: (1) it naturally handles the penumbra region through probability weighting; (2) it allows for partial correction of potential missed detections; and (3) it maintains the differentiability of end-to-end optimization.

[0023] A document image shadow removal method based on a pixel-adaptive shadow illumination residual model includes the following steps: S100. Obtain the original image with shadows, use filtering operators to generate a background image and a difference map between the shadow image and the background, extract the original image, background image and difference map at multiple scales to obtain three view features, fuse the three view features and input them into the decoder to generate a shadow mask probability map. S200. Perform feature encoding on the shadow probability map to obtain the encoded features; input the shadow image and difference map into the feature fusion model enhanced by Fast Fourier Transform (FFT) to output the FFT-processed features; fuse the encoded features and the FFT-processed features to obtain the enhanced feature representation; input the enhanced feature representation into the decoder to output the illumination residual coefficients. S300. Obtain the trained shadow and illumination residual model, input the shadow mask probability map and illumination residual coefficients into the trained shadow and illumination residual model, and output a shadowless image.

[0024] In step S100 of this embodiment, the multi-scale extraction uses ResNet-18.

[0025] In step S200 of this embodiment, inputting the shadow image and the difference map into the Fast Fourier Transform enhanced feature fusion model to obtain the enhanced feature representation includes processing the shadow image and the difference map through FFT and outputting the FFT-processed features; fusing the encoded features and the FFT-processed features to obtain the enhanced feature representation.

[0026] The difference map between the shadow image and the background image can effectively eliminate most of the shadow information while retaining the main text content. By using both the difference map and the background image as guiding inputs to the network, the system can accurately identify shadow areas and extract text features from them, thus achieving high-precision shadow removal and text fidelity preservation.

[0027] In the shadow removal decoding process, Fast Fourier Transform (FFT) of both the shadow image and the difference image is introduced to supplement global features. By leveraging the global representation capability of the Fourier Transform, both local texture details (such as character strokes) and global structural features (such as page layout) in the document image can be modeled simultaneously. Specifically, the amplitude spectrum mainly captures high-frequency text edge features, while the phase spectrum preserves the geometric topology of the document. This dual-spectral collaboration mechanism effectively alleviates content distortion during shadow removal and significantly improves the preservation of text region integrity.

[0028] In this embodiment, a Fast Fourier Transform-enhanced feature fusion model is proposed. For example... Figure 2 As shown, the specific implementation process is as follows: First, a Fast Fourier Transform (FFT) is performed on the shadow image, and then the transform result is downsampled to different scale levels. After extracting features through a multi-layer convolutional network, these features are concatenated with the encoder features. Subsequently, a Convolutional Attention Module (CBAM) is used to simultaneously extract channel attention and spatial attention features. Finally, the processed features are connected to the decoder. The same process is used for feature processing of difference images.

[0029] In step S300 of this embodiment, the shadow lighting residual model is: ; In the formula, Image without shadows This is a probability map of the shadow mask. The shadow image of the masked area. and It is the pixel in each shadow area The corresponding spatial variation in illumination coefficient.

[0030] The coefficient and The value is automatically adjusted through the following mechanism: ; ; ; in, and This is the residual correction amount. This is the brightness adjustment amount. For pixels The original brightness adjustment prediction value is unconstrained.

[0031] The trained shadow and illumination residual model uses a random sampling strategy to dynamically select training images. The training process is terminated when the loss function tends to stabilize and no longer decreases significantly.

[0032] The loss function is: ; in, , This results in a loss of appearance consistency. To counter the loss function, For gradient smoothing loss, For the identity loss in the non-shaded region, For shadow consistency loss, For regional similarity loss, This is due to cross-channel consistency loss.

[0033] During training, the network model is optimized by imposing constraints on the following four aspects: (1) Shadow mask probability (2) The learned residual correction coefficients and (3) Illumination coefficient and , and (4) shadow removal image.

[0034] The specific constraints are as follows: Appearance consistency loss: calculated by predicting shadowless images With real images The network is constrained by the pixel-wise 1-norm distance between them. ; Adversarial loss function: It not only evaluates the absolute discriminant scores between the real and output images, but also considers their relative differences. ; in, Let represent the Sigmoid activation function, BCE represent the binary cross-entropy, and g / d represent the training objective (g=1, d=0 represents the generator update, g=0, d=1 represents the discriminator update). This relative evaluation mechanism enhances the model's ability to discern subtle quality differences, thereby helping to generate more realistic images.

[0035] Gradient smoothing loss: ensuring shadow probability mask and illumination coefficient and Spatial transition smoothness, applying gradient smoothing constraints: ; in This represents the spatial gradient operator.

[0036] Non-shade area identity loss: To maintain color fidelity in non-shade areas, when the shadow probability... Approaching zero ( When applying identity constraints: ; Shadow consistency loss: To prevent excessive enhancement of illuminance levels in shadowed areas beyond those in unshaded areas, a secondary penalty term is introduced. Region similarity loss: To achieve spatially coherent shadow elimination within uniform regions while preserving natural boundaries, a novel color-guided similarity constraint is proposed in the brightness adjustment field: ; in express Pixel Adjacent pixels, , The index function is: ; The principle behind this loss function is that adjacent pixels with similar colors should undergo similar illumination correction.

[0037] Cross-channel consistency loss: To ensure consistent adjustment across RGB channels and avoid color distortion, we employ a covariance-based penalty term: ; in, This indicates the RGB channel.

[0038] In tests on public datasets, the PSNR (Peak Signal-to-Noise Ratio), SSIM (Structural Similarity), and RMSE (Root Mean Square) metrics outperform existing methods. Specifically: The neural network of this invention is implemented based on the PyTorch framework and trained for 200 epochs on an NVIDIA GeForce RTX 4080 GPU. The optimizer used is Adam with a learning rate of 0.0001. Input images were uniformly adjusted to a resolution of 512×512 pixels, and the batch size was set to 4 to balance memory efficiency and gradient update stability. To enhance model robustness and prevent overfitting, this invention employs data augmentation strategies such as random rotation and cropping.

[0039] This invention uses the SD7K and RDD datasets as training and testing benchmarks for its proposed method and comparative methods. These two datasets are currently publicly available large-scale document shadow removal datasets, each containing pairs of high-resolution document images. Specifically, the SD7K dataset contains 6,479 training samples and 760 test samples, while the RDD dataset consists of 4,371 training samples and 545 test samples. In the experiments, 4,000 samples were randomly selected from each training set for model training, and the model performance was evaluated on both test sets.

[0040] To verify the effectiveness of the proposed method, the method provided by this invention was comprehensively compared with advanced methods in two major fields: (1) seven document image shadow removal methods (Water-Filling, Liu, BEDSR-Net, BGShadowNet, DocShadow, DocRes and Liu2025); (2) six natural image shadow removal techniques (ShadowDiffusion, Omnisr, Des3, Diff-Shadow, Le and Self-ShadowGAN).

[0041] To ensure fairness, all learning-based methods were trained using the same hardware configuration and RDD and SDK datasets. Evaluation metrics were uniformly calculated based on the output results at 512×512 resolution. As shown in the table below, our method achieves state-of-the-art performance across all evaluation metrics—PSNR (Peak Signal-to-Noise Ratio), SSIM (Structural Similarity), and RMSE (Root Mean Square)—significantly outperforming existing methods.

[0042] Visual comparison results with existing methods on the RDD dataset are as follows: Figure 3As shown: (a) Input image, (b) Real image without shadow, (c) Our results, (d) BGShadowNet, (e) DocShadow, (f) DocRes, and (g) Liu2025. Except for Self-ShadowGAN (a method based on self-supervised learning of a single image, requiring no labeled data), all other comparison methods were trained using the same dataset. During the experiments, we used document images containing shadows (… Figure 3 a) Input the data into the pre-trained model and output the shadow removal result. Visual comparisons show that the proposed method not only accurately detects shadow areas and effectively eliminates shadows, but also better preserves the integrity of background texture, color consistency, and text details.

[0043] Example 2: The present invention also provides a device / equipment / system for document image shadow removal based on a pixel adaptive shadow lighting residual model, comprising a memory, a processor, and a computer program stored in the memory, characterized in that the processor executes the computer program to implement the steps of the above method.

[0044] The present invention also provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0045] The present invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0046] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A document image shadow removal method based on a pixel-adaptive shadow illumination residual model, characterized by: The process includes the following steps: acquiring the original image with shadows, using filtering operators to generate a background image and a difference map between the shadow image and the background, extracting the original image, background image and difference map at multiple scales to obtain three view features, and fusing the three view features into the decoder to generate a shadow mask probability map. The shadow probability map is subjected to feature encoding to obtain the encoded features; The shadow image and difference map are input into the feature fusion model enhanced by Fast Fourier Transform (FFT), and the FFT-processed features are output. The encoded features and the FFT-processed features are fused to obtain the enhanced feature representation. The enhanced feature representation is input into the decoder and the illumination residual coefficients are output. Obtain the trained shadow and lighting residual model, input the shadow mask probability map and lighting residual coefficients into the trained shadow and lighting residual model, and output a shadowless document image.

2. The document image shadow removal method according to claim 1, characterized in that: The shadow lighting residual model is as follows: ； In the formula, Image without shadows This is a probability map of the shadow mask. The shadow image of the masked area. and It is the pixel in each shadow area The corresponding spatial variation in illumination coefficient.

3. The document image shadow removal method according to claim 1, characterized in that: The coefficient and The value is automatically adjusted through the following mechanism: ； ； ； in, and This is the residual correction amount. This is the brightness adjustment amount. For pixels The original brightness adjustment prediction value is unconstrained.

4. The document image shadow removal method according to claim 1, characterized in that: The multi-scale extraction uses ResNet-18.

5. The document image shadow removal method according to claim 1, characterized in that: The trained shadow and illumination residual model uses a random sampling strategy to dynamically select training images. The training process is terminated when the loss function tends to stabilize and no longer decreases significantly.

6. The document image shadow removal method based on a pixel-adaptive shadow illumination residual model according to claim 5, characterized in that: The loss function is: ； in, , This results in a loss of appearance consistency. To counteract the loss function, For gradient smoothing loss, For the identity loss in the non-shaded region, For shadow consistency loss, For regional similarity loss, This is due to cross-channel consistency loss.

7. The document image shadow removal method based on a pixel-adaptive shadow illumination residual model according to claim 1, characterized in that: The shadow image and difference map are input into the feature fusion model enhanced by Fast Fourier Transform to obtain an enhanced feature representation. This includes processing the shadow image and difference map using FFT to output the FFT-processed features; The encoded features and the features processed by FFT are fused to obtain an enhanced feature representation.

8. An apparatus / device / system for a document image shadow removal method based on a pixel-adaptive shadow lighting residual model, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-7.

9. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method described in any one of claims 1-7.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method described in any one of claims 1-7.

Images