A method and system for enhancing low-light document images based on 3D rendering and document structure guidance

Abstract

This invention provides a method and system for enhancing low-light document images based on 3D rendering and document structure guidance. It includes: constructing paired training samples of low-light document images and normally lit document images based on a scanned document image dataset; training a low-light document image enhancement model based on the paired training samples; acquiring the low-light document image to be enhanced; inputting the low-light document image to be enhanced into the low-light document image enhancement model, extracting features and outputting an intermediate enhanced image; outputting a document structure prediction map based on the intermediate enhanced image; and having each layer of the decoder in the low-light enhancement backbone network scale-adaptively adjust the document structure prediction map and then fuse it with encoded features through a cross-attention module to reconstruct the enhanced document image. This invention combines 3D graphics rendering technology with deep learning, significantly reducing data acquisition costs and greatly improving the readability and OCR recognition accuracy of the enhanced document through a multi-level document structure perception mechanism.

Description

Technical Field

[0001] This invention relates to the field of computer vision and image processing technology, and in particular to a method, system, terminal device, and computer-readable storage medium for enhancing low-light document images based on 3D rendering and document structure guidance. Background Technology

[0002] With the widespread adoption of mobile smart terminals and the deepening of office digitalization, using smartphones, tablets, and other devices to photograph documents has become an important method of information collection, widely used in scenarios such as meeting minutes, contract management, invoice processing, and mobile office work. However, due to the uncontrollable nature of lighting conditions during shooting, especially in low-light indoor environments, nighttime scenes, or backlit conditions, the acquired document images often suffer from problems such as severely low overall brightness, insufficient contrast, and blurry text, seriously affecting the accuracy and information extraction effect of subsequent optical character recognition (OCR). Therefore, effectively enhancing low-light document images to restore text clarity and readability has become a crucial technical challenge in the field of document image processing.

[0003] Existing technologies and traditional image enhancement methods include histogram equalization, gamma correction, and Retinex theory. Histogram equalization expands the dynamic range by redistributing image grayscale, but often leads to over-enhancement, noise amplification, and color distortion, making it difficult to maintain the clarity of text strokes. Retinex theory decomposes images based on the principle of color constancy in human vision, but parameter adjustment is difficult, computational complexity is high, and its effectiveness in handling complex lighting distributions is limited. Traditional methods generally lack consideration for the specific characteristics of document images and fail to optimize for structured content such as text, lines, and tables, resulting in problems such as blurred text edges, stroke adhesion or breakage, and page layout distortion after enhancement.

[0004] In recent years, deep learning-based low-light image enhancement methods have made significant progress. These methods utilize architectures such as convolutional neural networks, generative adversarial networks, and Transformers to learn the mapping relationship from low-light images to normal-light images end-to-end. However, the effectiveness of deep learning methods is highly dependent on large-scale paired training data. In practical applications, obtaining paired data of real low-light document images and normal-light document images is extremely difficult. It requires taking photos in low-light and normal-light environments at the same location, angle, and with the same document content, which is difficult to implement on a large scale in practice, and the data acquisition is costly and time-consuming. Some studies have attempted to synthesize low-light images through simple brightness compression and noise addition, but this approach cannot realistically simulate the complex illumination distribution, non-uniform brightness attenuation, local shadow changes, and noise characteristics unique to camera sensors in real low-light scenes, resulting in weak model generalization ability.

[0005] Furthermore, existing methods primarily target natural scene images, focusing on overall visual enhancement while lacking explicit modeling and utilization of document structural information. Document images possess clear text structure and layout features; the clarity and recognizability of text regions are key indicators for evaluating enhancement effects and directly impact OCR accuracy. Existing methods treat documents as ordinary images, easily leading to problems such as blurred text edges, loss of detail, and character adhesion or breakage. More importantly, even when structural information is introduced in existing technologies, it is often only fused at a single location in the network, failing to continuously utilize prior knowledge of document structure across multiple levels of feature reconstruction, resulting in limited structure-guided effects.

[0006] With the development of 3D graphics and rendering technology, generating realistic images using 3D rendering engines has become an important data augmentation method in the field of computer vision. By flexibly controlling parameters such as lighting, viewpoint, and materials in virtual 3D scenes, large-scale and diverse synthetic data can be generated efficiently, and successful applications have been made in fields such as autonomous driving and robot vision. However, to date, no research has applied 3D rendering technology to the construction of training data for low-light document image enhancement, nor has a technical solution been found that fully utilizes document structural features for progressive structure-guided reconstruction at multiple levels of the decoder. How to effectively train the enhancement model through innovative data synthesis methods in the absence of real paired data, and continuously utilize prior structural knowledge such as text outlines and layout of the document through multi-level cross-attention mechanisms during the enhancement process to achieve the dual goals of brightness enhancement and accurate preservation of text details, has become a key technical challenge that urgently needs to be overcome in this field. Summary of the Invention

[0007] The technical problems addressed by this invention are the difficulties and high costs associated with acquiring training data in existing low-light document image enhancement methods, as well as the poor recovery of text details and readability due to the lack of full utilization of document structure information for multi-level guidance during the enhancement process. Using the technical solution proposed in this invention, based on 3D rendering and multi-level document structure guidance, these shortcomings can be overcome, achieving low-cost, high-quality training data construction and progressive structure-aware intelligent low-light document image enhancement, significantly improving document readability and subsequent text recognition accuracy.

[0008] To achieve the above objectives, the first aspect of the present invention provides a low-light document image enhancement method based on 3D rendering and document structure guidance, comprising the following steps:

[0009] Step S1: Based on the scanned document image dataset, construct paired training samples of low-light document images and normal-light document images through 3D rendering and low-light degradation.

[0010] Step S2: Based on the paired training samples, train a low-light document image enhancement model including a low-light enhancement backbone network and a document structure prediction branch;

[0011] Step S3: Obtain the low-light document image to be enhanced;

[0012] Step S4: Input the low-light document image to be enhanced into the low-light document image enhancement model. The encoder of the low-light enhancement backbone network extracts features and outputs an intermediate enhanced image. The document structure prediction branch outputs a document structure prediction map based on the intermediate enhanced image. Each layer of the decoder of the low-light enhancement backbone network adjusts the document structure prediction map by scale and then fuses and reconstructs it with the encoded features through the cross-attention module to obtain the enhanced document image.

[0013] Further, in step S1, the method for constructing paired training samples of low-light document images and normal-light document images includes:

[0014] Acquire a dataset of scanned document images collected by a scanning device, attach each scanned document image to a planar target surface in a 3D rendering scene, and render the planar target in a preset 3D rendering engine to obtain a 3D rendered image containing the document images.

[0015] The background of the 3D rendered image is cropped, retaining only the document image area to obtain the rendered document image;

[0016] The rendered document image is subjected to brightness compression and noise degradation based on the low-light degradation model to generate a low-light document image. The low-light document image is then adjusted to the same resolution as the corresponding scanned document image according to a preset size. The scanned document image serves as a normal lighting document image and together with the corresponding low-light document image, they constitute a pair of training samples.

[0017] Furthermore, the light sources in the 3D rendering scene include at least one of point light sources, area light sources, and parallel light sources, and the type, quantity, spatial position, and light intensity of the light sources are randomly set in different rendering processes to simulate various actual shooting environments.

[0018] Further, in step S2, the training of the low-light document image enhancement model employs a joint loss function, which includes: a reconstruction loss to constrain pixel-level differences between the intermediate enhanced document image and the normal-light document image, a structural loss to constrain the structural similarity between the document structure prediction map and the target edge map, and a reconstruction loss to constrain pixel-level differences between the final enhanced document image and the normal-light image; the network parameters of the low-light enhancement backbone network and the document structure prediction branch are jointly optimized by minimizing the joint loss function.

[0019] Furthermore, in step S4, the low-light enhancement backbone network adopts an encoder-decoder structure:

[0020] The encoder is used to extract multi-scale features from the input low-light document image. It extracts image features layer by layer through multiple convolutional layers and downsampling layers, and outputs an intermediate enhanced document image at the end of the encoder. During training, the difference between the corresponding normal-light document image after being scaled down by the same ratio and the intermediate enhanced document image is used as the first loss for supervision.

[0021] Each layer of the decoder includes a cross-attention module, which receives the scale-adaptively adjusted document structure prediction map and performs cross-attention fusion with the features of the current layer. Through progressive upsampling and structure-guided reconstruction, the enhanced document image is finally output.

[0022] Further, in step S4, the document structure prediction branch adopts an independently configured encoder-decoder network structure, taking the intermediate enhanced document image output by the encoder of the low-light enhancement backbone network as input, and outputting a document structure prediction map after passing through the encoder and decoder. The document structure prediction map is used to represent the text outline and layout structure in the document. During the training process, the corresponding normal lighting document image is subjected to edge detection by an edge detection algorithm to obtain a target edge map, and the target edge map is used to supervise the construction of a second loss for the document structure prediction map.

[0023] Furthermore, in step S4, the operation of the cross-attention modules at each level of the backbone network decoder includes:

[0024] The document structure prediction map output by the document structure prediction branch is adjusted to a spatial resolution that matches the current layer feature map of the decoder of the low-light enhancement backbone network through a scale adaptive adjustment operation, thereby obtaining scale-adapted structure guidance features, and the structure guidance features are used as query features.

[0025] The input features of the current layer of the decoder of the low-light enhancement backbone network are used as key features and value features. The relevance weight between the query features and the key features is calculated based on the cross-attention mechanism. The value features are then weighted and fused to obtain an enhanced feature map containing document text outline details.

[0026] The enhanced feature map is used as the output of the current layer and passed to the next layer of the decoder of the low-light enhancement backbone network to continue structure-guided feature reconstruction until the final enhanced document image is output.

[0027] A second aspect of the present invention provides a low-light document image enhancement system based on 3D rendering and document structure guidance, comprising:

[0028] The sample construction module is used to acquire a dataset of scanned document images and construct paired training samples of low-light document images and normal-light document images through 3D rendering and low-light degradation.

[0029] The model training module is used to train a low-light document image enhancement model, including a low-light enhancement backbone network and a document structure prediction branch, based on the paired training samples.

[0030] The document structure prediction module is used to predict the text structure features of a document based on the intermediate enhanced image output by the encoder of the low-light enhancement backbone network during the low-light document image enhancement process.

[0031] The low-light document image enhancement module receives the low-light document image to be enhanced, inputs the low-light document image to be enhanced into the trained low-light document image enhancement model, and the document structure prediction map output by the document structure prediction module is scale-adaptively adjusted and input into the cross-attention modules of each layer of the decoder of the low-light enhancement backbone network to guide the decoder of the low-light enhancement backbone network to perform structure-aware feature reconstruction and output the enhanced document image.

[0032] A third aspect of the present invention provides a terminal device, the terminal device including a memory, a processor, and a program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the steps of a low-light document image enhancement method based on three-dimensional rendering and document structure guidance as described in any of the foregoing methods.

[0033] A fourth aspect of the present invention provides a computer-readable storage medium storing a program that, when executed by a processor, implements the steps of a low-light document image enhancement method based on 3D rendering and document structure guidance as described in any of the preceding methods.

[0034] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0035] (1) Innovative training data construction method: This invention creatively introduces 3D rendering technology into the construction of training data for low-light document image enhancement tasks. By flexibly controlling lighting parameters in a virtual 3D scene and combining them with a low-light degradation model, it achieves automated expansion from limited scanned documents to large-scale paired training samples, fundamentally solving the pain point of difficulty in obtaining real low-light document paired data, and significantly reducing the time and economic costs of data collection. By randomizing lighting parameters, diverse samples covering a variety of actual shooting scenarios can be generated, greatly improving the generalization ability and robustness of the model.

[0036] (2) Enhancement mechanism of multi-level document structure perception: This invention achieves progressive document structure-guided feature reconstruction by embedding cross-attention modules at each level of the backbone network decoder. Unlike existing technologies that only fuse structural information at a single location, this invention enables the document structure prediction map to continuously participate in the feature reconstruction process of each level of the decoder after scale adaptive adjustment. It enhances the restoration of text details layer by layer from coarse to fine granular, explicitly models and makes full use of the text outline and layout structure information of the document, so that the enhancement network can focus on and accurately restore the details and clarity of the text area at multiple levels while improving the overall brightness. This effectively avoids the problems of text blurring, stroke sticking, and edge distortion that are common in traditional methods, and significantly improves the readability of the enhanced document and the accuracy of subsequent text recognition.

[0037] (3) End-to-end joint optimization strategy with dual branches: This invention adopts an architecture design in which the backbone network and the document structure prediction branch work together. The encoder of the backbone network outputs the intermediate enhanced image to provide clear input for structure prediction. The structural information output by the document structure prediction branch guides the backbone network to perform structure-aware reconstruction through the cross-attention modules of each level of the decoder. The two branches are jointly trained end-to-end through a joint loss function. The two branches promote each other and evolve together during the optimization process, forming a virtuous cycle, so that the final model achieves excellent performance in both brightness restoration and structure preservation.

[0038] (4) Scale-adaptive flexible fusion mechanism: This invention enables a single document structure prediction map to flexibly adapt to feature maps of different resolutions at each level of the decoder through scale-adaptive adjustment operation, realizing cross-scale structural information reuse and multi-level fine-grained guidance, thereby improving the flexibility of the network architecture and the effectiveness of feature fusion.

[0039] (5) Strong versatility and wide adaptability: The technical solution of this invention adopts a generalized design for the encoding and decoding network structure, cross-attention mechanism, scale adjustment method, loss function type, etc., and is not limited to a specific network architecture or algorithm implementation, thus having strong versatility and scalability. At the same time, the training samples built based on 3D rendering can simulate a variety of complex lighting conditions, making the trained model more adaptable and able to handle various complex low-light shooting scenarios in practical applications.

[0040] (6) Efficiency and Practicality: This invention adopts an end-to-end deep learning architecture and efficient network design. It achieves rapid and intelligent enhancement of low-light document images without complex parameter adjustments and post-processing steps during the inference stage, meeting the dual requirements of processing speed and image quality in practical applications. This invention can be widely applied in mobile office, intelligent scanning, document management, electronic archives, online education, and other fields, providing intelligent quality improvement services for documents taken in low-light environments. It has significant practical application value and broad market prospects. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings are briefly described below. The following drawings only show some exemplary embodiments of the present invention; those skilled in the art can obtain other forms of drawings based on these drawings without creative effort.

[0042] Figure 1 This is a flowchart of the low-light document image enhancement method based on 3D rendering and document structure guidance of the present invention;

[0043] Figure 2 This is a flowchart illustrating the construction of paired training samples based on 3D rendering and low-light degradation in this invention.

[0044] Figure 3 This is a flowchart illustrating the training process of the low-light document image enhancement model in this invention.

[0045] Figure 4 This is a schematic diagram of the low-light document image enhancement system based on 3D rendering and document structure guidance according to the present invention. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] To address the technical problems of poor image quality and text recognition in low-light document images captured in existing technologies, as well as difficulties in acquiring training data for enhancement models and poor text detail recovery, this invention proposes a low-light document image enhancement method based on 3D rendering and document structure guidance. This scheme innovatively introduces 3D rendering technology into training data construction. By configuring multiple light source parameters in a virtual 3D scene and combining it with a low-light degradation model from a real camera sensor, it automatically generates a large-scale, high-quality pairwise training sample, fundamentally solving the problem of data acquisition difficulties. Simultaneously, this invention designs an innovative dual-branch collaborative network architecture. The backbone encoder outputs intermediate enhanced images for the document structure prediction branch to extract structural features, while each layer of the decoder embeds a cross-attention module. Through scale adaptation, the document structure prediction map is continuously integrated into the multi-level feature reconstruction process, achieving coarse-to-fine structure perception enhancement. Furthermore, this invention employs an end-to-end training strategy with a joint loss function, simultaneously optimizing brightness reconstruction and structure prediction, enabling the model to achieve excellent performance on low-light document enhancement tasks.

[0048] Example 1

[0049] like Figure 1 As shown, this embodiment provides a low-light document image enhancement method based on 3D rendering and document structure guidance, comprising steps S1 to S4. The following explanation uses a document image captured in a low-light environment as an example.

[0050] Step S1: Constructing paired training samples

[0051] like Figure 2 As shown, this step uses 3D rendering technology and a low-light degradation model to construct paired training samples of low-light document images and normal-light document images, thus solving the problem of difficulty in obtaining real data.

[0052] Acquire a dataset of scanned document images obtained by a scanning device. The scanning device may include a flatbed scanner, a document scanner, or other document acquisition devices. The scanned images have high resolution and clear text content, serving as standard document images under normal lighting conditions.

[0053] A virtual 3D scene is constructed using a 3D rendering engine. Planar targets are created within the scene; these targets can be planes or surfaces with curvature. Scanned document images are then attached as textures to the surface of the planar targets, simulating the physical form of the actual document and allowing the plane to represent the document's appearance in the virtual scene.

[0054] Light sources are configured in the 3D scene. These light sources include at least one of point lights, area lights, and parallel lights. During different rendering processes, the type, number, spatial location, and intensity of the light sources are randomly set to simulate various real-world shooting environments such as indoor table lamps, ceiling lights, and natural light from windows, increasing the diversity of the composite data. Through a randomization strategy, a large number of rendered images with different lighting conditions are generated from a limited set of scanned images.

[0055] Perform a 3D rendering operation to obtain a 3D rendering result containing the document image. The rendering can employ ray tracing, rasterization, or other rendering algorithms to calculate factors such as lighting, shadows, and materials in the scene, generating a realistic rendered image.

[0056] The rendered result is then subjected to background cropping to extract the document image region and remove other elements from the scene, resulting in the rendered document image. The background cropping can employ texture coordinate-based cropping, image segmentation-based cropping, or other methods.

[0057] The rendered document image is processed based on a low-light degradation model to generate a low-light document image. The low-light degradation model includes brightness compression and noise degradation. Brightness compression non-linearly reduces the image pixel values ​​to simulate underexposure, and can employ gamma correction, logarithmic transformation, linear scaling, or a combination thereof. Noise degradation superimposes noise onto the brightness-compressed image to simulate the noise characteristics of a camera sensor under low-light conditions; the noise can include Gaussian noise, Poisson noise, or a combination thereof.

[0058] The generated low-light document images are adjusted to the same resolution as the corresponding scanned document images. At this point, the scanned document images, as normally lit document images, together with the low-light document images, constitute paired training samples. Through this process, a large-scale paired training dataset can be automatically expanded from a limited scanned document dataset.

[0059] Through the above process, large-scale paired training data can be automatically generated from a limited scanned document dataset, significantly reducing data acquisition costs. At the same time, by generating diverse training samples through randomized lighting parameters, the generalization ability of the model can be improved.

[0060] Step S2: Train the low-light document image enhancement model

[0061] like Figure 3 As shown, this step constructs and trains a low-light document image enhancement model.

[0062] The enhancement model includes a low-light enhancement backbone network and a document structure prediction branch.

[0063] The low-light enhancement backbone network adopts an encoder-decoder structure:

[0064] The encoder of the low-light enhancement backbone network includes multiple convolutional layers and downsampling layers for multi-scale feature extraction from the input low-light document image. The encoder extracts image features layer by layer, from low-level texture to high-level semantics. After multi-scale feature extraction, the encoder outputs an intermediate enhanced document image at its end. During training, the difference between the intermediate enhanced document image and a scaled-down version of the corresponding normal-light document image is used as the first loss for supervision, guiding the backbone network encoder to learn preliminary brightness restoration and image reconstruction capabilities. The intermediate enhanced image has already undergone preliminary brightness enhancement and quality improvement compared to the input low-light image, but its resolution is typically lower than the original input image.

[0065] The decoder of the low-light enhancement backbone network includes multiple upsampling layers and convolutional layers for progressive reconstruction of encoded features. Each layer of the decoder embeds a cross-attention module to receive a scale-adapted document structure prediction map and perform structure-guided feature reconstruction. The decoder restores the spatial resolution of the image through layer-by-layer upsampling, ultimately outputting a full-resolution enhanced document image. The decoder restores the spatial resolution of the image through layer-by-layer upsampling, with each layer receiving a scale-adapted document structure prediction map. The cross-attention mechanism integrates structural information into the feature reconstruction process of the current layer, achieving progressive structure-aware enhancement.

[0066] In low-light enhanced backbone networks, feature fusion can be performed between the encoder and decoder via skip connections, preserving multi-scale detail information.

[0067] The document structure prediction branch employs an encoder-decoder network structure. It takes the intermediate enhanced document image output from the encoder of the low-light enhancement backbone network as input, extracts structural features through the encoder, and reconstructs and outputs a document structure prediction map through the decoder. This document structure prediction map is used to characterize the outline edges and layout structure information of text in the document, with higher response values ​​in text edge regions and lower response values ​​in background regions.

[0068] During training, edge detection technology is used to process document images under normal lighting to obtain target edge maps as supervision signals. The target edge maps are then used to construct a second loss for the document structure prediction map to supervise the document structure prediction branch and guide it to learn to accurately extract the structural features of the document.

[0069] The operation of the cross-attention modules at each level of the decoder in the low-light enhancement backbone network is as follows:

[0070] The document structure prediction map output by the document structure prediction branch undergoes scale adaptive adjustment to match its spatial resolution with the feature map of the current level of the decoder. This scale adaptive adjustment can employ methods such as upsampling, downsampling, bilinear interpolation, bicubic interpolation, or combinations thereof, to ensure that the document structure prediction map maintains spatial consistency with the feature maps of each level of the decoder, thus ensuring the effectiveness of cross-attention computation. The adjusted map yields scale-adapted structure-guided features.

[0071] The scale-adapted structure-guided features are used as query features, and the input features of the current level of the decoder are used as key and value features. A cross-attention mechanism is used to calculate the relevance weights between the query features and key features, and the value features are then weighted and fused to obtain an enhanced feature map containing details of the document's text outlines.

[0072] The enhanced feature map is used as the output feature of the current layer and passed to the next layer of the decoder to continue structure-guided feature reconstruction until the final enhanced document image is output. This process is repeated at each layer of the decoder, allowing document structure information to continuously and progressively guide the feature reconstruction process.

[0073] By performing cross-attention fusion at each level of the decoder, document structure information can continuously and progressively guide the feature reconstruction process, ensuring that structural prior knowledge can be fully utilized at each level from coarse to fine granular, thereby achieving more accurate restoration of text details.

[0074] The difference between the final output enhanced document image and the corresponding normally lit document image is used as a third loss for supervision, which is used to supervise the end-to-end brightness restoration, detail restoration and image reconstruction capabilities of the backbone network.

[0075] The training process uses a joint loss function:

[0076] Reconstruction loss: The difference between the normally illuminated document image (adjusted proportionally) and the intermediate enhanced image output by the backbone encoder is used as the reconstruction loss. The difference between the final enhanced document image and the corresponding normally illuminated document image is also used as the reconstruction loss. The reconstruction loss can be L1 loss (absolute error loss), L2 loss (squared error loss), perceptual loss, or a combination thereof. This is used to supervise the backbone encoder and decoder in learning their initial brightness restoration capabilities.

[0077] Structural Loss: A target edge map is obtained by applying edge detection technology to a normally illuminated document image, serving as a supervisory signal. A structural loss is calculated between the document structure prediction map and the target edge map to guide the document structure prediction branch in accurately extracting document structural features. The structural loss can be binary cross-entropy loss, Dice loss, structural similarity index, or a combination thereof.

[0078] The overall loss is a weighted sum of the reconstruction loss and the structural loss. The network parameters are optimized by minimizing the overall loss. The optimization process includes forward propagation to calculate the loss, backpropagation to calculate the gradient, and updating the network parameters. Through iterative training, the backbone network and the structural prediction branch mutually reinforce each other during training: the intermediate augmented images provided by the backbone network encoder help the structural prediction branch extract document structure more accurately, while the structural information output by the structural prediction branch, through the cross-attention modules at each level of the decoder, in turn guides the backbone network decoder to better recover text details.

[0079] The technical solution is further optimized. The encoder includes multiple convolutional layers and downsampling layers to progressively reduce the resolution of the feature map and extract multi-scale features. The decoder includes multiple upsampling layers and convolutional layers to progressively restore the resolution of the feature map and reconstruct the image. Feature fusion can be performed between the encoder and decoder through skip connections to preserve multi-scale detail information.

[0080] To further optimize the technical solution, the brightness compression in the low-light degradation model uses a nonlinear mapping function to reduce pixel values, simulating the effect of underexposure; the noise degradation includes adding Gaussian noise, Poisson noise, salt-and-pepper noise or a combination thereof to simulate the noise characteristics of the camera sensor under low-light conditions.

[0081] To further optimize the technical solution, the 3D rendering engine can be a ray tracing-based rendering engine, a rasterization-based rendering engine, or other types of rendering engines; the rendering process can include operations such as material settings, shadow calculation, and global illumination calculation to generate realistic document images.

[0082] Step S3: Obtain the low-light document image to be enhanced

[0083] Users upload or photograph low-light document images that require enhancement via mobile devices or computers. These low-light document images can be photographs of documents taken in poorly lit indoor environments, nighttime scenes, or backlit conditions. The system preprocesses the images, including resizing and pixel value normalization, to ensure they meet the model's input requirements.

[0084] Step S4: Model Inference and Image Enhancement Output

[0085] The preprocessed low-light document image is input into the trained augmentation model to perform forward inference.

[0086] The backbone network encoder performs multi-scale feature extraction on the input image and outputs an intermediate enhanced document image at the end of the encoder.

[0087] The document structure prediction branch receives intermediate augmented images as input and outputs a document structure prediction map after processing by the encoder and decoder. This prediction map highlights the edge contours of text and the layout structure of the document.

[0088] The decoder of the low-light enhancement backbone network begins to reconstruct features layer by layer. At each layer of the decoder, the document structure prediction map, after scale adaptive adjustment, is fused with the features of the current layer through a cross-attention module.

[0089] The scale-adjusted structural prediction map is used as the query feature; the input features of the current level are used as key and value features; cross-attention weights are calculated and weighted fusion is performed; an enhanced feature map that incorporates structural information is obtained; and the enhanced feature map is passed to the next level.

[0090] In this way, the decoder can make full use of the document structure information in the reconstruction process at each level, and gradually enhance the restoration of details of the text area from coarse to fine granular.

[0091] After the decoder of the low-light enhancement backbone network completes reconstruction at all levels, it outputs the final enhanced document image. The system performs post-processing on the output image, including pixel value denormalization, resolution adjustment, and format conversion, to obtain an enhanced document image usable by the user. The enhanced image has appropriate overall brightness, clear text, and complete structure, and can be saved, shared, or transmitted to the text recognition module for further processing.

[0092] This invention addresses and optimizes existing low-light document image enhancement methods, which suffer from difficulties in acquiring training data, high costs, and issues such as blurred text and loss of detail during enhancement. It innovatively combines 3D graphics rendering technology with deep learning, significantly reducing data acquisition costs. Furthermore, through a multi-level document structure perception mechanism, it greatly improves the readability of the enhanced document and the accuracy of OCR recognition, solving key technical challenges in document photography and digitization in low-light environments. This invention can automatically generate diverse low-light document image training samples using 3D rendering technology even in the absence of large-scale real-world paired data. It explicitly extracts text outlines and layout features from intermediate enhanced images using a document structure prediction branch. By embedding cross-attention modules at each level of the backbone network decoder, document structure information is gradually integrated into the feature reconstruction process, guiding the low-light enhancement network to focus on restoring text details while improving overall brightness, resulting in high-quality document images with appropriate brightness, clear text, and complete structure. Firstly, the data construction method based on 3D rendering can quickly expand from limited scanned documents to generate large-scale training samples, significantly reducing data acquisition costs. Randomizing lighting parameters generates diverse data, improving the model's generalization ability. Secondly, the multi-level document structure guidance mechanism enables the enhancement process to fully utilize prior document knowledge at each layer of the decoder, achieving progressive structure-aware reconstruction. This accurately restores text details while improving brightness, effectively avoiding problems such as blurry text and edge distortion found in traditional methods, significantly improving the readability and OCR accuracy of the enhanced document. Finally, this solution adopts an end-to-end architecture, eliminating the need for complex post-processing during the inference stage, enabling rapid intelligent enhancement. It has significant application value in mobile office, intelligent scanning, document management, and electronic archives.

[0093] Example 2

[0094] like Figure 4 As shown, this embodiment provides a low-light document image enhancement system based on 3D rendering and document structure guidance, including a sample construction module, a model training module, a document structure prediction module, and a low-light document image enhancement module.

[0095] The sample construction module integrates a 3D rendering engine interface and a low-light degradation algorithm library for batch processing of scanned document image datasets. It automates operations such as 3D scene construction, lighting settings, rendering execution, background cropping, and low-light degradation, generating paired training samples. Supporting parallel processing and batch generation, the module can efficiently generate large-scale training data by constructing low-light document images and normally lit document images using 3D rendering technology and low-light degradation models.

[0096] The model training module is responsible for configuring the training process. It reads paired samples from the training database, configures training parameters, constructs the joint loss function, performs forward and backpropagation, updates network parameters, and monitors loss changes and model performance metrics during training. The module trains a low-light document image enhancement model that includes a low-light enhancement backbone network and a document structure prediction branch. The training process includes forward propagation, loss calculation, backpropagation, and parameter updates. The module monitors the training loss and performance metrics in real time and saves the trained model. Specifically, this module ensures that each layer of the backbone network decoder correctly embeds the cross-attention module, achieving multi-level structure-guided training.

[0097] The document structure prediction module encapsulates the trained document structure prediction branch network. During the inference phase, the document structure prediction module receives the intermediate enhanced image output from the encoder of the low-light enhancement backbone network, extracts document structure features through the encoder-decoder network, and outputs a document structure prediction map, providing structural guidance information for subsequent multi-level cross-attention fusion.

[0098] The low-light document image enhancement module is the core application interface of the system. This module receives the low-light document image to be enhanced and calls the pre-trained full model to perform inference. The module coordinates the workflow of the backbone network and the document structure prediction branch: the encoder of the low-light enhancement backbone network extracts features and outputs an intermediate enhanced image; the document structure prediction module outputs a structure prediction map based on the intermediate enhanced image; each layer of the backbone network decoder receives the scale-adjusted structure prediction map through a cross-attention module and performs multi-level structure-guided reconstruction, finally outputting the enhanced document image. The low-light document image enhancement module provides a user interface, supporting functions such as image uploading, result display, saving, and exporting.

[0099] The system may also include a result evaluation module and a user interaction module. The result evaluation module is used to assess the quality of the enhanced document images, and the evaluation metrics may include peak signal-to-noise ratio, structural similarity index, character recognition accuracy, character error rate, etc. The user interaction module provides a user-friendly graphical interface for receiving user-uploaded images to be enhanced, displaying enhancement results, providing parameter adjustment interfaces, and saving or exporting enhanced images. It supports functions such as parameter adjustment, batch processing, and real-time preview.

[0100] Through the collaborative work of the modules described above, the system achieves automated and intelligent enhancement of low-light document images. The system supports multiple deployment methods, including desktop applications, mobile applications, and web services, to meet the needs of different application scenarios.

[0101] Example 3

[0102] This embodiment provides a terminal device including a memory, a processor, and a program stored in the memory and executable on the processor. The program is configured to perform the steps of the aforementioned low-light document image enhancement method based on 3D rendering and document structure guidance. When the processor executes the program, it can implement the low-light document image enhancement function as described in any of the foregoing method embodiments.

[0103] To further optimize the technical solution, the terminal device can be a personal computer, server, mobile smart device, embedded device, edge computing device, or other types of computing device.

[0104] To further optimize the technical solution, the processor may be a central processing unit, a graphics processing unit, a tensor processor, a neural network processor, or a combination thereof, used to perform inference calculations for deep learning models.

[0105] Example 4

[0106] This embodiment provides a computer-readable storage medium storing a program. When the program is executed by a processor, it implements the steps of the low-light document image enhancement method as described in any of the foregoing method embodiments.

[0107] To further optimize the technical solution, the computer-readable storage medium includes read-only memory, random access memory, disk, optical disk, solid-state drive, flash memory, or other types of storage media.

[0108] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for enhancing low-light document images based on 3D rendering and document structure guidance, characterized in that, Includes the following steps: Step S1: Based on the scanned document image dataset, construct paired training samples of low-light document images and normal-light document images through 3D rendering and low-light degradation. Step S2: Based on the paired training samples, train a low-light document image enhancement model including a low-light enhancement backbone network and a document structure prediction branch; Step S3: Obtain the low-light document image to be enhanced; Step S4: Input the low-light document image to be enhanced into the low-light document image enhancement model. The encoder of the low-light enhancement backbone network extracts features and outputs an intermediate enhanced image. The document structure prediction branch outputs a document structure prediction map based on the intermediate enhanced image. Each layer of the decoder of the low-light enhancement backbone network scales the document structure prediction map and then fuses and reconstructs it with the encoded features through the cross-attention module to obtain the enhanced document image. In step S4, the operation of the cross-attention modules at each level of the backbone network decoder includes: The document structure prediction map output by the document structure prediction branch is adjusted to a spatial resolution that matches the current layer feature map of the decoder of the low-light enhancement backbone network through a scale adaptive adjustment operation, so as to obtain scale-adapted structure guidance features, and the structure guidance features are used as query features. The input features of the current layer of the decoder of the low-light enhancement backbone network are used as key features and value features. The correlation weight between the query features and the key features is calculated based on the cross-attention mechanism. The value features are then weighted and fused to obtain an enhanced feature map containing document text outline details. The enhanced feature map is used as the output of the current layer and passed to the next layer of the decoder of the low-light enhancement backbone network to continue structure-guided feature reconstruction until the final enhanced document image is output.

2. The low-light document image enhancement method based on 3D rendering and document structure guidance according to claim 1, characterized in that, In step S1, the method for constructing paired training samples of low-light document images and normal-light document images includes: Acquire a dataset of scanned document images collected by a scanning device, attach each scanned document image to a planar target surface in a 3D rendering scene, and render the planar target in a preset 3D rendering engine to obtain a 3D rendered image containing the document images. The background of the 3D rendered image is cropped, retaining only the document image area to obtain the rendered document image; The rendered document image is subjected to brightness compression and noise degradation based on the low-light degradation model to generate a low-light document image. The low-light document image is then adjusted to the same resolution as the corresponding scanned document image according to a preset size. The scanned document image serves as a normal lighting document image and together with the corresponding low-light document image, they constitute a pair of training samples.

3. The low-light document image enhancement method based on 3D rendering and document structure guidance according to claim 2, characterized in that, The light sources in the 3D rendering scene include at least one of point light sources, area light sources, and parallel light sources. The type, number, spatial position, and light intensity of the light sources are randomly set in different rendering processes to simulate various actual shooting environments.

4. The low-light document image enhancement method based on 3D rendering and document structure guidance according to claim 1, characterized in that, In step S2, the training of the low-light document image enhancement model employs a joint loss function, which includes: a reconstruction loss to constrain pixel-level differences between intermediate enhanced document images and normally lit document images, a structural loss to constrain structural similarity between the document structure prediction map and the target edge map, and a reconstruction loss to constrain pixel-level differences between the final enhanced document image and the normally lit image; the network parameters of the low-light enhancement backbone network and the document structure prediction branch are jointly optimized by minimizing the joint loss function.

5. The low-light document image enhancement method based on 3D rendering and document structure guidance according to claim 1, characterized in that, In step S4, the low-light enhancement backbone network adopts an encoder-decoder structure: The encoder is used to perform multi-scale feature extraction on the input low-light document image. It extracts image features layer by layer through multiple convolutional layers and downsampling layers, and outputs an intermediate enhanced document image at the end of the encoder. During training, the difference between the normally lit document image scaled down by the same ratio and the intermediate enhanced document image is used as the first loss for supervision. Each layer of the decoder includes a cross-attention module, which receives the scale-adaptively adjusted document structure prediction map and performs cross-attention fusion with the features of the current layer. Through progressive upsampling and structure-guided reconstruction, the enhanced document image is finally output.

6. The low-light document image enhancement method based on 3D rendering and document structure guidance according to claim 1, characterized in that, In step S4, the document structure prediction branch adopts an independently configured encoder-decoder network structure, taking the intermediate enhanced document image output by the encoder of the low-light enhancement backbone network as input, and outputting a document structure prediction map after passing through the encoder and decoder. The document structure prediction map is used to characterize the text outline and layout structure in the document. During training, the corresponding normally lit document image is subjected to edge detection by an edge detection algorithm to obtain a target edge map, and the target edge map is used to construct a second loss for supervision of the document structure prediction map.

7. A low-light document image enhancement system based on 3D rendering and document structure guidance, characterized in that, The method for enhancing low-light document images based on 3D rendering and document structure guidance as described in any one of claims 1-6 includes: The sample construction module is used to acquire a dataset of scanned document images and construct paired training samples of low-light document images and normal-light document images through 3D rendering and low-light degradation. The model training module is used to train a low-light document image enhancement model, including a low-light enhancement backbone network and a document structure prediction branch, based on the paired training samples. The document structure prediction module is used to predict the text structure features of a document based on the intermediate enhanced image output by the encoder of the low-light enhancement backbone network during the low-light document image enhancement process. The low-light document image enhancement module receives the low-light document image to be enhanced, inputs the low-light document image to be enhanced into the trained low-light document image enhancement model, and the document structure prediction map output by the document structure prediction module is scale-adaptively adjusted and input into the cross-attention modules of each layer of the decoder of the low-light enhancement backbone network to guide the decoder of the low-light enhancement backbone network to perform structure-aware feature reconstruction and output the enhanced document image.

8. A terminal device, characterized in that, The terminal device includes a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of a low-light document image enhancement method based on three-dimensional rendering and document structure guidance as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program that, when executed by a processor, implements the steps of a low-light document image enhancement method based on 3D rendering and document structure guidance as described in any one of claims 1-6.

Images