A document image geometry correction method and related device

Abstract

This invention belongs to the field of digital image processing and computer vision technology, and relates to a method and related equipment for geometric correction of document images. The method includes: preprocessing a distorted document image to extract document regions from the image; using a deep learning model to extract geometric features from the document regions, obtaining structural feature lines and local angular distortion information in the image; representing the local angular distortion information as a Beltrami coefficient field, and constructing a geometric constraint model based on the structural feature lines and the Beltrami coefficient field; obtaining a quasi-conformal mapping by solving the Beltrami equation according to the geometric constraint model, and mapping the distorted image domain to a standard plane image domain based on the quasi-conformal mapping, thereby obtaining the corrected document image. This invention extracts macroscopic structural feature lines and microscopic local distortion information through a deep learning model, and combines this with quasi-conformal mapping theory to construct a geometric constraint model, which can maintain the consistency of local character shapes while correcting the overall structure of the document image, effectively improving the quality of geometric restoration of the document image.

Description

Technical Field

[0001] This invention belongs to the field of digital image processing and computer vision technology, and relates to a method and related equipment for geometric correction of document images. Background Technology

[0002] With the widespread use of mobile devices and smartphones, capturing images of paper documents using mobile cameras has become a common method of information collection. However, in actual shooting processes, due to factors such as shooting angle, paper curvature, curling, folding, or local undulations, the obtained document images often suffer from perspective distortion and non-rigid geometric deformation. This leads to problems such as curved text lines, distorted page boundaries, and distorted character proportions, which in turn affect the document reading experience and the accuracy of subsequent tasks such as optical character recognition, document analysis, and information entry.

[0003] Existing document image correction methods can be broadly categorized into two types. The first type is based on traditional geometric models, typically using cylindrical surface models, developable surface models, or hand-drawn geometric rules based on text lines to correct distorted documents. While these methods offer some interpretability, they often rely on strong deformation assumptions, making them ill-suited to the complex and diverse paper deformations encountered in real-world scenarios. The second type is based on deep learning, which typically uses neural networks to directly regress pixel displacement fields, control points, or geometric transformation parameters. Although these methods can achieve good results with ample data support, their geometric constraints are mostly learned implicitly, lacking explicit mathematical modeling and theoretical guarantees, making it difficult to simultaneously restore the overall document structure and preserve the shape of local characters.

[0004] Therefore, how to establish a document image distortion removal method that can express the overall layout structure of a document image, depict local geometric deformations, and has clear mathematical theoretical support has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] To address the problem that existing technologies lack clear geometric constraints and struggle to simultaneously ensure text structure restoration and character shape preservation during document image geometric correction, this invention proposes a document image geometric correction method and related equipment. By extracting the structural features and local geometric deformation information of the image through a deep learning model and combining it with a theoretically guaranteed quasi-conformal mapping model, stable correction and structural restoration of distorted images can be achieved.

[0006] This invention is achieved through the following technical solution: A document image geometric correction method, comprising: Obtain the document region from the distorted document image; Based on a deep learning model, structural feature lines of document regions and Beltrami coefficient fields of local angular distortion are extracted from distorted document images. A geometric constraint model is constructed based on the structural feature lines and the Beltrami coefficient field of the local angular distortion. The quasi-conformal mapping from the distorted image domain to the target plane domain is solved based on the geometric constraint model. Based on the quasi-conformal mapping, the distorted document image is mapped and reconstructed to generate a corrected document image.

[0007] Preferably, the method for extracting structural feature lines and Beltrami coefficient fields of local angular distortion in document regions of distorted document images based on a deep learning model specifically includes: A deep learning-based structural feature line detection model is used to learn and predict features of document regions in distorted document images, outputting a probability map of structural feature lines. Structural feature lines describing the macroscopic geometric structure in the document image are obtained through thresholding or post-processing. The structural feature lines include at least one of text lines, page boundaries, or page structure lines. Using a deep learning-based Beltrami coefficient field prediction model, local geometric deformation prediction is performed on the document region of the distorted document image to obtain the Beltrami coefficient field corresponding to the local angular distortion. Each coefficient in the Beltrami coefficient field consists of a real part and an imaginary part, which are used to represent the direction and degree of local deformation, and its magnitude is less than 1.

[0008] Preferably, the geometric constraint model is constructed based on the structural feature lines and the Beltrami coefficient field of the local angular distortion, specifically including: The document region of the distorted document image is discretized into a grid structure, and the structural feature lines are embedded in the grid structure as constraints. Based on the Beltrami coefficient field of the local angular distortion and the constraint conditions, a geometric constraint model satisfying the Beltrami equation is constructed.

[0009] Preferably, the equation satisfies the Beltrami equation:

[0010] in: The Beltrami coefficient; Used to ensure that the mapping is a differential homeomorphism.

[0011] Preferably, in the process of solving the quasi-conformal mapping from the distorted image domain to the target plane domain, a straight line constraint is applied to the embedded structural feature lines, so that the text lines in the mapping result tend to be arranged in a horizontal or vertical direction.

[0012] Preferably, the texture mapping and image reconstruction include: According to the quasi-conformal mapping, the pixel information of the original distorted document image is projected onto the target planar grid; The grayscale or color value of each pixel in the target image is calculated using interpolation methods to generate the corrected document image.

[0013] Preferably, the document region of the distorted document image is obtained as follows: Document region extraction is performed on the acquired distorted document image. The document region extraction is implemented through an image segmentation model, which is used to separate the document region from the original distorted document image and remove background interference to obtain the document region of the distorted document image.

[0014] A document image correction system for performing a document image geometric correction method, comprising: Preprocessing module: Used to extract document regions from distorted document images; Feature extraction module: Extracts structural feature lines of document regions and Beltrami coefficient fields of local angular distortion from distorted document images using a deep learning model; Mapping calculation module: Constructs a geometric constraint model based on the structural feature lines and the Beltrami coefficient field of the local angular distortion, and solves the quasi-conformal mapping from the distorted image domain to the target plane domain based on the geometric constraint model; Image generation module: Based on the quasi-conformal mapping, the distorted document image is mapped and reconstructed to generate a corrected document image.

[0015] A computer device, comprising: One or more processors; Memory; The computer program stored in the memory; When the computer program is executed by the processor, the device performs the document image geometric correction method.

[0016] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the document image geometric correction method.

[0017] Compared with the prior art, the present invention has the following beneficial technical effects: This invention provides a document image geometric correction method. It automatically extracts prior geometric information of the document through deep learning and combines it with theoretically guaranteed quasi-conformal mapping to achieve geometric restoration of the document image. While restoring the overall document structure, it maintains the consistency of character shape and proportion, improving the accuracy and stability of document image correction. The method simultaneously extracts structural feature lines and local deformation information of the image through a deep learning model, achieving multi-scale geometric prior fusion and improving the accuracy of image geometric restoration. This invention also proposes an implementation scheme that uses a deep learning model to predict the Beltrami coefficient field and combines feature line constraints with the Beltrami coefficient field to drive the mapping solution. By introducing quasi-conformal mapping theory, it provides mathematically grounded geometric constraints during image geometric transformation, ensuring the bijectivity and smoothness of the mapping, thereby avoiding structural folding or distortion. Quasi-conformal mapping can maintain the consistency of character shape and proportion while restoring the overall planar structure of the page. By considering both overall structural restoration and local shape preservation during the correction process, it can effectively improve the straightness of text lines and maintain consistent character proportions, thus obtaining a more stable image correction effect. This invention effectively restores the straight-line structure of text lines and improves geometric consistency in document image correction tasks, achieving superior correction results compared to existing methods on multiple public datasets. The method of this invention has good generalization ability and can be applied to various image processing scenarios with geometric distortions.

[0018] Furthermore, this invention makes the extraction of structural feature lines more accurate and reliable by using deep learning models. Through probability maps and post-processing, it can obtain accurate macroscopic structural information such as text lines and page boundaries. At the same time, the complex-valued coefficients output by the Beltrami coefficient field prediction model can finely characterize the direction and intensity of local deformation, and the constraint that the modulus is less than 1 ensures the physical rationality of the deformation field, thus providing high-quality input features for subsequent geometric modeling. Furthermore, this invention constructs a geometric constraint model that conforms to the Beltrami equation by discretizing the document region into a grid and embedding structural feature lines into it as constraints. This achieves the organic integration of macroscopic structural constraints and microscopic deformation information, so that the solution of the quasi-conformal mapping has a clear geometric basis. Furthermore, this invention constructs a geometric constraint model that conforms to the Beltrami equation by discretizing the document region into a grid and embedding structural feature lines into it as constraints. This achieves the organic integration of macroscopic structural constraints and microscopic deformation information, so that the solution of the quasi-conformal mapping has a clear geometric basis. Furthermore, this invention applies straight-line constraints to structural feature lines during the process of solving the quasi-conformal mapping, causing text lines to tend to be arranged horizontally or vertically, effectively restoring the overall layout structure of the document and improving the visual regularity of the corrected image; by projecting the original pixels onto the target grid through texture mapping and interpolation methods, the pixel continuity and image quality of the corrected image are guaranteed, avoiding artifacts such as jagged edges or holes. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the overall process of document image distortion removal based on quasi-conformal mapping in an embodiment of the present invention; Figure 2 This is a schematic diagram of document region foreground segmentation and boundary extraction in an embodiment of the present invention; Figure 3 This is a schematic diagram of the network architecture of the structural feature line detection model in an embodiment of the present invention; Figure 4 This is a schematic diagram of the network architecture of the Beltrami coefficient prediction model in an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the structural feature line data construction method in an embodiment of the present invention; Figure 6 This is a schematic diagram illustrating the Beltrami coefficient supervision data construction method in an embodiment of the present invention; Figure 7 This is a schematic diagram of the constraint quasi-conformal mapping and texture mapping process in an embodiment of the present invention; Figure 8 This is a schematic diagram comparing the distortion removal effect before and after introducing the Beltrami coefficient field in an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0022] This invention provides a document image geometric correction method using deep learning, specifically including: First, the input distorted document image is preprocessed by size standardization and noise suppression. Then, the document region in the image is extracted by image segmentation or semantic segmentation methods to remove the influence of the background region on subsequent geometric analysis.

[0023] Subsequently, a deep learning model is used to learn features from the distorted document image to extract geometric prior information. An end-to-end neural network model automatically predicts the document's geometric prior information from the distorted document image, including: structural feature lines reflecting the overall document structure, and angular distortion information describing the degree of local deformation. The structural feature lines, predicted by the deep learning model, include text lines, page boundaries, or other geometrically meaningful structural lines, representing the overall layout structure of the document page. The angular distortion information is estimated by the deep learning model and represented as a Beltrami coefficient field, used to characterize the degree of local angular deformation at various locations in the image, thus describing the fine-grained deformation features of the document image and providing geometric prior information for the subsequent construction of quasi-conformal mappings.

[0024] Then, a corresponding geometric constraint model is constructed based on the structural feature lines and the Beltrami coefficient field, and a mapping relationship between the distorted image domain and the target planar image domain is established based on the quasi-conformal mapping theory. Specifically, the input image domain is discretized into a two-dimensional mesh or triangular mesh structure, and the structural feature lines are embedded into this mesh structure as geometric constraints. Simultaneously, the Beltrami coefficient field is used to describe the degree of angular distortion in each local region of the mesh. Based on this, by solving the quasi-conformal mapping equation related to the Beltrami coefficients, the geometric mapping relationship from the distorted image domain to the standard planar image domain is calculated, thereby achieving the restoration of the overall geometric structure of the image.

[0025] Furthermore, during the quasi-conformal mapping calculation process, by applying linear constraints to the structural feature lines, the corresponding text lines or structural lines in the corrected image tend to align horizontally or vertically, thereby restoring the original document layout structure. Simultaneously, by constraining local angular distortion through the Beltrami coefficient field, the mapping process controls the degree of local deformation while maintaining overall structural correction, thus preserving the consistency of character shape and proportion, and avoiding distortions such as character stretching or compression.

[0026] Finally, based on the calculated geometric mapping relationship, texture mapping is performed on the original distorted image, and the pixel information of the original image is reprojected into the target planar image domain according to the mapping relationship. The final corrected image is then generated through interpolation, thereby obtaining a planar document image with a regular structure and consistent character proportions.

[0027] This invention automatically extracts structural feature lines and local deformation information from images using a deep learning model, achieving the fusion of multi-scale geometric priors and thus improving the accuracy of image geometric restoration. Simultaneously, it introduces a theoretically guaranteed quasi-conformal mapping theory, providing rigorous mathematical constraints for geometric transformations to ensure bijectivity and smoothness, avoiding structural folding or distortion, and maintaining consistent character shape and proportion while restoring the overall planar structure of the document. Furthermore, this invention proposes a technical solution based on deep learning-predicted Beltrami coefficient fields, combined with feature line constraints and Beltrami coefficient field-driven mapping solutions. This solution balances overall structural restoration and local shape preservation during the correction process, effectively improving text line straightness and geometric consistency. It achieves superior correction results compared to existing methods on multiple public datasets and demonstrates strong generalization ability, making it applicable to various image processing scenarios with geometric distortions.

[0028] The method of this invention mainly includes the following steps: S1. Input Image and Document Region Extraction: First, acquire the document image to be processed. Since document images captured by mobile devices often contain complex backgrounds, document region detection and extraction are first performed on the input image to obtain the main document region. This document region extraction can be achieved using an image segmentation model to separate the paper region from the original image and remove background interference.

[0029] S2. Structural Feature Line Detection Model: After obtaining the document region, a deep learning network is used to detect structural features in the image and extract structural feature lines from the document image. These structural feature lines include geometric structural information such as text lines and page boundaries.

[0030] In one embodiment, a neural network based on an encoder-decoder structure is used for feature line detection. This network receives the document image as input and outputs a probability map of the corresponding structural feature lines. Thresholding or post-processing methods are then used to obtain the final text line and boundary line positions, which are used to describe the overall layout structure of the document image.

[0031] S3. Beltrami Coefficient Field Prediction Model: To characterize the local geometric deformation of an image, this invention further predicts the Beltrami coefficient field in the image using a deep learning model. Beltrami coefficients describe the degree of local angular distortion in an image, and their values ​​satisfy the constraint that the magnitude is less than 1. In one embodiment, a dense prediction network based on a Transformer structure is used to process the input image and output the Beltrami coefficients at the corresponding locations. The Beltrami coefficients typically consist of real and imaginary parts, used to represent the direction and degree of local deformation, thus forming a Beltrami coefficient field defined over the entire image domain.

[0032] S4. Construct a quasi-conformal mapping model: Based on the structural feature lines obtained in step S2 and the Beltrami coefficient field obtained in step S3, construct the corresponding geometric constraint model, and establish the mapping relationship between the image domains based on the quasi-conformal mapping theory.

[0033] Specifically, the input image domain is first discretized into a two-dimensional triangular mesh structure, and the detected structural feature lines are embedded in this mesh as constraints. Then, a mathematical model of the quasi-conformal mapping is constructed based on the Beltrami coefficient field. The quasi-conformal mapping satisfies the Beltrami equation:

[0034] in, μ This represents the Beltrami coefficient, used to describe the degree of local angular deformation.

[0035] In this mapping calculation process, by applying linear constraints to the structural feature lines, the text lines in the mapping result tend to be aligned horizontally or vertically, thereby restoring the overall structure of the document. Simultaneously, the Beltrami coefficient field controls local deformation, ensuring that characters maintain a consistent proportion and shape.

[0036] The quasi-conformal mapping can be obtained by solving a system of sparse linear equations, thereby obtaining the mapping relationship from the distorted image domain to the standard planar image domain.

[0037] S5. Texture Mapping and Image Reconstruction: After obtaining the quasi-conformal mapping, texture mapping is performed on the original image according to the mapping relationship, projecting the pixel information of the original image onto the target planar grid. The grayscale value or color value of each pixel in the target image is calculated using an interpolation method, thereby generating the final corrected image.

[0038] By following the steps above, a document image with geometric distortion can be restored to a well-structured planar image, maintaining the straight-line structure of text lines while ensuring the consistency of character shape and proportion.

[0039] S6. System Implementation: In one embodiment, the method of the present invention can be implemented by a computer system. The system includes an image input module, a feature extraction module, a mapping calculation module, and an image reconstruction module. Wherein: 1. The image input module is used to acquire the image to be processed; 3. The feature extraction module is used to detect structural feature lines and predict the Beltrami coefficient field; 3. The mapping calculation module is used to calculate the quasi-conformal mapping based on the geometric priors; 4. The image reconstruction module is used to generate corrected images based on the mapping relationship.

[0040] Each module can be implemented by running software programs on a computing device, or by hardware or a combination of hardware and software.

[0041] This invention provides a document image geometric correction method employing deep learning. This method extracts geometric prior information from the document image using a deep learning model and establishes a mapping relationship between the distorted image domain and the target plane image domain using theory, thereby achieving geometric correction and structural restoration of the document image. Figure 1 As shown, the method of the present invention includes the following steps.

[0042] S1. Document Region Extraction S1.1 Input Image Acquisition: Acquire the document image to be processed. The image is defined in the two-dimensional image domain: ; image This represents an RGB color image.

[0043] S1.2, Document region segmentation: such as Figure 2 As shown, an image segmentation model is used to extract document regions from the input image, resulting in a document region mask.

[0044] in

[0045] Represents a document area.

[0046] S1.3, Region cropping: Based on the mask The input image is cropped to obtain the document image.

[0047] S2, Structural Feature Line Detection S2.1, Structural Feature Line Prediction: Predicting the image Input a feature detection network to predict structural feature lines in a document.

[0048] S2.2 Definition of the set of characteristic lines: The set of structural characteristic lines is denoted as... Each feature line , represents the curve structure in the image.

[0049] The structural feature lines include: text lines and page borders. S2.3 Structural Constraints: These curves will serve as curve straightening constraints in subsequent geometric mapping calculations.

[0050] S3 and Beltrami coefficient field prediction S3.1 Local Deformation Modeling: To describe local angular distortion in an image, this invention predicts the Beltrami coefficient field:

[0051] S3.2 Definition of Beltrami coefficients: Beltrami coefficients originate from quasi-conformal mappings.

[0052] Satisfying the Beltrami equation

[0053] in

[0054] S3.3 Geometric meaning: The Beltrami coefficient describes local angular distortion. Indicates the local deformation intensity. Indicates the direction of deformation S4. Solving Quasi-conformal Mappings S4.1, Grid Discretization: Distributing the image domain Discretized into a two-dimensional triangular mesh

[0055] in, For the set of vertices, Let be the set of edges. It is a set of triangular facets.

[0056] S4.2 Geometric Constraint Establishment: Using the structural feature lines obtained in step S2 and the Beltrami coefficient field obtained in step S3 A quasi-conformal mapping constraint model is established. The structural feature line constraint is as follows:

[0057] Make the text lines tend to be horizontal or vertical.

[0058] S4.3 Quasi-conformal mapping calculation: Calculate the mapping function by solving the Beltrami equation.

[0059] The mapping satisfies

[0060] It is obtained by solving the sparse linear system.

[0061] According to quasi-conformal theory, when

[0062] Time mapping It is a diffeomorphism, thus ensuring the bijectivity and continuity of the mapping.

[0063] S5, Texture Mapping and Image Reconstruction S5.1 Mesh Transformation: Based on the mapping function , grid Mapped to target planar mesh

[0064] S5.2 Texture Mapping: Mapping the original image The pixel values ​​are mapped to the new grid.

[0065] S5.3 Image Reconstruction: Calculating the target image through interpolation.

[0066] Obtain the corrected document image

[0067] The technical solution of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0068] Example 1 In one optional implementation, a document image geometric correction method based on quasi-conformal mapping is provided, the specific steps of which are as follows: 1. Receive the input distorted document image The image is then preprocessed to extract document regions. Preferably, the document foreground region is obtained using a foreground segmentation model. This is done to remove background interference and determine document boundaries. The deep learning models include a deep learning-based structural feature line detection model and a deep learning-based Beltrami coefficient field prediction model. 2. Using a deep learning-based structural feature line detection model, the foreground image is processed. Geometric feature extraction is performed to obtain structural feature lines in the image. The structural feature lines include text lines, page boundaries, and column separators, which are used to describe the overall geometric structure of the document image.

[0069] 3. A deep learning-based Beltrami coefficient field prediction model to predict the Beltrami coefficient field in an image. ,in

[0070] The Beltrami coefficients are used to characterize the degree of local angular distortion in the mapping and ensure that the mapping is differentiable and orientation-preserving, thereby describing fine-grained deformation information in the image.

[0071] 4. Based on the structural feature lines With Beltrami coefficient field Construct a triangular mesh on the document area. The structural feature lines are then embedded into the mesh to form a constrained geometric representation model. .

[0072] 5. For example Figure 7 As shown, the mapping function is solved based on the quasi-conformal mapping theory.

[0073] in The target plane is a standard rectangular planar domain. By solving the discrete Laplace equation under an auxiliary metric, a quasi-conformal mapping that satisfies the structural constraints is calculated, making the structural feature lines tend to be horizontal or vertical in the target plane. At the same time, the local angular deformation is controlled by the Beltrami coefficient field to maintain the consistency of character shape and proportion.

[0074] 6. Based on the obtained quasi-conformal mapping Texture mapping is performed on the original image, projecting the pixel values ​​in the distorted image onto the target plane domain, thereby generating a corrected planar document image. .

[0075] like Figure 8 As shown, quantitative experiments were conducted on several public datasets, including DocUNet, DIR300, and UVDoc. Multiple evaluation metrics were used to assess the quality of document image correction, including structural similarity (MS-SSIM), local distortion (LD), edge distortion (ED), and character error rate (CER).

[0076] Table 1 presents the quantitative experimental results of the method of the present invention on different datasets.

[0077]

[0078] Table 1 presents the experimental results of the method of this invention on different datasets. All the above metrics were obtained statistically from the test images in each dataset. The experimental results show that the method of this invention can effectively restore the geometric structure of distorted document images, making text lines more horizontal and maintaining character shape consistency. Furthermore, it achieves superior results on multiple evaluation metrics, indicating that the method has high accuracy and stability in terms of geometric structure restoration and visual quality.

[0079] Example 2 This embodiment describes the structural feature line detection model in step S2, such as... Figure 3 As shown, the structural feature line detection model uses the document foreground image. As input, the output is a probability map of structural feature lines in multiple channels.

[0080] Preferably, the structural feature line detection model uses Swin-Unet as the backbone network. The network's encoding end utilizes multi-layered Swin Transformer blocks to extract contextual information from the image at different scales, while the decoding end gradually restores the spatial resolution through skip connections, thereby outputting high-resolution structural feature prediction results. To improve the structural feature line detection model's ability to detect thin text lines, a combination of Dice loss and binary cross-entropy loss is used during training: on the one hand, Dice loss enhances the optimization of overlap for thin line targets; on the other hand, binary cross-entropy loss ensures pixel-level classification stability. Specifically, the structural feature line detection model adopts the Swin-Unet encoder-decoder architecture, which combines the long-range dependency modeling capabilities of the Swin Transformer with the multi-scale feature fusion advantages of U-Net. The model's input is the document image after foreground segmentation. The resolution is B×H×W×3, where B is the batch size, and H and W are the image height and width, respectively. The model outputs a two-channel binary mask, representing the horizontal feature lines. and vertical feature lines The output dimensions are B×H×W×2.

[0081] The encoder employs a four-layer downsampling structure, progressively reducing the feature map resolution while increasing the number of channels. The first layer downsamples the input B×H×W×3 to B×H / 4×W / 4×C, where C is the base number of channels (typically set to 64). The second layer continues downsampling to B×H / 8×W / 8×2C, the third to B×H / 16×W / 16×4C, and the fourth to B×H / 32×W / 32×8C. Each downsampling layer contains a Swin Transformer block, which uses a shifted window mechanism (window size 7×7) for multi-head self-attention computation, with 8 attention heads and a feature dimension of 64. Following the Swin Transformer block is a feedforward network (MLP) with hidden layer dimensions 4 times the feature dimension and GELU activation function. Each layer includes layer normalization and residual connections to ensure gradient flow and training stability.

[0082] The decoder employs a symmetrical four-layer upsampling structure to progressively restore the feature map resolution. The first layer upsamples B×H / 32×W / 32×8C to B×H / 16×W / 16×4C, the second layer to B×H / 8×W / 8×2C, the third layer to B×H / 4×W / 4×C, and the fourth layer to B×H×W×2. Each upsampling layer includes bilinear interpolation upsampling, skip connections to fuse features from the corresponding layer of the encoder, and a Swing Transformer block to further process the fused features. The skip connection design allows detailed information from the encoder to be directly transmitted to the decoder, which helps to accurately locate the position of feature lines.

[0083] Furthermore, during the training set construction process, text lines, page boundaries, and vertical separator lines can be converted into pixel-level supervised maps, and the lines can be morphologically expanded to a certain width to improve the convergence stability of the network in the early stages of training. For document images of different languages, fonts, and layout types, data augmentation methods can be used to improve the model's generalization ability. These data augmentations include, but are not limited to, random rotation, scaling, affine perturbation, brightness and contrast changes, blur noise perturbation, and local occlusion simulation.

[0084] Specifically, such as Figure 5 As shown, the structural feature line detection model is modeled as a semantic segmentation task, employing a combination of Dice loss and binary cross-entropy (BCE) loss.

[0085] The total loss function is .

[0086] Wherein, Dice loss is defined as , in p This is a probability plot for prediction. t For the truth mask, ε =10 -5 It is the numerical stability constant. i The index is the pixel. Dice loss can effectively handle the class imbalance problem (the thin line structure has a small proportion relative to the background), while BCE loss provides stable pixel-level gradients. Combining the two can achieve better training results.

[0087] The structural feature line detection model is trained using the Adam optimizer, where β 1 =0.9，β 2 = 0.999, ε = 10 -8 The learning rate employs a multinomial decay strategy, with an initial learning rate of 5 × 10⁻⁶. -4 The attenuation formula is ,in power =0.9, T This represents the total number of iterations. The batch size was set to 16, and the training rounds consisted of 1000 epochs. Training was performed on an NVIDIA RTX 3090 GPU using the DocLine dataset, which contains approximately 12,000 samples with an image resolution of 2016×1344.

[0088] To enhance the model's generalization ability, various data augmentation techniques were applied during training, including random horizontal flipping (probability 0.5), random vertical flipping (probability 0.5), random rotation (-15° to 15°), random scaling (0.8 to 1.2 times), random brightness adjustment (±0.2), and random contrast adjustment (±0.2). During training, the Dice coefficients on the validation set were monitored. An early stopping mechanism was triggered when the validation Dice coefficients showed no improvement for 50 consecutive epochs, and the optimal model weights at that point were saved.

[0089] During the inference phase, the probability map output by the structural feature line detection module undergoes post-processing steps such as thresholding, connected component analysis, thinning, and curve fitting to obtain a more stable set of discrete structural feature lines. For locally broken text lines, neighborhood connections or curve fitting can be used for completion; for redundant false detection segments, they can be screened out based on consistency in length, position, and direction, thus providing more reliable structural constraints for subsequent geometric solutions.

[0090] Example 3 This embodiment explains the Beltrami coefficient field prediction model in step S3, such as... Figure 4As shown, the Beltrami coefficient field is an important geometric quantity describing local angular distortion, capable of characterizing fine-grained geometric deformations caused by page curling, folding, and perspective changes in document images.

[0091] Preferably, a DPT network is used to perform dense regression on the real and imaginary parts of the Beltrami coefficients. The DPT network uses a Transformer to extract global context and then obtains high-resolution prediction results through multi-scale feature fusion. Since the Beltrami coefficients are mainly defined within the document region, a mean squared error loss function with foreground mask is used during training to accumulate prediction errors only within the document region, thereby improving the model's ability to learn effective regions.

[0092] Specifically, the Beltrami coefficient field prediction model employs a Dense Prediction Transformer (DPT) neural network architecture, specifically designed for dense prediction tasks. The model input is the foreground segmented document image. The resolution is B×H×W×3. The model outputs a two-channel Beltrami coefficient field, representing the real part of the coefficients. and the virtual part The output size is B×H×W×2. The Beltrami coefficients are complex numbers, and their real and imaginary parts encode angular distortion information mapped in different directions.

[0093] DPT first segments the input image into 16×16 patches, linearly projecting each patch onto a 768-dimensional feature space and adding positional encoding. The Transformer encoder contains 12 layers, each with 12 attention heads, and the hidden layer dimension is 3072. Each layer includes a multi-head self-attention mechanism, where attention is calculated as follows: ,in =64 dimensions per head. Following multi-head self-attention is a feedforward network (FFN), defined as... Each layer includes layer normalization and residual connections to ensure the training stability of deep networks.

[0094] DPT employs a multi-scale feature fusion strategy to capture information at different scales. It extracts features from different layers of the Transformer encoder: Reassemble. 32 Extract features with a resolution of H / 32×W / 32 from layer 3, and reassemble. 16Features with a resolution of H / 16×W / 16 are extracted from layer 6, Reassemble8 extracts features with a resolution of H / 8×W / 8 from layer 9, and Reassemble4 extracts features with a resolution of H / 4×W / 4 from layer 12. These features are fused through the Fusion module, including bilinear interpolation upsampling to a uniform resolution, element-wise addition fusion, and further processing by convolutional layers. Multi-scale fusion allows the model to utilize both global contextual information and local detail information simultaneously.

[0095] The fused multi-scale features are processed by a regression head, which contains three 1×1 convolutional layers: the first layer reduces the number of channels from C to 128, the second layer reduces it from 128 to 64, and the third layer reduces it from 64 to 2 (output). and The ReLU activation function is used between each convolutional layer. For layers from the twisted domain... M To the normative domain M' mapping f The Beltrami coefficient is defined as follows: ,in and This is the Wirtinger derivative. For a triangular mesh, for each vertex... (Twisted Domain) and For a triangular facet within the (gauge domain), the Beltrami coefficient is calculated as follows:

[0096] To enhance the physical plausibility of Beltrami coefficient predictions, some implementations may impose a modulus constraint on the network output to ensure it is less than 1, thus maintaining stability. For example, the output range can be limited by the activation function, or a penalty term for out-of-bounds regions can be added to the loss function to reduce the probability of numerically unstable predictions.

[0097] Furthermore, such as Figure 6 As shown, joint training can also be performed by combining structural feature line information, so that the Beltrami coefficient field has better geometric consistency near the text line, thereby improving the overall mapping quality.

[0098] Specifically, Beltrami coefficient prediction is modeled as a dense regression task, employing Masked Mean Squared Error (MaskedMSE) loss. Loss function:

[0099] in For the predicted Beltrami coefficient plot, The true value is the Beltrami coefficient. m Foreground mask, i`⊙` represents the pixel index, and `⊙` denotes element-wise multiplication. The use of a mask ensures that only pixels within the document region contribute to the loss, avoiding the influence of background noise on training. The model uses the Adam optimizer with parameters... β 1 = 0.9 β 2 = 0.999 ε =10 -8 The learning rate employs a multinomial decay strategy, with an initial learning rate of 5 × 10⁻⁶. -4 The attenuation formula is .

[0100] The batch size was set to 16, the training epochs were 1000, and the training was performed on an NVIDIA RTX 3090 GPU. The DocBeltrami dataset was used, containing approximately 12,000 samples with an image resolution of 2016×1344. Data augmentations included random horizontal flipping (probability 0.5), random vertical flipping (probability 0.5), random rotation (-15° to 15°), random scaling (0.8 to 1.2x), random brightness adjustment (±0.2), random contrast adjustment (±0.2), and random Gaussian noise (standard deviation 0.01). During training, the MSE loss on the validation set was monitored, and early stopping was triggered when the validation MSE showed no improvement for 50 consecutive epochs, thus saving the optimal model weights.

[0101] During the inference phase, the Beltrami coefficient field can be lightly smoothed or edge-preserving filtered as needed to reduce interference from isolated local noise points on the quasi-conformal mapping solution. For document images with large blank areas, a foreground mask can be used to restrict the Beltrami coefficients to specific regions, retaining prediction results only within the valid document area. During inference, the model predicts pixel-level... and Figure. For each triangular mesh facet, the facet-level Beltrami coefficient is obtained by averaging the predicted values ​​within that facet. To ensure the effectiveness of the quasi-conformal mapping, it is necessary to ensure... The clip operation restricts out-of-range values ​​to the valid range.

[0102] Example 4 This embodiment further explains the process of solving the constrained quasi-conformal mapping in step S5, such as... Figure 7 As shown, the distorted document image domain is discretized into a triangular mesh. M Its vertex set is V The set of edges is E The set of face pieces is F The local metric for each edge is corrected based on the Beltrami coefficient field to construct the edge length under an auxiliary metric. Then, two harmonic scalar fields are solved under this auxiliary metric. and ,in The horizontal coordinates corresponding to the target rectangular region. The vertical coordinates correspond to the target rectangular domain. For boundary vertices, rectangular boundary conditions are applied; for vertices on structural feature lines, constraints are applied to maintain constant coordinates within the target domain. This is obtained by solving the discrete Laplace equations. and Then, the corresponding complex-valued form can be constructed, and the coordinates of the mesh vertices in the target rectangular domain can be obtained through discrete integration, thus obtaining the constrained quasi-conformal mapping. .

[0103] In this process, structural feature line constraints are mainly used to restore the overall straight structure of text lines and page boundaries, while the Beltrami coefficient field is used to provide local geometric supplementary information in areas where structural features are insufficient. The two work together to improve both the overall shape of the document and the quality of local characters.

[0104] Furthermore, in some implementations, the horizontal and vertical structure lines can be mapped to two mutually orthogonal harmonic coordinate fields, and the final rectangular parameter domain can be determined jointly by boundary conditions and internal line constraints. If the document has a complex layout, such as multi-column layout, table areas, or mixed text and image areas, the adaptability of the mapping can be enhanced by regional constraints, local weighting, or multi-level solutions. For sparsely defined local feature areas, the Beltrami coefficient field is mainly relied upon to provide supplementary geometric information to prevent unreasonable deformation of the mapping in weakly constrained areas.

[0105] Example 5 This embodiment describes the construction of training data and experimental verification methods for the method of the present invention. In a preferred embodiment, the structural feature line detection model and the Beltrami coefficient field prediction model can be trained based on a document image de-distortion dataset. The dataset includes distorted document images, corresponding standard planar document images, and dense correspondence information between the two. Based on the dense correspondence information, structural feature line supervision labels and Beltrami coefficient supervision labels can be constructed.

[0106] In experimental evaluation, the distortion removal effect can be evaluated using metrics such as structural similarity, multi-scale structural similarity, local distortion, edge distortion, character error rate, and text line straightness. Experimental results show that the method of this invention achieves superior geometric correction accuracy and visual quality on multiple public datasets, especially in terms of text line straightness and character shape preservation.

[0107] Furthermore, to fully verify the effectiveness of the method of this invention, tests were conducted on various publicly available datasets and real-world photographic samples. The test objects included samples with strong perspective distortion, as well as samples with significant curling, folding, and local bulging. Experimental results show that this invention has better structural recovery capabilities than methods relying solely on displacement regression in complex nonlinear deformation scenarios; and in scenarios with sparse structural lines, it has better local character shape preservation capabilities than methods relying solely on line constraints, demonstrating the advantages of dual-priority collaborative design. (Example 6) This embodiment provides a system for implementing the above method. The system includes: (1) Foreground segmentation module, used to extract document regions from the input image; (2) Structural feature detection module, used to detect structural feature lines such as text lines and page boundaries; (3) Beltrami coefficient prediction module, used to predict the Beltrami coefficient field corresponding to local angular distortion of the image; (4) Mapping solution module, used to solve constrained quasi-conformal mappings by combining structural feature lines and Beltrami coefficient fields; (5) Image reconstruction module, used to perform texture mapping according to the mapping relationship and generate a dedistorted planar document image.

[0108] In practical deployment, the above system can run as a standalone software program on personal computers, servers, mobile terminals, or dedicated scanning equipment, or it can be integrated into application systems such as document acquisition, archive digitization, OCR preprocessing, mobile office, or intelligent question answering. The modules can exchange intermediate data such as document foreground masks, structural feature lines, Beltrami coefficient fields, triangular meshes, and mapped coordinates through a unified data interface, thereby forming a complete document image distortion removal processing chain.

[0109] The present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory. When executed by the processor, the computer program is used to implement the above-described document image de-distortion method based on quasi-conformal mapping. The present invention also provides a computer-readable storage medium storing a computer program thereon for implementing the above-described method.

[0110] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Those skilled in the art can readily implement the present invention based on the accompanying drawings and the above description. However, any modifications, alterations, or variations made by those skilled in the art without departing from the scope of the present invention, utilizing the disclosed technical content, are equivalent embodiments of the present invention. Furthermore, any modifications, alterations, or variations made to the above embodiments based on the essential technology of the present invention are still within the protection scope of the present invention.

Claims

1. A method for geometric correction of document images, characterized in that, include: Obtain the document region from the distorted document image; Based on a deep learning model, structural feature lines of document regions and Beltrami coefficient fields of local angular distortion are extracted from distorted document images. A geometric constraint model is constructed based on the structural feature lines and the Beltrami coefficient field of the local angular distortion. The quasi-conformal mapping from the distorted image domain to the target plane domain is solved based on the geometric constraint model. Based on the quasi-conformal mapping, the distorted document image is mapped and reconstructed to generate a corrected document image.

2. The document image geometric correction method according to claim 1, characterized in that, Based on a deep learning model, structural feature lines of document regions and Beltrami coefficient fields of local angular distortion in distorted document images are extracted, specifically including: A deep learning-based structural feature line detection model is used to learn and predict features of document regions in distorted document images, outputting a probability map of structural feature lines. Structural feature lines describing the macroscopic geometric structure in the document image are obtained through thresholding or post-processing. The structural feature lines include at least one of text lines, page boundaries, or page structure lines. Using a deep learning-based Beltrami coefficient field prediction model, local geometric deformation prediction is performed on the document region of the distorted document image to obtain the Beltrami coefficient field corresponding to the local angular distortion. Each coefficient in the Beltrami coefficient field consists of a real part and an imaginary part, which are used to represent the direction and degree of local deformation, and its magnitude is less than 1.

3. The document image geometric correction method according to claim 1, characterized in that, A geometric constraint model is constructed based on the structural feature lines and the Beltrami coefficient field of the local angular distortion, specifically including: The document region of the distorted document image is discretized into a grid structure, and the structural feature lines are embedded in the grid structure as constraints. Based on the Beltrami coefficient field of the local angular distortion and the constraint conditions, a geometric constraint model satisfying the Beltrami equation is constructed.

4. The document image geometric correction method according to claim 3, characterized in that, The equation satisfies the Beltrami equation: in: The Beltrami coefficient; Used to ensure that the mapping is a differential homeomorphism.

5. The document image geometric correction method according to claim 1, characterized in that, In the process of solving the quasi-conformal mapping from the distorted image domain to the target plane domain, a straight line constraint is applied to the embedded structural feature lines, so that the text lines in the mapping result tend to be arranged in a horizontal or vertical direction.

6. The document image geometric correction method according to claim 1, characterized in that, The texture mapping and image reconstruction include: According to the quasi-conformal mapping, the pixel information of the original distorted document image is projected onto the target planar grid; The grayscale or color value of each pixel in the target image is calculated using interpolation methods to generate the corrected document image.

7. The document image geometric correction method according to claim 1, characterized in that, To obtain the document region of a distorted document image, specifically: Document region extraction is performed on the acquired distorted document image. The document region extraction is implemented through an image segmentation model, which is used to separate the document region from the original distorted document image and remove background interference to obtain the document region of the distorted document image.

8. A document image correction system for performing the document image geometric correction method according to any one of claims 1 to 7, characterized in that, include: Preprocessing module: Used to extract document regions from distorted document images; Feature extraction module: Extracts structural feature lines of document regions and Beltrami coefficient fields of local angular distortion from distorted document images based on a deep learning model; Mapping calculation module: Constructs a geometric constraint model based on the structural feature lines and the Beltrami coefficient field of the local angular distortion, and solves the quasi-conformal mapping from the distorted image domain to the target plane domain based on the geometric constraint model; Image generation module: Based on the quasi-conformal mapping, the distorted document image is mapped and reconstructed to generate a corrected document image.

9. A computer device, characterized in that, include: One or more processors; Memory; The computer program stored in the memory; When the computer program is executed by the processor, the device performs the document image geometric correction method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the document image geometric correction method according to any one of claims 1 to 7.

Images