Document correction method based on axis alignment property

Abstract

A document correction method based on an axis alignment property comprises the following steps: S100: obtaining a data set containing a distorted document image and a corresponding flattening label, constructing a convolutional neural network model and training, and adding an axis alignment geometric constraint in a loss function in the training process; s200, performing preliminary distortion-removing prediction reasoning on a to-be-corrected document image, calculating a minimum circumscribed rotating rectangle, and performing axis alignment rotating cutting to obtain an image area of an aligned document; and S300, re-inputting the obtained image of the aligned document into the neural network model obtained through training, and outputting the corrected flattened document image as a final result. According to the method, the correction precision and robustness can be remarkably improved in a complex document deformation scene, and the method has high application value.

Description

Technical Field

[0001] The present disclosure belongs to the fields of computer vision and image processing technology, and in particular relates to a document correction method based on axis alignment properties. Background Art

[0002] With the increasing popularity of mobile devices, capturing and digitizing paper documents with cameras has become a common practice. However, document images captured by cameras often exhibit geometric distortion and uneven lighting, which impacts the reading experience and subsequent processing. In recent years, deep learning methods have been widely used in document dedistortion. A typical approach involves training a convolutional neural network to directly regress the distorted document image to predict its deformation map, or flattened result. Existing learning-based methods rely on supervised training using a large number of synthetic warped-flattened image pairs, incorporating various supervisory signals such as depth maps, geometric semantics, control points, and document layout as training constraints. While these data-driven methods have achieved some success with simple deformations, they often fail when processing documents with complex distortions. This is primarily due to their heavy reliance on strong, manually annotated supervisory signals. For example, control points, while effective, lack clear geometric and physical meaning, while text lines, while possessing some geometric meaning, are difficult to robustly extract and lack generalizability. These methods fail to fully utilize the inherent geometric information inherent in the paper document as constraints, making it difficult for the network to learn the correct flattening rules in complex situations. In addition, existing evaluation indicators often cannot effectively measure the consistency between the residual distortion degree of the flattened result and the human eye's perception. Therefore, it is necessary to propose a new evaluation standard to better measure the effect of document dedistortion. Summary of the Invention

[0003] In order to solve the above problems, the present disclosure provides a document correction method based on axis alignment, which includes the following steps:

[0004] S100: Obtain a dataset containing distorted document images and corresponding flattened annotations, build a convolutional neural network model and train it, and add axis-aligned geometric constraints to the loss function during the training process;

[0005] S200: performing preliminary dedistortion prediction inference on the document image to be corrected, calculating the minimum circumscribed rotation rectangle, and performing axis-aligned rotation cropping to obtain an image region aligned with the document;

[0006] S300: The obtained aligned document image is re-inputted into the trained neural network model, and the corrected flattened document image is output as the final result.

[0007] In addition, the present invention also discloses a document correction device based on axis alignment properties, comprising:

[0008] A device for acquiring a dataset containing distorted document images and corresponding flattened annotations, constructing a convolutional neural network model and training it, and incorporating axis-aligned geometric constraints into the loss function during training;

[0009] A device for performing preliminary dedistortion prediction inference on the document image to be corrected, calculating the minimum circumscribed rotation rectangle, and performing axis-aligned rotation cropping to obtain an image area aligned with the document;

[0010] A device for re-inputting the obtained aligned document image into the trained neural network model and outputting the corrected flattened document image as the final result.

[0011] In addition, the present invention also discloses a computer storage medium, wherein the storage medium includes computer instructions, and when the computer instructions are run on the computer, the computer executes the method.

[0012] In addition, the present invention also discloses an electronic device, wherein the electronic device includes:

[0013] A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein:

[0014] When the processor executes the program, the method described is implemented.

[0015] This method introduces the document's axis-aligned geometric prior into the algorithm, and uses appropriate training strategies and evaluation metrics to improve the accuracy and robustness of dewarping. This method achieves the best performance among existing technologies on multiple public document dewarping benchmarks, and significantly improves the AAD metric by approximately 18.2% to 34.5% compared to existing methods. This method not only demonstrates excellent results in objective metrics, but also verifies, through the AAD metric, that the correction results are visually more flat and natural. This method achieves an efficient, robust, and well-evaluated document dewarping method that can be widely used in document scanning, OCR preprocessing, and other fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a flow chart of a document correction method based on axis alignment provided in one embodiment of the present disclosure;

[0017] Figure 2 This is a diagram of the fully convolutional document correction network architecture provided in one embodiment of the present disclosure;

[0018] Figure 3 is a schematic diagram of utilizing the axis alignment property in the document correction process provided in one embodiment of the present disclosure;

[0019] Figure 4This is a flow chart of a calculation process for adding axis-aligned geometric constraints during training, provided in one embodiment of the present disclosure;

[0020] Figure 5 is a flow chart of a document correction reasoning process including an axis alignment preprocessing method provided in one embodiment of the present disclosure;

[0021] Figure 6 This is a visualization diagram of the AAD index calculation results provided in one embodiment of the present disclosure;

[0022] Figure 7 This is a diagram comparing ablation results of different applications of axis alignment properties provided in one embodiment of the present disclosure. DETAILED DESCRIPTION

[0023] In one embodiment, Figure 1 As shown, the present disclosure provides a document correction method based on axis alignment properties, which includes the following steps:

[0024] S100: Obtain a dataset containing distorted document images and corresponding flattened annotations, build a convolutional neural network model and train it, and add axis-aligned geometric constraints to the loss function during the training process;

[0025] S200: performing preliminary dedistortion prediction inference on the document image to be corrected, calculating the minimum circumscribed rotation rectangle, and performing axis-aligned rotation cropping to obtain an image region aligned with the document;

[0026] S300: The obtained aligned document image is re-inputted into the trained neural network model, and the corrected flattened document image is output as the final result.

[0027] In this embodiment, the method uses a deep learning network to predict a two-dimensional dewarped grid. During training, a specialized axis-aligned geometric constraint loss is designed to ensure that document feature lines in the predicted grid tend to align horizontally or vertically. During inference, a corresponding axis-aligned preprocessing strategy is designed to reduce the correction difficulty, thereby achieving high-precision flattening correction. Furthermore, the correction effect is comprehensively evaluated using the newly proposed axis-aligned distortion (AAD) metric. This method significantly improves correction accuracy and robustness in complex document deformation scenarios, demonstrating its high application value.

[0028] This method incorporates axis-aligned geometric constraints into the training of a document dewarping neural network. For a well-flattened document image, its internal feature lines (such as text lines and table lines) should be aligned horizontally or vertically as much as possible, that is, aligned with the image axes. Leveraging the inherent property of paper documents, which form a regular grid and are axis-aligned when flattened, this method designs an axis-aligned geometric constraint loss function. This allows the neural network to train not only based on pixel-level errors but also guided by geometric constraints. Specifically, the two-dimensional dewarped grid predicted by the network is mapped to the document plane coordinate system (UV space). The horizontal (row) and vertical (column) coordinate variances of the grid are calculated to measure the deviation of the flattened result from the ideal axis-aligned state. This deviation is then incorporated into the training as a constraint loss. By minimizing this axis-aligned loss, the document feature lines in the dewarped result predicted by the network are forced to align horizontally or vertically, significantly improving the accuracy of document dewarping. This geometric constraint has a clear physical meaning and is universal, addressing the lack of geometric meaning in existing strong supervisory signals.

[0029] To address the issue of unstable dewarping results under varying input conditions during the inference (i.e., practical application) phase, this method proposes an axis-aligned input preprocessing strategy. Ideally, the document to be rectified should occupy as large a proportion of the input image as possible, and its main orientation should be aligned with the image coordinate axes to reduce the network's correction complexity. To this end, this method performs axis-alignment preprocessing on the input image before dewarping it. This preprocessing uses the positional distribution of the initially predicted document grid to calculate the minimum bounding rectangle of the document region. The image is then rotated according to this rectangle, aligning the document's outline with the image coordinate axes. A rectangular region is then cropped as the new input. The document dewarping algorithm is then re-applied to the preprocessed image to obtain a more refined flattened result. This process can be iterated multiple times, if necessary, to gradually approach the optimal correction. Unlike existing preprocessing methods that require additional segmentation or detection models, this axis-alignment preprocessing directly utilizes the grid information output by the dewarping network to infer the document's pose, eliminating the need for a new model and resulting in a more streamlined and efficient process. This strategy effectively improves the robustness of document dewarping under different initial rotation angles and sizes, enabling the method of the present invention to achieve good correction effects under various conditions.

[0030] In another embodiment, the loss function further includes a loss for supervising two-dimensional mesh prediction, a loss for supervising three-dimensional document shape reconstruction, and a structural similarity loss for measuring the similarity between the output image and the true flattened image.

[0031] In another embodiment, the convolutional neural network model adopts a fully convolutional structure, wherein the encoder extracts multi-scale features of the distorted document image layer by layer, and the decoder gradually upsamples to restore the spatial resolution and outputs the document deformation mesh.

[0032] In this embodiment, Figure 2 As shown in the figure, the convolutional neural network model uses a relatively simple two-headed fully convolutional encoder architecture. Specifically, the input image first passes through two convolutional downsampling layers, each of which uses a 5×5 convolution kernel and reduces the image size by half. Subsequently, the network contains three residual modules, which form a spatial pyramid in which the void convolution is stacked. Finally, the network is divided into two output heads, each of which contains two layers of convolution to predict the two-dimensional dewarped grid. and 3D document surface coordinates .

[0033] Taking into account all aspects of the document dedistortion task, this method constructs the following total training loss function to optimize the neural network parameters:

[0034]

[0035] in, is the L1 loss for the 2D undistorted grid, L1 loss for 3D shape reconstruction; is the structural similarity loss, which is used to measure the corrected image Flattened image with real differences in perception; is the weight hyperparameter of each loss item, which can be set according to training needs. During training, we use the Adam optimizer to optimize the loss. We can achieve better convergence by adjusting the loss parameters of each weight. =0.05.

[0036] In each training iteration, using the prepared training samples, the distorted image is input into the network to obtain the predicted output, and the above losses are calculated and summed to obtain the total loss , adjust the network weight parameters through the back-propagation algorithm. Axis-aligned geometric constraint loss In the early stages of training, this strategy can be used to guide the network to learn the basic flattening geometry, while also providing effective constraints for fine-tuning network details later. By training with this strategy, an optimized document dewarping model can be obtained.

[0037] Obtain a sample dataset for training the document dewarping model. The dataset should include pairs of distorted document images and their corresponding ideal flattened results (or equivalent deformation map annotations). Synthetic data can be used to generate large-scale training pairs, for example, using 3D deformable modeling software to distort flat document images to generate annotated training samples. Each training sample should at least contain: a distorted document image , the corresponding two-dimensional dedistorted grid true value (a two-dimensional vector field that maps distorted image coordinates to flattened image coordinates), and the corresponding three-dimensional document surface coordinates (Shape mapping of the document in three-dimensional space). The two-dimensional grid is usually The coordinate matrix is ​​represented by The training set can also include distorted documents taken in real life and manually corrected and annotated to improve the model's adaptability to real data.

[0038] Figure 3 This method is the use of the axis-aligned property of the correction result in the document correction method proposed by this method. The first row is based on the discovery of the axis-aligned property of the feature lines in the expected correction result. The second row is a deep learning document correction method based on the Grid grid designed for this discovery. On this basis, based on this method, different effective strategies and methods based on the axis-aligned property are proposed in the training stage, inference stage and evaluation stage to help the network better learn and correct, improve the robustness of the correction effect, and better evaluate the correction results. Figure 3 As shown in the figure, a convolutional neural network is built for document dewarping. A fully convolutional architecture is used: the encoder extracts multi-scale features of the distorted document image layer by layer, and the decoder gradually upsamples to restore the spatial resolution and outputs the deformed document mesh.

[0039] The network input is a distorted document image , the output includes two branches: one is the two-dimensional dedistorted grid , used to directly map the distorted image to the flattened image coordinates; the second is the three-dimensional grid , used to reconstruct the three-dimensional shape of the document (optional branch, decide whether to output it according to needs). is the input image resolution, is the grid resolution (usually much smaller than , where h=45 and w=31 are set here). The network can predict the above two-dimensional and three-dimensional grid coordinates through multiple layers of convolution during the decoding phase. For two-dimensional grid prediction, the network output can be designed by referring to the UV coordinate expansion method.

[0040] In another embodiment, the axis-aligned geometric constraint loss is:

[0041]

[0042] in, To get the horizontal axis alignment error, we need to sum the vertical variances of all rows. The vertical axis alignment error is obtained by summing the horizontal variances of all columns.

[0043] In this embodiment, during the training process, in order to inject the document's axis alignment prior into the network learning, this method introduces the axis alignment geometric constraint loss . Figure 4 This is a schematic diagram of applying axis alignment geometric constraints during the training phase. The first row is the two-dimensional grid predicted by the deep learning network from the distorted image during network training in image space; the right part of the second row shows how the predicted grid is mapped to the document's UV plane coordinate system using an interpolation function; the left part of the second row illustrates the method for calculating the horizontal and vertical axis alignment errors in UV space, that is, calculating the vertical coordinate variance of each row and the horizontal coordinate variance of each column of the grid, and using them as constraint losses. Feedback to network training optimization. Figure 4 As shown, first the two-dimensional grid currently predicted by the network is , according to the mapping pairs in the GT results ( ), mapped to the document's UV plane coordinate system through interpolation, and the corresponding grid point set is obtained ,in Indicates the Rank The coordinates of the column grid points in the UV plane. Ideally, if the document is completely flattened and aligned, the mapped grid should be a regular axis-aligned grid, that is, all points in each row should be on the same horizontal line, and all points in each column should be on the same vertical line. Based on this, the present invention calculates the deviation of the grid in the horizontal and vertical directions respectively: for each row , get the vertical coordinates of all grid points in this row , calculate its variance ; For each column , get the horizontal coordinates of all grid points in this column , calculate its variance The horizontal axis alignment error is obtained by summing the vertical variance of all rows. , the vertical axis alignment error is obtained by summing the horizontal variances of all columns . Then the axis-aligned geometric constraint loss is defined as:

[0044]

[0045] By minimizing this loss, the grid predicted by the network can be constrained to be as close as possible to a regular axis-aligned grid in the UV plane, thereby ensuring that the document feature lines in the output image are corrected in the horizontal and vertical directions. The weight of the axis-alignment loss can be adjusted during training to balance it with other loss terms. In actual implementation, a small constant term can be added to the variance calculation to avoid numerical instability. .

[0046] In another embodiment, step S200 further includes:

[0047] S201: Inputting the document image to be corrected into the trained neural network model to obtain a preliminary dedistortion result and its corresponding prediction grid;

[0048] S202: For the point set of the prediction grid in the flattened image coordinate system, calculate the rectangle with the smallest area that can enclose all the grid points;

[0049] S203: Rotate and correct the document image to be corrected according to the tilt angle of the rectangle obtained above, so that the document body is roughly aligned with the image frame, and then crop the image within the minimum circumscribed rectangle after rotation to obtain an aligned and centered document image.

[0050] In this embodiment, after training is completed, the model is deployed for actual document correction applications. In the inference stage, for any input document image to be corrected, this method preferably performs axis alignment preprocessing before dewarping to obtain the best effect. The process is as follows Figure 5 shown. Figure 5 This is a flowchart of the axis alignment preprocessing strategy implemented in the inference stage. The distorted document is input for the first inference, and the corresponding 2D Grid control points are predicted. The outer rectangular frame is the minimum area rotated rectangle wrapped by this Grid point. After that, if preprocessing is selected, the rotated rectangular area is rotated and cropped before inference prediction. If no preprocessing is performed, the correction result is directly output through interpolation and remapping methods. Through two different output correction results, the figure vividly compares the difference between the dewarping results before and after preprocessing, highlighting the role of the preprocessing strategy of the present invention in improving the correction effect.

[0051] 1. Preliminary dedistortion prediction inference. Input the original input distorted document image into the trained neural network model to obtain the preliminary flattened result image and the corresponding predicted two-dimensional grid. (Can be output through the network Sampling the original image to obtain the preliminary correction result).

[0052] 2. Calculate the minimum circumscribed rotation rectangle. Based on the initial predicted two-dimensional grid The distribution of grid points in the prediction is used to estimate the boundaries of the document region in the prediction result. Preferably, a minimum bounding rectangle algorithm is used: for the set of points in the prediction grid in the flattened image coordinate system, a rectangle with the smallest area that encloses all grid points is calculated. This rectangle is typically rotated by a certain angle. This minimum bounding rectangle approximately reflects the tilt of the document in the original image, with its long side corresponding to the general direction of the document content.

[0053] 3. Axis-aligned rotation and cropping. The original distorted image is rotated and corrected according to the tilt angle of the rectangle obtained above, so that the document body is roughly aligned with the image border (that is, the rectangle side is parallel to the horizontal and vertical axes of the image). Then the image is cropped within the minimum bounding rectangle after rotation to obtain an aligned and centered document image. Figure 4 As shown in the figure, after this preprocessing, the proportion of the document in the image is increased and the angle is corrected, which is beneficial for subsequent processing.

[0054] 4. Dewarping correction. The obtained preprocessed image is input into the neural network model again for dewarping. Since the document posture has been adjusted at this time, the network can correct the distorted details more easily, thereby outputting a flattened result image that is finer and more accurate than the initial prediction result. If necessary, the new output can be used again to calculate the updated rotation rectangle and repeat steps 3 and 4, iterating until the result is satisfactory. In most cases, one preprocessing and one dewarping correction can meet the requirements. Finally, the output flattened document image is the correction result. Based on the above-mentioned axis alignment preprocessing strategy, this method improves the correction effect by using its own network prediction information without adding other models. Compared with the direct correction without preprocessing, the method in this paper effectively reduces the residual distortion caused by the initial tilt or poor scale of the document, making the output document more regular.

[0055] In another embodiment, the method further includes using an evaluation index AAD to quantitatively evaluate the dewarping effect.

[0056] As far as this embodiment is concerned, this method introduces a new evaluation index in the evaluation of the document dedistortion effect - the Axis-Aligned Distortion (AAD) index. This index has a clear geometric and physical meaning, can objectively quantify the degree of distortion in the flattening result, and is highly correlated with the subjective judgment of the human eye. Specifically, the AAD index measures the correction effect by evaluating the degree of deviation of each local part of the dedistorted image relative to the horizontal / vertical direction. In implementation, the optical flow algorithm (such as SIFT-flow) is first used to calculate the pixel correspondence field between the ideal flattened image and the corrected image, that is, the horizontal displacement of each pixel is obtained. and vertical displacement Then apply the Sobel operator to the original image to extract the horizontal and vertical gradient maps, and perform normalization to obtain the gradient weights and Next, for each row of the dewarped result , calculate the gradient weighted average of the vertical displacement of each pixel in the row , and calculate the vertical displacement deviation of each pixel The weighted difference ; Correspondingly, for each column , calculate the gradient weighted average of the horizontal displacement of each pixel in the column, and calculate the horizontal displacement deviation of each pixel The weighted difference The row and column deviations are then combined to get the total deviation for that pixel: Finally, the AAD indicator is defined as the total The average value of the total deviation of pixels:

[0057]

[0058] Smaller deviations indicate better axis alignment of the corrected image. Because AAD comprehensively considers image gradient distribution and distortion displacement, it more directly quantifies the skew of important features such as table lines and text rows in documents. Therefore, compared to previous evaluation methods based solely on pixel error or OCR accuracy, it more accurately reflects the human eye's perception of residual distortion.

[0059] After obtaining the correction result, this method also provides the evaluation index AAD to quantitatively evaluate the dedistortion effect. Developers can choose whether to calculate it according to their needs. Figure 6 As shown, the AAD heat map can be superimposed on the flattened results to assist in analysis. Figure 6 The axis alignment error indicator proposed in the evaluation phase is used to visualize the results. The first row is the true value image and the directional gradient thermal image required for indicator calculation. and The second and third rows are two different correction result images and the calculated AAD index superimposed heat maps, where AAD_H and AAD_V represent the horizontal and vertical components of the AAD index respectively, and the heat map color indicates the degree of regional distortion of the correction result relative to the true image, which corresponds to the numerical results below.

[0060] In another embodiment, Figure 7 This is a comparison of two ablation experiments proposed by this method to effectively improve the correction performance. Each row in the figure shows the different correction results obtained by using different ablation designs for the input distorted document image, as well as the results overlaid on the horizontal component heat map of the AAD indicator. The constraint in the figure is the addition of axis alignment constraints during the training phase, and the preprocessing is the addition of axis alignment preprocessing strategies during the inference phase. As can be seen from the figure, in the first and third rows of images, due to the large proportion of documents in the images, the improvement brought by the use of axis alignment constraints is more obvious, while in the middle image, the improvement brought by the use of axis alignment preprocessing strategies is more obvious. The two have the characteristics of complementary performance improvement.

[0061] In another embodiment, the AAD value is obtained by calculating the optical flow field between the images before and after correction, combining the gradient weight of the original image, calculating the axis alignment deviation of each pixel and taking the average value.

[0062] In this embodiment, the specific calculation method is: by calculating the optical flow field between the images before and after correction , combined with the original image gradient weight, calculate the axis alignment deviation of each pixel The average is then taken to obtain the AAD value. A smaller AAD value indicates less deviation of the document feature lines from the horizontal / vertical axes, indicating a more successful correction. This evaluation process does not affect the document correction results, but it helps optimize and improve the model. For example, when AAD detects significant deviation in certain areas, targeted adjustments to the training data or model parameters can be made to further reduce distortion. This feedback mechanism, combined with the guidance of the AAD metric, allows for iterative improvement of document dewarping model performance.

[0063] The technical solution provided in this embodiment significantly improves the document correction effect in complex scenarios by integrating the axis-aligned geometric properties of the document into the deep learning dewarping process. The collaborative optimization of multiple losses in the training phase and the adaptive preprocessing in the inference phase enable the model to have both accurate geometric correction capabilities and good robustness and generalization. Actual tests show that this method has achieved quantitative indicators that are superior to existing technologies on standard datasets, and the corrected document images are visually flat and standardized. All of these verify the effectiveness and superiority of this axis-aligned document dewarping method.

[0064] In another embodiment, comprising:

[0065] A device for acquiring a dataset containing distorted document images and corresponding flattened annotations, constructing a convolutional neural network model and training it, and incorporating axis-aligned geometric constraints into the loss function during training;

[0066] A device for performing preliminary dedistortion prediction inference on the document image to be corrected, calculating the minimum circumscribed rotation rectangle, and performing axis-aligned rotation cropping to obtain an image area aligned with the document;

[0067] A device for re-inputting the obtained aligned document image into the trained neural network model and outputting the corrected flattened document image as the final result.

[0068] In this embodiment, model training involves obtaining a dataset containing distorted document images and corresponding flattened annotations, constructing a convolutional neural network model, and training it. During training, axis-aligned geometric constraints are added to the loss function. This constrains the two-dimensional dedistorted grid predicted by the network so that its rows and columns are as collinear and co-column as possible in the document plane coordinate system, thereby promoting axis alignment of document feature lines in the output image. The loss function also includes a loss for supervising two-dimensional grid prediction, a loss for supervising three-dimensional document shape reconstruction, and a structural similarity loss for measuring the similarity between the output image and the true flattened image, thereby comprehensively optimizing network parameters.

[0069] Axis alignment preprocessing: The document image to be corrected is input into the trained neural network model to obtain the preliminary dewarping result and its corresponding prediction grid; the minimum circumscribed rotation rectangle of the document area is calculated based on the distribution of the prediction grid in the flattened plane, and the original document image is rotated and corrected according to the angle of the rotation rectangle, and the image area of ​​the aligned document is cropped.

[0070] Document dewarping correction: The aligned document image obtained through preprocessing is re-input into the neural network model, and the corrected flattened document image is output as the final result; wherein, in the process of the neural network model predicting the document deformation, the axis alignment geometric constraint introduced during training is affected, and the characteristic lines of the document in the final result are aligned in the horizontal direction or the vertical direction.

[0071] In addition, the present invention also discloses a computer storage medium, wherein the storage medium includes computer instructions, which, when executed on a computer, enable the computer to execute any of the methods described above.

[0072] In addition, the present invention also discloses an electronic device, wherein the electronic device includes:

[0073] A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein:

[0074] When the processor executes the program, any of the above methods is implemented.

[0075] Although the embodiments of the present invention have been described above with reference to the accompanying drawings, the present invention is not limited to the above-mentioned specific embodiments and application fields. The above-mentioned specific embodiments are merely illustrative and instructive, and are not restrictive. A person skilled in the art, guided by this specification and without departing from the scope of protection of the claims of the present invention, may also devise various forms, all of which fall within the scope of protection of the present invention.

Claims

1. A document correction method based on axis alignment property, comprising the following steps: S100: Obtain a dataset containing distorted document images and corresponding flattened annotations, build a convolutional neural network model and train it, and add axis-aligned geometric constraints to the loss function during the training process; S200: performing preliminary dedistortion prediction inference on the document image to be corrected, calculating the minimum circumscribed rotation rectangle, and performing axis-aligned rotation cropping to obtain an image region aligned with the document; S300: The obtained aligned document image is re-inputted into the trained neural network model, and the corrected flattened document image is output as the final result.

2. According to the method of claim 1, preferably, the loss function also includes a loss for supervising two-dimensional grid prediction, a loss for supervising three-dimensional document shape reconstruction, and a structural similarity loss for measuring the similarity between the output image and the true flattened image.

3. The method according to claim 1, wherein the convolutional neural network model adopts a fully convolutional structure, wherein The encoder extracts multi-scale features of the distorted document image layer by layer, and the branch network outputs the document deformation mesh.

4. The method according to claim 1, wherein the axis-aligned geometric constraint loss is: ， in, To get the horizontal axis alignment error, we need to sum the vertical variances of all rows. The vertical axis alignment error is obtained by summing the horizontal variances of all columns.

5. The method according to claim 1, step S200 further comprising: S201: Inputting the document image to be corrected into the trained neural network model to obtain a preliminary dedistortion result and its corresponding prediction grid; S202: For the point set of the prediction grid in the flattened image coordinate system, calculate the rectangle with the smallest area that can enclose all the grid points; S203: Rotate and correct the document image to be corrected according to the tilt angle of the rectangle obtained above, so that the document body is roughly aligned with the image frame, and then crop the image within the minimum circumscribed rectangle after rotation to obtain an aligned and centered document image. The method according to claim 1 , further comprising using an evaluation index AAD to quantitatively evaluate the dewarping effect.

7. The method according to claim 6, wherein the AAD value is obtained by calculating the optical flow field between the images before and after correction, combining the gradient weight of the original image, calculating the axis alignment deviation of each pixel and taking the average value.

8. A document correction device based on axis alignment properties, comprising: A device for acquiring a dataset containing distorted document images and corresponding flattened annotations, constructing a convolutional neural network model and training it, and incorporating axis-aligned geometric constraints into the loss function during training; A device for performing preliminary dedistortion prediction inference on the document image to be corrected, calculating the minimum circumscribed rotation rectangle, and performing axis-aligned rotation cropping to obtain an image area aligned with the document; A device for re-inputting the obtained aligned document image into the trained neural network model and outputting the corrected flattened document image as the final result.

9. A computer storage medium, wherein: The storage medium includes computer instructions, which, when executed on a computer, enable the computer to execute the method according to any one of claims 1 to 7.

10. An electronic device, wherein: The electronic device comprises: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

Images