Document image correction method based on deep learning

Abstract

The invention provides a document image correction method based on deep learning, and belongs to the field of computer vision, and the method comprises the steps: carrying out texture rendering and offset mapping processing based on an original texture grid image in a training set, and generating a spatial image to be rendered; performing data enhancement on the spatial to-be-rendered image through a flat image corresponding to the original texture grid image, and performing random disturbance based on a transformation space to obtain a corresponding sample training pair; based on the sample training pairs in the training set, training the initial detection correction model, optimizing control parameters, and obtaining a detection correction model; detecting a four-point detection model and a document correction model in the correction model, and processing the document image to be corrected in sequence to generate a corrected output image; according to the method, model training is carried out based on the self-constructed sample training pair, the document image correction effect of the model is effectively improved, meanwhile, the method can adapt to a complex real scene, and subsequent detection and recognition of the document image are facilitated.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a document image correction method based on deep learning. Background Art

[0002] With the popularity of smart devices, there are more and more scenarios where users transmit document images through smartphones, computers, etc. However, when capturing document images, physical deformation, shooting angles, and environmental interference often occur, causing the document images to be distorted.

[0003] In order to improve the detection and recognition accuracy of document images, they need to be corrected. There is little research on the correction of document images in traditional technologies. Usually, the English pdf document images in the dataset Doc3D are used to train the image correction model. However, since the distortion and deformation of the document images in this dataset tend to be the distortion and creases of a single page, there is a certain gap with the actual document image distortion. Therefore, the image correction effect of the model trained with this dataset is poor and it is difficult to adapt to the actual complex real scenes.

[0004] Therefore, a document image correction method based on deep learning is proposed. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a document image correction method based on deep learning, which is used to solve the problem that the document image correction effect of the model after data set training in traditional technology is poor and it is difficult to adapt to complex scenes.

[0006] An embodiment of the present invention provides a document image correction method based on deep learning, comprising:

[0007] Acquire a document image to be corrected;

[0008] Based on the original texture grid image in the training set, texture rendering and offset mapping processing are performed to generate a spatial image to be rendered; and data enhancement is performed on the spatial image to be rendered through the flattened image corresponding to the original texture grid image, and random perturbations are performed based on the transformation space to obtain corresponding sample training pairs;

[0009] By constructing an initial four-point detection model and an initial document correction model, an initial detection correction model is obtained;

[0010] Based on the sample training pairs in the training set, the initial detection and correction model is trained, the control parameters are optimized, and the detection and correction model is obtained;

[0011] Based on the four-point detection model in the detection and correction model, the corner point coordinates of the document image to be corrected are detected, and affine transformation is performed to obtain a detection and correction image; based on the document correction model in the detection and correction model, feature extraction and fusion are performed on the detection and correction image to generate a corrected output image.

[0012] Preferably, the present invention provides a document image correction method based on deep learning, the steps of: performing texture rendering and offset mapping processing based on the original texture grid image in the training set to generate a spatial image to be rendered; and performing data enhancement on the spatial image to be rendered through a flattened image corresponding to the original texture grid image, performing random perturbations based on the transformation space, and obtaining corresponding sample training pairs; comprising:

[0013] Obtaining an original texture grid image in the training set;

[0014] Perform cropping and sampling according to the original texture grid image to obtain a preprocessed texture grid image;

[0015] Based on the scan data of the preset scene, the camera shooting angle and distance are set, and the pre-processed texture grid image is rendered using a graphic image software to obtain a rendered texture image;

[0016] Performing two-dimensional plane mapping and backward mapping according to the rendered texture image to obtain a mapping processed image;

[0017] Perform four-point mapping and four-point offset mapping according to the mapped processed image to obtain a sample mapping image and a sample offset mapping image;

[0018] Performing a cosine simulated bending transformation on the original texture grid image to obtain a cosine grid image;

[0019] Processing the cosine grid image by a thin plate spline interpolation algorithm to obtain an interpolated grid image;

[0020] Generate a spatial image to be rendered according to the rendered texture image and the interpolated grid image;

[0021] Obtaining a flattened image corresponding to the original texture grid image in the training set;

[0022] Performing data enhancement on the spatial image to be rendered according to the flattened image, and performing random perturbation based on the transformed space to obtain a normal spatial rendering image, a spatial offset rendering image, and a spatial zoom rendering image;

[0023] According to the normal space rendering image, the space offset rendering image and the space zoom rendering image, four-point mapping and four-point offset mapping are performed respectively to obtain a normal space mapping image and a normal space four-point offset mapping image, a space offset mapping image and a space offset four-point offset mapping image, a space zoom mapping image and a space zoom four-point offset mapping image;

[0024] Optionally, the step also includes: constructing the normal space mapping image and the sample mapping image into a first sample training pair, constructing the spatial offset mapping image and the sample mapping image into a second sample training pair, constructing the spatial scaling mapping image and the sample mapping image into a third sample training pair, constructing the normal space four-point offset mapping image and the sample offset mapping image into a fourth sample training pair, constructing the spatial offset four-point offset mapping image and the sample offset mapping image into a fifth sample training pair, and constructing the spatial scaling four-point offset mapping image and the sample offset mapping image into a sixth sample training pair.

[0025] Preferably, the present invention provides a document image correction method based on deep learning, wherein the step of processing the cosine grid image by a thin plate spline interpolation algorithm to obtain an interpolated grid image comprises:

[0026] Acquire a pixel point set and a pre-generated interpolation point set in the cosine grid image;

[0027] Constructing an interpolation function according to the pixel point set and the interpolation point set;

[0028]

[0029] Among them, f(a,b) is the interpolation function, α0, α1, α2 are the trend coefficients to be solved, (a,b) is the pixel point in the pixel point set, μ j is the control point p in the radial basis function j The corresponding weight, U() is the radial basis function, ‖p i -(a,b)‖ is the Euclidean distance between the control point and the pixel point. Assume that the interpolation function has a total of c control points;

[0030] An interpolation transformation process is performed on a set of pixel points in the cosine grid image based on the interpolation function to obtain an interpolation grid image.

[0031] Preferably, the present invention provides a document image correction method based on deep learning, the steps of: performing data enhancement on the spatial image to be rendered according to the flattened image, and performing random perturbation based on the transformed space to obtain a normal spatial rendering image, a spatial offset rendering image, and a spatial scaled rendering image; comprising:

[0032] Acquire color features and texture features in the flattened image;

[0033] Based on the transformation space, the spatial image to be rendered is subjected to undisturbance, offset disturbance and scaling disturbance processing to obtain a normal spatial image, a spatial offset image and a spatial scaled image;

[0034] A mapping relationship between pixel points in the flattened image and pixel points in the normal space image, the spatial offset image, and the spatial zoomed image is constructed; the color features and texture features of the pixel points in the flattened image are projected onto the normal space image, the spatial offset image, and the spatial zoomed image, texture features are fused between the pixel points, and spatial color and texture rendering is performed to obtain a normal space rendered image, a spatial offset rendered image, and a spatial zoomed rendered image.

[0035] Preferably, the present invention provides a document image correction method based on deep learning, the steps of: training the initial detection and correction model based on sample training pairs in a training set, optimizing control parameters, and obtaining the detection and correction model; comprising:

[0036] Obtain an initial four-point detection model in an initial detection correction model;

[0037] Obtain the sample training pairs to train the initial four-point detection model, and optimize the initial four-point detection model by analyzing the degree of deviation between the sample corner point coordinates output by the initial four-point detection model and the sample standard corner point coordinates;

[0038]

[0039] Among them, L a is the coordinate shift loss function, (x i ,y i ) is the i-th coordinate information in the sample corner point coordinates, is the i-th coordinate information of the sample standard corner point coordinates, T r is the coordinate offset threshold.

[0040] Preferably, the present invention provides a document image correction method based on deep learning, the steps of: training the initial detection and correction model based on sample training pairs in a training set, optimizing control parameters, and obtaining the detection and correction model; comprising:

[0041] Obtaining an initial document correction model in the initial detection correction model;

[0042] The initial document correction model is trained according to the sample training pair, and the control parameters of the initial document correction model are optimized by calculating the joint loss function of the sample backward mapping coordinate image and the sample standard coordinate image output by the initial document correction model; optionally, including:

[0043] Acquire coordinate information of sample control points in the sample backward mapping coordinate image and coordinate information of sample standard control points in the sample standard coordinate image;

[0044] Calculating the control point absolute position loss function and the control point relative position loss function according to the sample control point coordinate information and the sample standard control point coordinate information;

[0045]

[0046] Among them, L1 is the absolute position loss function of the control point, l i is the deviation value of the control point coordinates, n is the number of control points, p i is the coordinate value of the i-th control point in the sample control point coordinate information, is the coordinate value of the i-th standard control point in the sample standard control point coordinate information;

[0047]

[0048] Among them, L r is the relative position loss function of the control point, r i is the coordinate value p of the i-th control point in the sample control point coordinate information i The coordinate value p of the jth control point around it j The relative position value of the control point is set to p in the sample control point coordinate information. i There are m control points around; is the coordinate value of the i-th control point in the sample standard control point coordinate information The coordinates of the jth control point around it The relative position value of

[0049] Obtaining sample reference point coordinate information in the sample backward mapping coordinate image and sample standard reference point coordinate information in the sample standard coordinate image, and calculating a reference point interval position loss function;

[0050]

[0051] Among them, L c is the reference point interval position loss function, when k = h, l k is the horizontal interval value of the reference point in the sample reference point coordinate information, is the horizontal interval value of the standard reference point in the sample standard reference point coordinate information; when k = v, l k is the vertical interval value of the reference point in the sample reference point coordinate information, is the vertical interval value of the standard reference point in the sample standard reference point coordinate information; ‖‖1 is the L1 norm;

[0052] Constructing a joint loss function according to the control point absolute position loss function, the control point relative position loss function and the reference point interval position loss function;

[0053] L=L1+α1L r +α2L c

[0054] Wherein, L is the joint loss function, α1 is the first adjustment weight parameter, and α2 is the second adjustment weight parameter;

[0055] The initial document correction model is supervised and trained according to the joint loss function to optimize the control parameters of the initial document correction model.

[0056] Preferably, the present invention provides a document image correction method based on deep learning, the steps of: based on a four-point detection model in a detection and correction model, detecting the corner coordinates of the document image to be corrected, performing affine transformation, and obtaining a detection and correction image; based on a document correction model in the detection and correction model, performing feature extraction and fusion on the detection and correction image to generate a correction output image; comprising:

[0057] Performing noise reduction processing on the document image to be corrected to generate a preprocessed corrected image, and performing grayscale transformation to obtain a grayscale document image to be corrected;

[0058] Evaluate the grayscale values ​​of all pixels in the grayscale document image to be corrected to obtain the coordinates of candidate corner points; optionally, include:

[0059] Randomly select a certain pixel point in the grayscale document image to be corrected, calculate the absolute value of the vertical grayscale difference between it and the adjacent pixel point in the vertical direction, and when the absolute value of the vertical grayscale difference is greater than a preset screening threshold, calculate the absolute value of the horizontal grayscale difference between it and the adjacent pixel point in the horizontal direction, and when the absolute value of the horizontal grayscale difference is greater than the preset screening threshold, use the pixel point as the candidate corner point coordinate;

[0060]

[0061] Where T is the preset screening threshold, h(i,j) is the gray value of the pixel (i,j) in the document image to be corrected, is the grayscale mean value of the pixels in the grayscale document image to be corrected, s is the number of rows of pixels in the grayscale document image to be corrected, and d is the number of columns of pixels in the grayscale document image to be corrected;

[0062] According to the candidate corner point coordinates, the corresponding corner point response function is calculated, and non-maximum suppression is performed to obtain the corner point coordinates of the document image to be corrected;

[0063] Constructing an image correction canvas, obtaining the coordinates of the corner points of the image correction canvas; constructing an affine transformation matrix according to the corner point coordinates and the canvas corner point coordinates;

[0064] Affine transformation is performed on the pre-processed corrected image based on the affine transformation matrix to obtain the detected corrected image.

[0065] Preferably, the present invention provides a document image correction method based on deep learning, the steps of: performing noise reduction processing on the document image to be corrected, generating a preprocessed corrected image, and performing grayscale transformation to obtain a grayscale document image to be corrected; comprising:

[0066] Acquiring the document image to be corrected;

[0067] Performing wavelet transform on the document image to be corrected based on the wavelet base to obtain the corresponding low-frequency component and high-frequency component of the image;

[0068] Based on an adaptive threshold algorithm, a horizontal high-frequency sub-component, a vertical high-frequency sub-component and a diagonal high-frequency sub-component in the high-frequency component of the image are subjected to noise reduction processing respectively, and the image high-frequency processing component;

[0069] Performing inverse wavelet transform on the low-frequency component of the image and the high-frequency processing component of the image to generate a noise reduction processed image;

[0070] Filtering the noise reduction processed image through a bilateral filter to obtain a preprocessed corrected image;

[0071]

[0072] Among them, p ′ (i, j) is the pixel value of the pixel point (i, j) in the pre-processed rectified image, p(i, j) is the pixel value of the pixel point (i, j) in the denoised image, θ p A pixel point set with a radius range of 2p+1 is constructed based on the pixel point (i, j) in the denoised image as the center and the filter radius p, p(u, v) is the pixel value of another pixel point (u, v) in the pixel point set, δ1 is the spatial distance difference parameter, and δ2 is the pixel difference parameter.

[0073] Preferably, the present invention provides a document image correction method based on deep learning, the steps of: based on a four-point detection model in a detection and correction model, detecting the corner coordinates of the document image to be corrected, performing affine transformation, and obtaining a detection and correction image; based on a document correction model in the detection and correction model, performing feature extraction and fusion on the detection and correction image to generate a correction output image; comprising:

[0074] Performing average pooling and maximum pooling operations on the detected and corrected image to obtain an average pooling feature vector and a maximum pooling feature vector; processing the average pooling feature vector and the maximum pooling feature vector through a fully connected neural network to obtain a first feature vector and a second feature vector; superimposing the first feature vector and the corresponding elements in the second feature vector, and calculating through a channel activation function to obtain a channel attention value; performing weighted calculation based on the detected and corrected image and the channel attention value to obtain a channel feature image;

[0075] Optionally, the step further includes: performing average pooling and maximum pooling operations according to the channel feature image to obtain an average pooling feature matrix and a maximum pooling feature matrix; splicing the average pooling feature matrix and the maximum pooling feature matrix based on the channel dimension, performing convolution dimensionality reduction processing through a convolutional neural network, and obtaining a spatial attention matrix; and performing normalization operation through a spatial activation function to obtain a spatial attention value; performing weighted calculation according to the channel feature image and the spatial attention value to obtain a feature fusion image;

[0076] Optionally, the step also includes: inputting the feature fusion image into the Transformer decoder for feature extraction, obtaining control point coordinate information and reference point coordinate information, constructing a backward mapping coordinate image based on the control point coordinate information and the reference point coordinate information based on an interpolation algorithm, processing the detected correction image, and generating the corrected output image.

[0077] Compared with the traditional technology, the beneficial effects of the present invention are: according to the flattened image and the original texture grid image in the training set, a distortion transformation processing is performed, the sample data in the training set is enriched, and a sample training pair is constructed to train the initial detection and correction model, thereby improving the correction processing performance of the document correction model for the distorted and deformed image, and the document image to be corrected is corrected through the detection and correction model to obtain a corrected output image; the document image correction method based on deep learning proposed in the present invention performs model training based on self-constructed sample training pairs, which solves the problem that the traditional technology relies on the Chinese and English pdf document images of the data set Doc3D for training and the training effect is poor, and effectively improves the document image correction effect of the document correction model, and also enables it to adapt to complex real scenes, which is convenient for the detection and recognition of document images in many subsequent scenes.

[0078] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0079] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 A schematic diagram of a process flow of a document image correction method based on deep learning provided by the present invention;

[0081] Figure 2 A diagram showing an embodiment of a document image correction method based on deep learning provided by the present invention;

[0082] Figure 3 A diagram showing another embodiment of a document image correction method based on deep learning provided by the present invention;

[0083] Figure 4 A diagram showing a four-point mapping processing embodiment of a document image correction method based on deep learning provided by the present invention;

[0084] Figure 5 An embodiment diagram of a normal space four-point offset mapping image of a document image correction method based on deep learning provided by the present invention;

[0085] Figure 6 This is a diagram of an implementation example of a four-point detection model transformation process for a document image correction method based on deep learning provided by the present invention. DETAILED DESCRIPTION

[0086] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0087] Embodiment 1:

[0088] The embodiment of the present invention provides a document image correction method based on deep learning, referring to Figure 1 ,include:

[0089] Acquire a document image to be corrected;

[0090] Based on the original texture grid image in the training set, texture rendering and offset mapping processing are performed to generate a spatial image to be rendered; and data enhancement is performed on the spatial image to be rendered through the flattened image corresponding to the original texture grid image, and random perturbations are performed based on the transformation space to obtain corresponding sample training pairs;

[0091] By constructing an initial four-point detection model and an initial document correction model, an initial detection correction model is obtained;

[0092] Based on the sample training pairs in the training set, the initial detection and correction model is trained, the control parameters are optimized, and the detection and correction model is obtained;

[0093] Based on the four-point detection model in the detection and correction model, the coordinates of the corner points of the document image to be corrected are detected, and affine transformation is performed to obtain the detection and correction image; based on the document correction model in the detection and correction model, feature extraction and fusion are performed on the detection and correction image to generate a corrected output image.

[0094] In the above embodiments, the four-point detection model detects the coordinates of the corner points of the document image to be corrected, performs affine transformation, and obtains a detection correction image; based on the document correction model in the detection correction model, feature extraction and fusion are performed on the detection correction image to generate a corrected output image.

[0095] In the above embodiments, texture rendering and offset mapping processing are performed based on the original texture grid image in the training set to generate a spatial image to be rendered; data enhancement is performed on the spatial image to be rendered by flattening the image, and random perturbations are performed based on the transformation space to obtain corresponding sample training pairs; according to the sample training pairs in the training set, the initial four-point detection model and the initial document correction model in the initial detection and correction model are trained, the control parameters are optimized, and the detection and correction model is obtained.

[0096] In one embodiment, reference Figure 2 and Figure 3 , the above method processes the document image to be corrected to obtain a corrected output image.

[0097] The beneficial effects of the above technology are: according to the flattened image and the original texture grid image in the training set, a distortion transformation processing is performed, the sample data in the training set is enriched, and a sample training pair is constructed to train the initial detection and correction model, thereby improving the correction processing performance of the document correction model for distorted and deformed images. The above method corrects the document image to be corrected through the detection and correction model to obtain a corrected output image; compared with the traditional technology, the document image correction method based on deep learning proposed in the present invention performs model training based on self-constructed sample training pairs, which solves the problem that the traditional technology relies on the Chinese and English pdf document images of the data set Doc3D for training and the training effect is poor, and effectively improves the document image correction effect of the document correction model, while also enabling it to adapt to complex real scenes, which is convenient for the detection and recognition of document images in many subsequent scenes.

[0098] Embodiment 2:

[0099] The embodiment of the present invention provides a document image correction method based on deep learning, comprising the steps of: performing texture rendering and offset mapping processing based on an original texture grid image in a training set to generate a spatial image to be rendered; performing data enhancement on the spatial image to be rendered through a flattened image corresponding to the original texture grid image, performing random perturbations based on a transformation space, and obtaining corresponding sample training pairs; comprising:

[0100] Get the original texture mesh image in the training set;

[0101] Perform cropping and sampling according to the original texture grid image to obtain a preprocessed texture grid image;

[0102] Based on the scan data of the preset scene, the camera shooting angle and distance are set, and the pre-processed texture grid image is rendered using the graphic image software to obtain the rendered texture image;

[0103] Perform two-dimensional plane mapping and back mapping according to the rendered texture image to obtain a mapping processed image;

[0104] Perform four-point mapping and four-point offset mapping according to the mapped image to obtain a sample mapping image and a sample offset mapping image;

[0105] Perform a cosine simulated bending transformation according to the original texture grid image to obtain a cosine grid image;

[0106] The cosine grid image is processed by a thin plate spline interpolation algorithm to obtain an interpolated grid image;

[0107] Generate a spatial image to be rendered according to the rendered texture image and the interpolated grid image;

[0108] Get the flattened image corresponding to the original texture grid image in the training set;

[0109] Performing data enhancement on the spatial image to be rendered according to the flattened image, and performing random perturbations based on the transformed space to obtain a normal spatial rendering image, a spatial offset rendering image, and a spatial zoom rendering image;

[0110] According to the normal space rendering image, the space offset rendering image and the space zoom rendering image, four-point mapping and four-point offset mapping are performed respectively to obtain a normal space mapping image and a normal space four-point offset mapping image, a space offset mapping image and a space offset four-point offset mapping image, a space zoom mapping image and a space zoom four-point offset mapping image;

[0111] Optionally, the steps also include: constructing a normal space mapping image and a sample mapping image into a first sample training pair, constructing a space offset mapping image and a sample mapping image into a second sample training pair, constructing a space scaling mapping image and a sample mapping image into a third sample training pair, constructing a normal space four-point offset mapping image and a sample offset mapping image into a fourth sample training pair, constructing a space offset four-point offset mapping image and a sample offset mapping image into a fifth sample training pair, and constructing a space scaling four-point offset mapping image and a sample offset mapping image into a sixth sample training pair.

[0112] In the above embodiments, cropping and sampling are performed based on the original texture grid image to obtain a preprocessed texture grid image; based on the scanning data of the preset scene, the camera shooting angle and distance are set, and the preprocessed texture grid image is rendered using graphics image software to obtain a rendered texture image; two-dimensional plane mapping and backward mapping are performed based on the rendered texture image to obtain a mapping processed image; four-point mapping and four-point offset mapping are performed on the mapping processed image to obtain a sample mapping image and a sample offset mapping image.

[0113] In the above embodiments, a cosine simulated bending transformation is performed according to the original texture grid image to obtain a cosine grid image, and the cosine grid image is processed by a thin plate spline interpolation algorithm (TPS interpolation) to obtain an interpolated grid image; a spatial image to be rendered is generated according to the rendered texture image and the interpolated grid image; data enhancement is performed on the spatial image to be rendered by flattening the image, and random perturbations are performed based on the transformation space to obtain a normal spatial rendering image, a spatial offset rendering image, and a spatial scaling rendering image; four-point mapping and four-point offset mapping are performed respectively to obtain a normal spatial mapping image and a normal spatial four-point offset mapping image, a spatial offset mapping image and a spatial offset four-point offset mapping image, a spatial scaling mapping image, and a spatial scaling four-point offset mapping image.

[0114] In the above embodiments, the normal space mapping image and the sample mapping image are constructed as the first sample training pair, the spatial offset mapping image and the sample mapping image are constructed as the second sample training pair, the spatial scaling mapping image and the sample mapping image are constructed as the third sample training pair, the normal space four-point offset mapping image and the sample offset mapping image are constructed as the fourth sample training pair, the spatial offset four-point offset mapping image and the sample offset mapping image are constructed as the fifth sample training pair, and the spatial scaling four-point offset mapping image and the sample offset mapping image are constructed as the sixth sample training pair.

[0115] In one embodiment, reference Figure 4 , four-point mapping is performed on the normal space rendering image, the space offset rendering image and the space scaling rendering image respectively, and the normal space mapping image, the space offset mapping image and the space scaling mapping image are obtained.

[0116] In one embodiment, reference Figure 5 , which is the normal space four-point offset mapping image.

[0117] The beneficial effects of the above technology are: through the flat images and original texture grid images in the training set, the images are distorted, and four-point mapping and four-point offset mapping are performed respectively to construct sample training pairs; compared with the fixed number of sample document images in the training set of traditional technology, the above technical scheme transforms the images in the training set to construct sample training pairs, which enriches the training samples and effectively improves the training effect of the model.

[0118] Embodiment 3:

[0119] The embodiment of the present invention provides a document image correction method based on deep learning, the steps of: processing a cosine grid image by a thin plate spline interpolation algorithm to obtain an interpolated grid image, including:

[0120] Obtain a pixel point set and a pre-generated interpolation point set in the cosine grid image;

[0121] Construct an interpolation function based on a pixel point set and an interpolation point set;

[0122]

[0123] Among them, f(a,b) is the interpolation function, α0, α1, α2 are the trend coefficients to be solved, (a,b) is the pixel point in the pixel point set, μ j is the control point p in the radial basis function j The corresponding weight, U() is the radial basis function, ‖p i -(a,b)‖ is the Euclidean distance between the control point and the pixel point. Assume that the interpolation function has a total of c control points;

[0124] An interpolation transformation process is performed on a set of pixel points in the cosine grid image based on an interpolation function to obtain an interpolated grid image.

[0125] In the above embodiment, an interpolation function is constructed by selecting a pixel point set in a cosine grid image and a pre-generated interpolation point set. The interpolation function performs interpolation transformation processing according to the pixel point set in the cosine grid image to obtain an interpolated grid image.

[0126] In the above embodiments, the thin plate spline interpolation algorithm can be implemented as a (Thin Plate Spline) algorithm.

[0127] Embodiment 4:

[0128] The embodiment of the present invention provides a document image correction method based on deep learning, comprising the steps of: performing data enhancement on a spatial image to be rendered according to a flattened image, and performing random perturbation based on a transformed space to obtain a normal spatial rendering image, a spatial offset rendering image, and a spatial zoom rendering image; comprising:

[0129] Obtain color features and texture features in a flattened image;

[0130] Based on the transformation space, the spatial image to be rendered is subjected to undisturbance, offset disturbance and scaling disturbance processing to obtain a normal spatial image, a spatial offset image and a spatial scaling image;

[0131] Construct a mapping relationship between pixel points in the flattened image and pixel points in the normal spatial image, spatial offset image, and spatial scaled image; project the color features and texture features of the pixel points in the flattened image to the normal spatial image, spatial offset image, and spatial scaled image, fuse texture features between pixels, and perform spatial color and texture rendering to obtain the normal spatial rendered image, spatial offset rendered image, and spatial scaled rendered image.

[0132] In the above embodiments, the spatial image to be rendered is subjected to undisturbance, offset disturbance and scaled disturbance processing to obtain corresponding normal spatial images, spatial offset images and spatial scaled images. According to the mapping relationship between the pixels in the flattened image and the pixels in the normal spatial image, the spatial offset image and the spatial scaled image, the color features and texture features of the pixels in the flattened image are projected into the normal spatial image, the spatial offset image and the spatial scaled image, and texture feature fusion and color rendering are performed to generate the normal spatial rendered image, the spatial offset rendered image and the spatial scaled rendered image.

[0133] The beneficial effects of the above technology are: by perturbing the spatial image to be rendered, the image distortion transformation based on the transformation space is realized, and by flattening the image, data enhancement is performed on the distorted normal spatial image, spatial offset image and spatial scaled image to generate normal spatial rendering image, spatial offset rendering image and spatial scaled rendering image, which is convenient for the subsequent steps to construct sample training pairs in the training set.

[0134] Embodiment 5:

[0135] The embodiment of the present invention provides a document image correction method based on deep learning, the steps of: training an initial detection and correction model based on sample training pairs in a training set, optimizing control parameters, and obtaining a detection and correction model; including:

[0136] Obtain an initial four-point detection model in an initial detection correction model;

[0137] Obtain sample training pairs to train the initial four-point detection model, and optimize the initial four-point detection model by analyzing the degree of deviation between the sample corner point coordinates output by the initial four-point detection model and the sample standard corner point coordinates;

[0138]

[0139] Among them, L a is the coordinate shift loss function, (x i ,y i ) is the i-th coordinate information in the sample corner point coordinates, is the i-th coordinate information of the sample standard corner point coordinates, T r is the coordinate offset threshold.

[0140] In the above embodiments, the initial four-point detection model is trained based on the sample training pairs in the training set, and the initial four-point detection model is optimized by analyzing the degree of deviation between the sample corner point coordinates output by the initial four-point detection model and the sample standard corner point coordinates.

[0141] In the above embodiments, a coordinate offset loss function is constructed to evaluate the offset between the sample corner point coordinates and the sample standard corner point coordinates, and the value of the coordinate offset loss function is reduced by adjusting the control parameters of the initial four-point detection model.

[0142] The beneficial effects of the above technology are: the initial four-point detection model is trained through sample training, the corner point detection performance of the initial four-point detection model is evaluated according to the coordinate offset loss function, and the control parameters of the model are further optimized to facilitate the four-point detection model to perform corner point detection and image correction processing on the document image to be corrected.

[0143] Embodiment 6:

[0144] The embodiment of the present invention provides a document image correction method based on deep learning, the steps of: training an initial detection and correction model based on sample training pairs in a training set, optimizing control parameters, and obtaining a detection and correction model; including:

[0145] Obtaining an initial document correction model in an initial detection correction model;

[0146] The initial document correction model is trained according to the sample training pairs, and the control parameters of the initial document correction model are optimized by calculating the joint loss function of the sample backward mapping coordinate image and the sample standard coordinate image output by the initial document correction model; optional, including:

[0147] Obtaining coordinate information of sample control points in the sample backward mapping coordinate image and coordinate information of sample standard control points in the sample standard coordinate image;

[0148] According to the coordinate information of the sample control points and the coordinate information of the sample standard control points, the absolute position loss function of the control points and the relative position loss function of the control points are calculated;

[0149]

[0150] Among them, L1 is the absolute position loss function of the control point, l i is the deviation value of the control point coordinates, n is the number of control points, p i is the coordinate value of the i-th control point in the sample control point coordinate information, is the coordinate value of the i-th standard control point in the sample standard control point coordinate information;

[0151]

[0152]

[0153] Among them, L r is the relative position loss function of the control point, r i is the coordinate value p of the i-th control point in the sample control point coordinate information i The coordinate value p of the jth control point around it j The relative position value of the control point is set to p in the sample control point coordinate information. i There are m control points around; is the coordinate value of the i-th control point in the sample standard control point coordinate information The coordinates of the jth control point around it The relative position value of

[0154] Obtaining the sample reference point coordinate information in the sample backward mapping coordinate image and the sample standard reference point coordinate information in the sample standard coordinate image, and calculating the reference point interval position loss function;

[0155]

[0156] Among them, L c is the reference point interval position loss function, when k = h, l k is the horizontal interval value of the reference point in the sample reference point coordinate information, is the horizontal interval value of the standard reference point in the sample standard reference point coordinate information; when k = v, l k is the vertical interval value of the reference point in the sample reference point coordinate information, is the vertical interval value of the standard reference point in the sample standard reference point coordinate information; ‖‖1 is the L1 norm;

[0157] Construct a joint loss function based on the absolute position loss function of the control points, the relative position loss function of the control points and the reference point interval position loss function;

[0158] L=L1+α1L r +α2L c

[0159] Wherein, L is the joint loss function, α1 is the first adjustment weight parameter, and α2 is the second adjustment weight parameter;

[0160] The initial document correction model is supervised and trained according to the joint loss function to optimize the control parameters of the initial document correction model.

[0161] In the above embodiments, the initial document correction model is trained by using sample training pairs in the training set, the joint loss function of the sample backward mapping coordinate image and the sample standard coordinate image output by the initial document correction model is calculated, and the control parameters of the initial document correction model are optimized.

[0162] In the above embodiments, the joint loss function is constructed according to the control point absolute position loss function, the control point relative position loss function and the reference point interval position loss function.

[0163] In the above embodiments, the control point absolute position loss function is calculated based on the sample control point coordinate information in the sample backward mapping coordinate image and the sample standard control point coordinate information in the sample standard coordinate image, and is specifically implemented using the SmoothL1 loss function.

[0164] In the above embodiments, the relative position loss function of the control points is calculated based on the sample control point coordinate information and the sample standard control point coordinate information, and is specifically implemented by using a differential coordinate loss function.

[0165] In the above embodiments, the reference point interval position loss function is calculated based on the sample reference point coordinate information in the sample backward mapping coordinate image and the sample standard reference point coordinate information in the sample standard coordinate image, specifically including the reference point intervals in the horizontal and vertical directions.

[0166] The beneficial effects of the above technology are: the initial document correction model is trained according to sample training, the joint loss function is calculated, and the control parameters of the initial document correction model are optimized. Specifically, the document correction model is obtained by optimizing the control point absolute position loss function, the control point relative position loss function and the reference point interval position loss function, thereby improving the accuracy of the model output control point coordinate information and reference point coordinate information.

[0167] Embodiment 7:

[0168] The embodiment of the present invention provides a document image correction method based on deep learning, comprising the steps of: based on a four-point detection model in a detection and correction model, detecting the coordinates of the corner points of a document image to be corrected, performing affine transformation, and obtaining a detection and correction image; based on a document correction model in the detection and correction model, performing feature extraction and fusion on the detection and correction image to generate a correction output image; comprising:

[0169] Performing noise reduction processing on the document image to be corrected to generate a pre-processed corrected image, and performing grayscale transformation to obtain a grayscale document image to be corrected;

[0170] Evaluate the grayscale values ​​of all pixels in the grayscale document image to be corrected and obtain the coordinates of candidate corner points; optional, including:

[0171] Randomly select a pixel point in the grayscale document image to be corrected, calculate the absolute value of the vertical grayscale difference between it and the adjacent pixel point in the vertical direction, and when the absolute value of the vertical grayscale difference is greater than the preset screening threshold, calculate the absolute value of the horizontal grayscale difference between it and the adjacent pixel point in the horizontal direction, and when the absolute value of the horizontal grayscale difference is greater than the preset screening threshold, take the pixel point as the candidate corner point coordinate;

[0172]

[0173] Where T is the preset screening threshold, h(i,j) is the gray value of the pixel (i,j) in the document image to be corrected, is the grayscale mean value of the pixels in the grayscale document image to be corrected, s is the number of rows of pixels in the grayscale document image to be corrected, and d is the number of columns of pixels in the grayscale document image to be corrected;

[0174] According to the candidate corner point coordinates, the corresponding corner point response function is calculated, and non-maximum suppression is performed to obtain the corner point coordinates of the document image to be corrected;

[0175] Construct an image correction canvas and obtain the coordinates of the corner points of the image correction canvas; construct an affine transformation matrix based on the corner point coordinates and the canvas corner point coordinates;

[0176] Affine transformation is performed on the preprocessed corrected image based on the affine transformation matrix to obtain a detected corrected image.

[0177] In the above embodiment, the four-point detection model performs noise reduction processing on the document image to be corrected, generates a preprocessed corrected image, and performs grayscale transformation to obtain a grayscale document image to be corrected; evaluates the grayscale values ​​of all pixels in the grayscale document image to be corrected, obtains the coordinates of the candidate corner points, calculates the corresponding corner point response function, and performs non-maximum suppression to obtain the corner point coordinates of the document image to be corrected.

[0178] In the above embodiment, the absolute value of the grayscale difference between the pixel point in the document image to be corrected and the adjacent pixel point is calculated by using the random grayscale. When the absolute value of the vertical grayscale difference and the absolute value of the horizontal grayscale difference are both greater than the preset screening threshold, the pixel point is used as the candidate corner point coordinate.

[0179] In the above embodiment, when the absolute value of the vertical grayscale difference between a pixel point in the random grayscale document image to be corrected and its adjacent pixel point is less than a preset screening threshold, the absolute value of the horizontal grayscale difference of the pixel point is no longer judged, that is, it is not determined as a candidate corner point coordinate.

[0180] In the above embodiments, an affine transformation matrix is ​​constructed according to the corner point coordinates and the canvas corner point coordinates of the image correction canvas, and an affine transformation process is performed on the pre-processed correction image to obtain a detection correction image.

[0181] In one embodiment, reference Figure 6 ,The four-point detection model detects the corner coordinates of the document image to be rectified, and performs affine transformation processing to generate the detected ,rectified image.

[0182] The beneficial effects of the above technology are: the corner point coordinates of the document image to be corrected are detected through the four-point detection model, and an affine transformation matrix is ​​constructed according to the canvas corner point coordinates of the image correction canvas, thereby realizing the affine transformation processing of the pre-processed corrected image according to the corner point coordinates; wherein the four-point detection model calculates the absolute value of the grayscale difference of the pixel points according to the grayscale document image to be corrected, thereby realizing the rapid acquisition of the candidate corner point coordinates, and calculating the corner point response function for non-maximum suppression, thereby realizing the screening of the corner point coordinates, effectively improving the accuracy and detection speed of the selected corner point coordinates.

[0183] Embodiment 8:

[0184] The embodiment of the present invention provides a document image correction method based on deep learning, comprising the steps of: performing noise reduction processing on a document image to be corrected, generating a preprocessed correction image, and performing grayscale transformation to obtain a grayscale document image to be corrected; comprising:

[0185] Acquire a document image to be corrected;

[0186] Perform wavelet transform on the document image to be corrected based on the wavelet base to obtain the corresponding low-frequency component and high-frequency component of the image;

[0187] Based on the adaptive threshold algorithm, the horizontal high-frequency sub-component, the vertical high-frequency sub-component and the diagonal high-frequency sub-component in the high-frequency component of the image are subjected to noise reduction processing respectively, and the image high-frequency processing component;

[0188] Perform inverse wavelet transform according to the low-frequency component and high-frequency processing component of the image to generate a noise reduction processed image;

[0189] The denoised image is filtered by a bilateral filter to obtain a pre-processed corrected image;

[0190]

[0191] Among them, p ′ (i, j) is the pixel value of pixel (i, j) in the pre-processed rectified image, p(i, j) is the pixel value of pixel (i, j) in the denoised image, θ p A pixel point set with a radius range of 2p+1 is constructed based on the pixel point (i, j) in the denoised image as the center and the filter radius p. p(u, v) is the pixel value of another pixel point (u, v) in the pixel point set. δ1 is the spatial distance difference parameter, and δ2 is the pixel difference parameter.

[0192] In the above embodiments, the document image to be corrected is subjected to wavelet transform based on the wavelet base to obtain the low-frequency component and the high-frequency component of the image, and the high-frequency component of the image is subjected to denoising to obtain the high-frequency processed component of the image; the low-frequency component and the high-frequency processed component of the image are subjected to inverse wavelet transform to generate a denoised processed image, and the denoised processed image is filtered through a bilateral filter to obtain a preprocessed corrected image.

[0193] In the above embodiments, the horizontal high-frequency sub-component, the vertical high-frequency sub-component and the diagonal high-frequency sub-component in the high-frequency component of the image are subjected to noise reduction processing based on the adaptive threshold algorithm to obtain the high-frequency processing component of the image.

[0194] The beneficial effects of the above technology are: the decomposition of low-frequency components and high-frequency components of the document image to be corrected is achieved through wavelet transform, the denoising of the high-frequency components of the image is achieved based on the adaptive threshold algorithm, and the denoised image is filtered through a bilateral filter after the inverse wavelet transform, thereby achieving denoising of the low-frequency components in the denoised image, and further achieving denoising of the document image to be corrected, which is convenient for the detection of corner coordinates in subsequent steps.

[0195] Embodiment 9:

[0196] The embodiment of the present invention provides a document image correction method based on deep learning, comprising the steps of: based on a four-point detection model in a detection and correction model, detecting the coordinates of the corner points of a document image to be corrected, performing affine transformation, and obtaining a detection and correction image; based on a document correction model in the detection and correction model, performing feature extraction and fusion on the detection and correction image to generate a correction output image; comprising:

[0197] Perform average pooling and maximum pooling operations according to the detected and corrected image to obtain an average pooling feature vector and a maximum pooling feature vector; process the average pooling feature vector and the maximum pooling feature vector through a fully connected neural network to obtain a first feature vector and a second feature vector; superimpose the first feature vector and the corresponding elements in the second feature vector, and calculate through a channel activation function to obtain a channel attention value; perform weighted calculation according to the detected and corrected image and the channel attention value to obtain a channel feature image;

[0198] Optionally, the steps further include: performing average pooling and maximum pooling operations according to the channel feature image to obtain an average pooling feature matrix and a maximum pooling feature matrix; splicing the average pooling feature matrix and the maximum pooling feature matrix based on the channel dimension, performing convolution dimensionality reduction processing through a convolutional neural network, and obtaining a spatial attention matrix; and performing normalization operation through a spatial activation function to obtain a spatial attention value; performing weighted calculation according to the channel feature image and the spatial attention value to obtain a feature fusion image;

[0199] Optionally, the steps also include: inputting the feature fusion image into the Transformer decoder for feature extraction, obtaining control point coordinate information and reference point coordinate information, constructing a backward mapping coordinate image based on the control point coordinate information and the reference point coordinate information based on the interpolation algorithm, processing the detected correction image, and generating a corrected output image.

[0200] In the above embodiments, channel attention and spatial attention are adjusted on the detection and correction image to obtain a feature fusion image, the Transformer decoder performs feature extraction, a backward mapping coordinate image is constructed, the detection and correction image is processed, and a corrected output image is generated.

[0201] In the above embodiments, the channel attention and spatial attention adjustment of the detection and correction image can be achieved through the CBAM module.

[0202] In the above embodiments, for the adjustment of the channel attention of the detection and correction image, the average pooling feature vector and the maximum pooling feature vector of the correction image are extracted, and the average pooling feature vector and the maximum pooling feature vector are processed by a fully connected neural network to obtain a first feature vector and a second feature vector; and the first feature vector and the corresponding elements in the second feature vector are superimposed and calculated through a channel activation function to obtain a channel attention value; and a weighted calculation is performed based on the detection and correction image and the channel attention value to obtain a channel feature image.

[0203] In the above embodiments, for the adjustment of the spatial attention of the detection and correction image, the average pooling feature matrix and the maximum pooling feature matrix are obtained by performing average pooling and maximum pooling operations on the channel feature image; the average pooling feature matrix and the maximum pooling feature matrix are spliced ​​based on the channel dimension, and convolution dimensionality reduction processing is performed through a convolutional neural network to obtain a spatial attention matrix; and normalization operation is performed through a spatial activation function to obtain a spatial attention value; the channel feature image and the spatial attention value are weightedly calculated to obtain a feature fusion image.

[0204] In the above embodiments, the Transformer decoder extracts features based on the feature fusion image, generates control point coordinate information and reference point coordinate information, and performs interpolation operations to construct a backward mapping coordinate image; and performs correction processing on the detection correction image based on the backward mapping coordinate image.

[0205] The beneficial effects of the above technology are: by adjusting the channel spatial attention of the detection and correction image in the document correction model and performing feature extraction based on the Transformer decoder, the feature extraction performance of the model is effectively improved, and interpolation operations are performed according to the control point coordinate information and the reference point coordinate information, thereby realizing the construction of the backward mapping coordinate image, which facilitates the correction processing of the detection and correction image and obtains the corrected output image.

[0206] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A document image correction method based on deep learning, characterized in that: include: Acquire the document image to be corrected; Based on the original texture grid image in the training set, texture rendering and offset mapping processing are performed to generate a spatial image to be rendered; and data enhancement is performed on the spatial image to be rendered through the flattened image corresponding to the original texture grid image, and random perturbations are performed based on the transformation space to obtain corresponding sample training pairs; By constructing an initial four-point detection model and an initial document correction model, an initial detection correction model is obtained; Based on the sample training pairs in the training set, the initial detection and correction model is trained, the control parameters are optimized, and the detection and correction model is obtained; Based on the four-point detection model in the detection and correction model, the coordinates of the corner points of the document image to be corrected are detected, and an affine transformation is performed to obtain a detection and correction image; Based on the document correction model in the detection and correction model, feature extraction and fusion are performed on the detection and correction image to generate a corrected output image.

2. A document image correction method based on deep learning according to claim 1, characterized in that: The steps include: performing texture rendering and offset mapping processing based on the original texture grid image in the training set to generate a spatial image to be rendered; performing data enhancement on the spatial image to be rendered through the flattened image corresponding to the original texture grid image, performing random perturbations based on the transformation space, and obtaining corresponding sample training pairs; including: Obtaining an original texture grid image in the training set; Perform cropping and sampling according to the original texture grid image to obtain a preprocessed texture grid image; Based on the scan data of the preset scene, the camera shooting angle and distance are set, and the pre-processed texture grid image is rendered using a graphic image software to obtain a rendered texture image; Performing two-dimensional plane mapping and backward mapping according to the rendered texture image to obtain a mapping processed image; Perform four-point mapping and four-point offset mapping according to the mapped processed image to obtain a sample mapping image and a sample offset mapping image; Performing a cosine simulated bending transformation on the original texture grid image to obtain a cosine grid image; Processing the cosine grid image by a thin plate spline interpolation algorithm to obtain an interpolated grid image; Generate a spatial image to be rendered according to the rendered texture image and the interpolated grid image; Obtaining a flattened image corresponding to the original texture grid image in the training set; Performing data enhancement on the spatial image to be rendered according to the flattened image, and performing random perturbation based on the transformed space to obtain a normal spatial rendering image, a spatial offset rendering image, and a spatial zoom rendering image; According to the normal space rendering image, the space offset rendering image and the space zoom rendering image, four-point mapping and four-point offset mapping are performed respectively to obtain a normal space mapping image and a normal space four-point offset mapping image, a space offset mapping image and a space offset four-point offset mapping image, a space zoom mapping image and a space zoom four-point offset mapping image; Optionally, the step also includes: constructing the normal space mapping image and the sample mapping image into a first sample training pair, constructing the spatial offset mapping image and the sample mapping image into a second sample training pair, constructing the spatial scaling mapping image and the sample mapping image into a third sample training pair, constructing the normal space four-point offset mapping image and the sample offset mapping image into a fourth sample training pair, constructing the spatial offset four-point offset mapping image and the sample offset mapping image into a fifth sample training pair, and constructing the spatial scaling four-point offset mapping image and the sample offset mapping image into a sixth sample training pair.

3. The document image correction method based on deep learning according to claim 2, characterized in that: The step of processing the cosine grid image by a thin plate spline interpolation algorithm to obtain an interpolation grid image comprises: Acquire a pixel point set and a pre-generated interpolation point set in the cosine grid image; Constructing an interpolation function according to the pixel point set and the interpolation point set; Among them, f(a,b) is the interpolation function, α0, α1, α2 are the trend coefficients to be solved, (a,b) is the pixel point in the pixel point set, μ j is the control point p in the radial basis function j The corresponding weight, U() is the radial basis function, ‖p i -(a,b)‖ is the Euclidean distance between the control point and the pixel point. Assume that the interpolation function has a total of c control points; An interpolation transformation process is performed on a set of pixel points in the cosine grid image based on the interpolation function to obtain an interpolation grid image.

4. The document image correction method based on deep learning according to claim 2, characterized in that: The step of performing data enhancement on the spatial image to be rendered according to the flattened image, and performing random perturbation based on the transformed space to obtain a normal spatial rendering image, a spatial offset rendering image, and a spatial zoom rendering image comprises: Acquire color features and texture features in the flattened image; Based on the transformation space, the spatial image to be rendered is subjected to undisturbance, offset disturbance and scaling disturbance processing to obtain a normal spatial image, a spatial offset image and a spatial scaled image; A mapping relationship between pixel points in the flattened image and pixel points in the normal space image, the spatial offset image, and the spatial zoomed image is constructed; the color features and texture features of the pixel points in the flattened image are projected onto the normal space image, the spatial offset image, and the spatial zoomed image, texture features are fused between the pixel points, and spatial color and texture rendering is performed to obtain a normal space rendered image, a spatial offset rendered image, and a spatial zoomed rendered image.

5. The document image correction method based on deep learning according to claim 1, characterized in that: The step of training the initial detection and correction model based on the sample training pairs in the training set, optimizing the control parameters, and obtaining the detection and correction model includes: Obtain an initial four-point detection model in an initial detection correction model; Obtain the sample training pairs to train the initial four-point detection model, and optimize the initial four-point detection model by analyzing the degree of deviation between the sample corner point coordinates output by the initial four-point detection model and the sample standard corner point coordinates; Among them, L a is the coordinate shift loss function, (x i ,y i ) is the i-th coordinate information in the sample corner point coordinates, is the i-th coordinate information of the sample standard corner point coordinates, T r is the coordinate offset threshold.

6. The document image correction method based on deep learning according to claim 1, characterized in that: The step: based on the sample training pairs in the training set, the initial detection and correction model is trained, the control parameters are optimized, and the detection and correction model is obtained; include: Obtaining an initial document correction model in the initial detection correction model; The initial document correction model is trained according to the sample training pair, and the control parameters of the initial document correction model are optimized by calculating the joint loss function of the sample backward mapping coordinate image and the sample standard coordinate image output by the initial document correction model; optionally, including: Acquire coordinate information of sample control points in the sample backward mapping coordinate image and coordinate information of sample standard control points in the sample standard coordinate image; Calculating the control point absolute position loss function and the control point relative position loss function according to the sample control point coordinate information and the sample standard control point coordinate information; Among them, L1 is the absolute position loss function of the control point, l i is the deviation value of the control point coordinates, n is the number of control points, p i is the coordinate value of the i-th control point in the sample control point coordinate information, is the coordinate value of the i-th standard control point in the sample standard control point coordinate information; Among them, L r is the relative position loss function of the control point, r i is the coordinate value p of the i-th control point in the sample control point coordinate information i The coordinate value p of the jth control point around it j The relative position value of the control point is set to p in the sample control point coordinate information. i There are m control points around; is the coordinate value of the i-th control point in the sample standard control point coordinate information The coordinates of the jth control point around it The relative position value of Obtaining sample reference point coordinate information in the sample backward mapping coordinate image and sample standard reference point coordinate information in the sample standard coordinate image, and calculating a reference point interval position loss function; Among them, L c is the reference point interval position loss function, when k = h, l k is the horizontal interval value of the reference point in the sample reference point coordinate information, is the horizontal interval value of the standard reference point in the sample standard reference point coordinate information; when k = v, l k is the vertical interval value of the reference point in the sample reference point coordinate information, is the vertical interval value of the standard reference point in the sample standard reference point coordinate information; ‖‖1 is the L1 norm; Constructing a joint loss function according to the control point absolute position loss function, the control point relative position loss function and the reference point interval position loss function; L=L1+α1L r +α2L c Wherein, L is the joint loss function, α1 is the first adjustment weight parameter, and α2 is the second adjustment weight parameter; The initial document correction model is supervised and trained according to the joint loss function to optimize the control parameters of the initial document correction model.

7. The document image correction method based on deep learning according to claim 1, characterized in that: The steps include: detecting the corner coordinates of the document image to be corrected based on the four-point detection model in the detection and correction model, performing affine transformation, and obtaining a detection and correction image; Based on the document correction model in the detection and correction model, feature extraction and fusion are performed on the detection and correction image to generate a correction output image; including: Performing noise reduction processing on the document image to be corrected to generate a preprocessed corrected image, and performing grayscale transformation to obtain a grayscale document image to be corrected; Evaluate the grayscale values ​​of all pixels in the grayscale document image to be corrected to obtain the coordinates of candidate corner points; optionally, include: Randomly select a certain pixel point in the grayscale document image to be corrected, calculate the absolute value of the vertical grayscale difference between it and the adjacent pixel point in the vertical direction, and when the absolute value of the vertical grayscale difference is greater than a preset screening threshold, calculate the absolute value of the horizontal grayscale difference between it and the adjacent pixel point in the horizontal direction, and when the absolute value of the horizontal grayscale difference is greater than the preset screening threshold, use the pixel point as the candidate corner point coordinate; Where T is the preset screening threshold, h(i,j) is the gray value of the pixel (i,j) in the document image to be corrected, is the grayscale mean value of the pixels in the grayscale document image to be corrected, s is the number of rows of pixels in the grayscale document image to be corrected, and d is the number of columns of pixels in the grayscale document image to be corrected; According to the candidate corner point coordinates, the corresponding corner point response function is calculated, and non-maximum suppression is performed to obtain the corner point coordinates of the document image to be corrected; Constructing an image correction canvas, obtaining the coordinates of the corner points of the image correction canvas; constructing an affine transformation matrix according to the corner point coordinates and the canvas corner point coordinates; Affine transformation is performed on the pre-processed corrected image based on the affine transformation matrix to obtain the detected corrected image.

8. The document image correction method based on deep learning according to claim 6, characterized in that: The step of performing noise reduction processing on the document image to be corrected to generate a pre-processed corrected image, and performing grayscale transformation to obtain a grayscale document image to be corrected comprises: Acquire the document image to be corrected; Performing wavelet transform on the document image to be corrected based on the wavelet base to obtain the corresponding low-frequency component and high-frequency component of the image; Based on an adaptive threshold algorithm, a horizontal high-frequency sub-component, a vertical high-frequency sub-component and a diagonal high-frequency sub-component in the high-frequency component of the image are subjected to noise reduction processing respectively, and the image high-frequency processing component; Performing inverse wavelet transform on the low-frequency component of the image and the high-frequency processing component of the image to generate a noise reduction processed image; Filtering the noise reduction processed image through a bilateral filter to obtain a preprocessed corrected image; Among them, p ′ (i, j) is the pixel value of the pixel point (i, j) in the pre-processed rectified image, p(i, j) is the pixel value of the pixel point (i, j) in the denoised image, θ p A pixel point set with a radius range of 2p+1 is constructed based on the pixel point (i, j) in the denoised image as the center and the filter radius p, p(u, v) is the pixel value of another pixel point (u, v) in the pixel point set, δ1 is the spatial distance difference parameter, and δ2 is the pixel difference parameter.

9. The document image correction method based on deep learning according to claim 1, characterized in that: The steps include: detecting the corner coordinates of the document image to be corrected based on the four-point detection model in the detection and correction model, performing affine transformation, and obtaining a detection and correction image; Based on the document correction model in the detection and correction model, feature extraction and fusion are performed on the detection and correction image to generate a correction output image; including: Performing average pooling and maximum pooling operations on the detected and corrected image to obtain an average pooling feature vector and a maximum pooling feature vector; processing the average pooling feature vector and the maximum pooling feature vector through a fully connected neural network to obtain a first feature vector and a second feature vector; superimposing the first feature vector and the corresponding elements in the second feature vector, and calculating through a channel activation function to obtain a channel attention value; performing weighted calculation based on the detected and corrected image and the channel attention value to obtain a channel feature image; Optionally, the step further includes: performing average pooling and maximum pooling operations according to the channel feature image to obtain an average pooling feature matrix and a maximum pooling feature matrix; splicing the average pooling feature matrix and the maximum pooling feature matrix based on the channel dimension, performing convolution dimensionality reduction processing through a convolutional neural network, and obtaining a spatial attention matrix; and performing normalization operation through a spatial activation function to obtain a spatial attention value; performing weighted calculation according to the channel feature image and the spatial attention value to obtain a feature fusion image; Optionally, the step also includes: inputting the feature fusion image into the Transformer decoder for feature extraction, obtaining control point coordinate information and reference point coordinate information, constructing a backward mapping coordinate image based on the control point coordinate information and the reference point coordinate information based on an interpolation algorithm, processing the detected correction image, and generating the corrected output image.

Images