Document image correction method, related device and computer program product

Abstract

The invention discloses a document image correction method, related equipment and a computer program product, and provides a solution of combining text line straight line detection with a majority voting mechanism aiming at the defects that an existing document image correction technology depends on a document physical boundary and the like. Straight line detection is carried out on a text line edge image obtained through edge detection to obtain a straight line set representing the text line direction, a majority voting mechanism is adopted to aggregate straight line angle information, the direction represented by a target quantization angle interval with the highest voting number is determined to serve as the main direction of a document image, document physical boundaries do not need to be relied on, and the text line direction is obtained. And local interference in the document image can be effectively overcome. The document image is subjected to geometric correction based on the main direction of the document image, so that the stability and reliability of a correction result can be remarkably improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically, to a document image correction method, related equipment, and computer program product. Background Technology

[0002] For document images captured by a camera, the images may be skewed or even curved due to factors such as the shooting angle. Therefore, image correction is needed to "straighten" the text in the image.

[0003] Traditional document correction methods primarily rely on the detection and location of the document's physical boundaries, achieving correction by detecting the corner points of the physical boundaries and performing perspective transformations. However, they perform poorly in application scenarios without obvious boundaries (such as a sticky note, a page of handwritten notes, or the middle section of a book). Summary of the Invention

[0004] In view of the above problems, this application is made to provide a document image correction method, related equipment, and computer program product to improve the document image correction effect. The specific solution is as follows:

[0005] Firstly, a document image correction method is provided, including:

[0006] Obtain the document image to be processed, perform edge detection on the document image, and obtain the text line edge image;

[0007] Line detection is performed on the text line edge image to obtain a set of lines representing the text lines;

[0008] Based on the angles of each line in the set of lines, a vote is taken on each set of quantized angle intervals, and the target quantized angle interval with the highest number of votes is determined. The direction represented by the target quantized angle interval is taken as the main direction of the document image.

[0009] Geometric correction is performed on the document image based on its main orientation to obtain a corrected image.

[0010] In one possible design, in another implementation of the first aspect of the embodiments of this application, the process of voting on each set of quantized angle intervals based on the angles of each line in the set of lines includes:

[0011] Based on the length and / or goodness of fit of each line in the set of lines, the voting weight of the line is determined. The voting weight is positively correlated with the length and goodness of fit of the line. The goodness of fit represents the degree of match between the detected line and the actual edge point in the text line edge image.

[0012] Voting is conducted on each set of quantized angle intervals based on the angles of the lines in the set of lines, and the voting result for each line is multiplied by the corresponding voting weight.

[0013] In one possible design, in another implementation of the first aspect of this application, the process of performing line detection on the text line edge image to obtain a set of lines representing the text line includes:

[0014] A set of first straight lines is detected from the text line edge image using a defined straight line detection algorithm;

[0015] Based on prior knowledge of the distribution direction of text lines in the document image, the first set of detected straight lines is filtered to obtain a filtered set of straight lines representing text lines.

[0016] In one possible design, in another implementation of the first aspect of the embodiments of this application, the process of geometrically correcting the document image based on the main direction of the document image to obtain a corrected image includes:

[0017] Based on prior knowledge of the main direction of the document image and the distribution direction of the text lines in the document image, the required rotation correction angle of the document image is calculated;

[0018] Based on the rotation correction angle, a 2D affine transformation matrix is ​​constructed with the center of the document image as the rotation reference point, and the document image is rotated and corrected according to the 2D affine transformation matrix to obtain the corrected image.

[0019] In one possible design, in another implementation of the first aspect of the embodiments of this application, before constructing the 2D affine transformation matrix based on the rotation correction angle and with the document image center as the rotation reference point, the following is further included:

[0020] Determine whether the absolute value of the rotation correction angle is less than a preset threshold. If so, directly use the document image as the correction image. If not, perform the step of constructing a 2D affine transformation matrix based on the rotation correction angle and with the center of the document image as the rotation reference point.

[0021] In one possible design, in another implementation of the first aspect of the embodiments of this application, before performing edge detection on the document image, the following is further included:

[0022] The document image is processed into grayscale, and then edge detection is performed on the grayscale image.

[0023] In one possible design, in another implementation of the first aspect of the embodiments of this application, after performing geometric correction on the document image based on the main direction of the document image to obtain a corrected image, the method further includes:

[0024] The corrected image is divided into multiple partitions, and the local principal direction of each partition is determined;

[0025] Based on the deviation between the local principal direction of each partition and the set reference direction, the vertical displacement of the control point of each partition is calculated, and the control point of each partition is the center point of the partition.

[0026] Based on the control points of each partition and the vertical displacement of the control points, a nonlinear displacement field function is constructed, which represents the correspondence between each pixel in the corrected image and the pixels in the global corrected image.

[0027] The nonlinear displacement field function is applied to perform global correction on the corrected image to obtain a globally corrected image.

[0028] In a second aspect, an electronic device is provided, comprising: a memory and a processor;

[0029] The memory is used to store programs;

[0030] The processor is configured to execute the program to implement the various steps of the document image correction method described in any of the first aspects of this application.

[0031] Thirdly, a readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the various steps of the document image correction method described in any of the first aspects of this application.

[0032] Fourthly, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the various steps of the document image correction method described in any of the first aspects of this application.

[0033] By employing the above technical solutions, this application addresses the shortcomings of existing document image correction techniques, such as reliance on document physical boundaries, by proposing a solution that combines text line line detection with a majority voting mechanism. This solution involves detecting lines on the text line edge images obtained from edge detection to obtain a set of lines representing the text line directions. A majority voting mechanism is then used to aggregate the angle information of these lines, determining the direction represented by the target quantized angle interval with the highest number of votes. This direction serves as the main direction of the document image, eliminating reliance on document physical boundaries and effectively overcoming local interference in the document image. Geometric correction of the document image based on its main direction significantly improves the stability and reliability of the correction results. Attached Figure Description

[0034] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0035] Figure 1 A schematic diagram of an implementation system architecture for the document image correction method provided in this application embodiment;

[0036] Figure 2 This is a schematic flowchart of a document image correction method provided in an embodiment of this application;

[0037] Figure 3a An example is a schematic diagram of the original document image to be processed;

[0038] Figure 3b An example is shown in the schematic diagram of a binarized edge image after edge detection;

[0039] Figure 3c An example is a schematic diagram of an image after Hough transform line detection;

[0040] Figure 3d An example of a first-time corrected image is shown;

[0041] Figure 3e An example is provided illustrating the effect of image partitioning.

[0042] Figure 3f An example of a globally corrected image is provided.

[0043] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0044] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0045] It is understood that before using the technical solutions disclosed in the various embodiments of this application, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this application in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0046] For document image correction technology, some solutions rely on the detection and localization of the document's physical boundaries. Correction is achieved by detecting the corner points of the document's physical boundaries and performing perspective transformation. This approach is difficult to handle application scenarios without obvious boundaries, such as a sticky note, a page of handwritten notes, or the middle area of ​​a book.

[0047] To address the aforementioned problems, the inventors of this case proposed a text line straight line detection method, which determines the document tilt angle based on the detected text line straight lines.

[0048] In the initial design phase, a relatively simple approach was to determine the document's tilt angle based on the angle of any detected text line. However, this method is easily affected by abnormal lines, resulting in low accuracy.

[0049] Furthermore, a scheme is provided to determine the document tilt angle based on the mean or median of the lines of text.

[0050] However, the above solution still has some shortcomings:

[0051] 1. The above-mentioned scheme has weak resistance to noise and outliers, especially in complex scenarios such as homework review, where the accuracy of the detection results is low.

[0052] Mean method: sensitive to outliers, a single outlier can affect the final result.

[0053] For example, in a document image, most text lines are tilted at 5 degrees, but one table line is tilted at 30 degrees. The mean method will be affected by this abnormal line.

[0054] Median method: For complex scenes, such as an image containing text tables and complex backgrounds, there are a large number of lines to be detected in all directions. The median method cannot filter out the correct lines or count the correct angles.

[0055] 2. Inability to handle multi-directional text well

[0056] Mean and median methods: cannot handle situations where multiple directions exist simultaneously in a document.

[0057] For example, in a bilingual document, Chinese text is arranged horizontally, while English text is arranged in italics. The mean and median methods may yield a meaningless intermediate value that fails to represent the document's main direction.

[0058] Based on the above research, the inventors in this case ultimately provided a line detection and majority voting (voting) mechanism to determine the main direction of a document.

[0059] By adopting a voting mechanism and considering angles that occur more frequently, it is easier to ignore outliers and has stronger resistance to noise and outliers.

[0060] By employing a voting mechanism, the dominant direction of a document can be identified, especially when multiple directions of text exist simultaneously. Taking the bilingual document example where Chinese text is arranged horizontally and English text is arranged in italics, if there are more horizontally arranged Chinese text lines, the voting mechanism will select the dominant Chinese text line direction, avoiding the problem of obtaining meaningless intermediate values ​​using the mean or median method.

[0061] The document image correction method of this application will be described in detail below. This application provides a document image correction method that can be applied to, for example... Figure 1 The system architecture shown may include a terminal 100 and a server 200. The server 200 may include one or more servers (…). Figure 1 (This example uses a server as an illustration).

[0062] Either terminal 100 or server 200 can be used independently to execute the document image correction method provided in the embodiments of this application. Alternatively, terminal 100 and server 200 can also be used collaboratively to execute the document image correction method provided in the embodiments of this application.

[0063] The following description Figure 1 The product form of the mid-terminal 100;

[0064] The terminal 100 in this application embodiment can be a mobile phone, tablet computer, learning machine, document scanner, translator, teaching large screen, wearable device, conference terminal, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc., and this application embodiment does not impose any restrictions on it.

[0065] This application provides a document image correction method, which is illustrated by applying the method to a computer device. Specifically, the computer device may be... Figure 1 The system consists of terminal 100 or a combination of terminal 100 and server 200. (Refer to...) Figure 2 The document image correction method specifically includes the following steps:

[0066] Step S100: Obtain the document image to be processed, perform edge detection on the document image, and obtain the text line edge image.

[0067] In this embodiment, the document image to be processed can be an image of various types of documents, including but not limited to: books, sticky notes, handwritten notes, etc.

[0068] In some optional examples, a grayscale processing step can be added to the document image before edge detection, followed by edge detection on the grayscale image. Adding grayscale processing can significantly reduce the computational complexity of subsequent processing and improve the accuracy of edge detection results.

[0069] The grayscale processing involves converting an RGB three-channel color document image into a single-channel grayscale image. The conversion formula uses a standard luminance-weighted method.

[0070]

[0071] In this step, the process of edge detection of the document image can use the Canny edge detection algorithm or other edge detection algorithms to extract the contour features of the text lines from the document image, clearly separating the text region from the background, and providing a high-quality binary text line edge image for subsequent line detection.

[0072] For the Canny edge detection algorithm, a dual-threshold mechanism is used to distinguish between strong and weak edges, and strong edges are connected to form continuous text contours. For example, the dual thresholds can be set to 50 and 200 respectively to effectively suppress noise interference while ensuring clear and complete text boundaries. It should be noted that the above thresholds can be adaptively adjusted according to the resolution, contrast, and other characteristics of the input image.

[0073] Reference Figure 3a and Figure 3b Examples are provided for the original document image to be processed and the text line edge image after edge detection.

[0074] Step S110: Perform line detection on the text line edge image to obtain a set of lines representing the direction of the text lines.

[0075] In this step, a line detection algorithm can be used to detect lines in the text line edge image obtained in the previous step, resulting in a set of lines representing the text lines.

[0076] One example of a line detection algorithm is the Hough transform. The line detection process based on the Hough transform includes:

[0077] The Hough transform with polar coordinate parameterization is used to map edge points in the image space to the parameter space (ρ, θ) for cumulative voting to detect line segments in the image.

[0078] In one alternative example, the parameters of the Hough transform can be configured as follows: distance resolution ρ approximately 1.0 pixels, angular resolution θ approximately 1°, and voting threshold T approximately 150.

[0079] Understandably, the above parameters can be adaptively adjusted based on the resolution of the input image and the text features.

[0080] The voting threshold is used to control the balance between the number of detected lines and their reliability, and satisfies the following formula:

[0081]

[0082] The voting threshold T typically ranges from 50 to 200.

[0083] The quantitative indicator of image sharpness, Q, ranges from 0 to 1, where 0 represents the worst quality and 1 represents the highest quality.

[0084] Quantity constant K: A proportionality constant related to the image content. Influencing factors include text density, text size, and image resolution. Images with dense text have a larger K value, while images with sparse text have a smaller K value.

[0085] Reliability sparsity : Convert the threshold into a reliability scaling factor to ensure that the reliability value is reasonable.

[0086] The number of lines detected, N:

[0087]

[0088] Increasing the voting threshold T will reduce the number of detected straight lines.

[0089] Reliability R: The probability that the detected line is indeed a real line, ranging from 0 to 1.

[0090]

[0091] Overall effect The overall effect is proportional to the square of the image quality Q, representing a balance between quantity and reliability.

[0092] As the threshold T decreases (e.g., from 150 to 50):

[0093] It can detect more straight lines. Increased sensitivity: It can detect shorter, blurrier straight lines. Increased noise: It may detect more false, noise-generated "straight lines."

[0094] As the threshold T increases (e.g., from 150 to 200):

[0095] Fewer lines can be detected. Noise reduction: False detections are significantly reduced. Possible missed detections: Some real but weak lines may be filtered out.

[0096] Relationship between voting threshold T and reliability R:

[0097] When the voting threshold is low (e.g., 50 - 100), the corresponding reliability R is low, and it is easy to detect false lines formed by random noise or texture. At the same time, the recall rate is high, and it is less likely to miss real lines.

[0098] When the voting threshold is high (e.g., 150 - 200), the corresponding reliability R is high, and only lines with a large accumulation of edge points will be detected. At the same time, the recall rate is low, and some real lines with fewer edge points may be missed.

[0099] Trade - off in the selection of specific parameters:

[0100] In the embodiments of this application, for document images with a general resolution (for example, T2 < resolution < T1), the voting threshold T can be set to 150, which balances the detection quantity and reliability, is suitable for general document images, can detect the main text lines, and filters out most of the noise and short interfering lines.

[0101] For document images with a high resolution (for example, resolution ≥ T1), the threshold T can be appropriately increased. For example, the voting threshold T can be set to 200.

[0102] For document images with a low resolution (for example, resolution ≤ T2), the threshold T can be appropriately decreased. For example, the voting threshold T can be set to 100.

[0103] In addition, in some specific application scenarios, such as when all possible lines need to be detected, a lower voting threshold can be adopted; if lines with high confidence need to be detected, a higher voting threshold can be used.

[0104] Parameter adjustment needs to find the best balance between detection sensitivity and false - detection rate.

[0105] In the Hough transform, the voting threshold determines how many votes a cell in the parameter space (ρ, θ) needs to obtain to be considered a valid line.

[0106] In practical applications, this application sets the voting threshold to 150, which can achieve a good balance between the detection rate and the false - detection rate in most document images.

[0107] Refer to Figure 3c , which exemplifies the image example after Hough transform line detection.

[0108] In some possible implementations, this step of performing line detection on the text line edge image to obtain a set of lines representing the text lines may include:

[0109] A set of first straight lines is detected from the text line edge image using a defined straight line detection algorithm.

[0110] Based on prior knowledge of the distribution direction of text lines in the document image, the first set of detected straight lines is filtered to obtain the filtered set of straight lines representing text lines.

[0111] In some scenarios, if we have prior knowledge of the direction of the text lines in a document image, such as knowing that the text lines in a document image are approximately horizontally distributed, we can use this prior knowledge to filter the first set of detected straight lines to exclude interfering line segments that are obviously not in the direction of the text lines (such as the vertical physical boundaries of the document).

[0112] In this embodiment, the text lines in a document image are approximately horizontally distributed as an example. The line detection algorithm uses the Hough transform. Therefore, the process of filtering the first set of lines can retain lines whose Hough transform angle θ corresponds to a near-horizontal direction in the image space. A Hough transform angle θ of 90° represents a horizontal line; therefore, in some examples, lines with Hough transform angle θ in the range of 45° to 135° can be filtered.

[0113] For images that primarily contain vertical text (such as some East Asian language typography), the filter range can be adjusted accordingly to be closer to the vertical direction.

[0114] Step S120: Based on the angles of each line in the line set, vote on each set of quantized angle intervals, and determine the target quantized angle interval with the highest number of votes. The direction represented by the target quantized angle interval is taken as the main direction of the document image.

[0115] This application proposes a document main direction estimation method based on a statistical voting mechanism, which can be implemented by constructing an angle statistical histogram.

[0116] For the angles of all lines in the set of lines obtained in the previous step, they can be quantized with a set precision (e.g., 1°), and the number of lines in each quantized angle interval (i.e., "votes") can be counted. The main direction of the document image is the direction θ_max represented by the target quantized angle interval with the highest number of votes.

[0117] The voting mechanism based on angular statistical histograms can be formalized as an optimization problem of finding the dominant pattern. More generally, the process can be expressed in the following mathematical form:

[0118]

[0119] in:

[0120] It is the main direction of the document image to be determined.

[0121] It is the direction angle of the i-th line.

[0122] N represents the total number of lines in the set of lines.

[0123] It is the confidence weight (voting weight) of the i-th line, which can be calculated based on the length of the line and / or the goodness of fit, so that a more reliable line can be given a larger confidence weight.

[0124] K(·) is a kernel function used to smooth the voting process. In the histogram method, it can be viewed as a rectangular kernel function.

[0125] The majority voting mechanism based on histograms or kernel density estimation described above is inherently robust from a mathematical perspective. This mechanism finds the dominant orientation of the document image by maximizing the objective function. Within this framework, the correct, majority text line orientations will form a significant global peak in the objective function, while local peaks caused by a few outliers or interference lines will be masked. This means that as long as the correct text lines are numerically dominant, or have a higher confidence weight, the voting result will not be swayed by a few biased data points.

[0126] Therefore, compared with existing methods that rely on a single text line or local region direction, this application can inherently resist interference from local noise and non-textual structures, and can significantly improve the stability and reliability of the correction results without relying on complex outlier detection and removal steps.

[0127] In some possible implementations, this step, which involves voting on each quantized angle interval based on the angles of the lines in the set of lines, may include:

[0128] The voting weight of each line is determined based on its length and / or goodness of fit in the set of lines. The voting weight is positively correlated with the length of the line and the goodness of fit. The goodness of fit indicates the degree of agreement between the detected line and the actual edge points in the text line edge image.

[0129] Voting is conducted on each set of lines within a defined quantized angle range based on their angles, and the voting result for each line is multiplied by the corresponding voting weight.

[0130] In this embodiment, when voting based on the angle of a straight line, the voting weight (confidence weight) of different straight lines is further considered. This voting weight is positively correlated with the length of the straight line and the goodness of fit.

[0131] A high-fit straight line means that a large number of edge points are closely arranged near the line, forming a clear and continuous line. This usually corresponds to a long, straight line of text with well-defined edges.

[0132] A straight line with low goodness of fit means that only a few scattered points are distributed around the line, or the points are loosely distributed. This could correspond to a short, discontinuous, or unclear line of text, or even just background noise.

[0133] In this application embodiment, several optional methods for calculating the goodness of fit of a straight line are introduced:

[0134] Method 1: Based on the distance from the point to the line

[0135] This method assesses the quality of the fit by calculating how many edge points actually "belong" to the line and how close they are to the line.

[0136] Determine the support point set of the line: First, we need to find all the edge pixels that vote for this line (ρ_i, θ_i). These points are called the "support points" of this line.

[0137] Calculate the average distance: Then, calculate the average distance from all these support points to the line. The smaller the distance, the better the points fit the line, and the higher the goodness of fit.

[0138] Convert distance to weights: the weights should be inversely proportional to the average distance. Example:

[0139] w i = 1 / (1 + average_distance).

[0140] Or use exponential decay: w i = exp(-average_distance), which makes it more sensitive to distance.

[0141] Method 2: Based on the number of support points

[0142] The longer a line of text is, the more edge points it typically has. Therefore, simply using the number of edge points that support this line as a weight is a very effective proxy metric.

[0143] Calculate: w i = Size of the support point set (i.e., how many points were projected onto this line).

[0144] By incorporating linear weights into the voting process, the accuracy of the voting results can be improved, leading to a more precise determination of the main direction of the final document.

[0145] Step S130: Perform geometric correction on the document image based on the main direction of the document image to obtain the corrected image.

[0146] Specifically, after obtaining the principal direction θ_max of the document image in the previous step, the required rotation correction angle of the document image can be calculated based on the prior knowledge of the principal direction of the document image and the distribution direction of the text lines in the document image.

[0147] Taking the prior knowledge of the text line distribution direction to represent the horizontal distribution of text lines as an example, when the Hough transform angle θ is 90°, it represents a horizontal line. Then, the required image rotation correction angle is calculated as follows:

[0148] rotationAngle = θ_max - 90°

[0149] Based on the rotation correction angle rotationAngle, a 2D affine transformation matrix is ​​constructed with the center of the document image as the rotation reference point, and the document image is rotated and corrected according to the 2D affine transformation matrix to obtain the corrected image.

[0150] In some possible implementations, before constructing the 2D affine transformation matrix in the above steps and performing rotation correction on the document image according to the 2D affine transformation matrix, the following steps can be added:

[0151] Determine whether the absolute value of the rotation correction angle is less than a preset threshold (e.g., 1°). If yes, directly use the document image as the correction image. If no, execute the step of constructing a 2D affine transformation matrix based on the rotation correction angle and with the center of the document image as the rotation reference point.

[0152] By adding a process to determine the absolute value of the rotation correction angle, when the rotation correction angle is determined to be small, in order to avoid overprocessing, the rotation correction process can be skipped, and the original document image can be directly used as the final corrected image, thereby suppressing minor corrections.

[0153] Reference Figure 3d This example illustrates a corrected image obtained by geometrically correcting a document image based on its principal direction. The principal direction of the document image is the direction dominated by the text portion; therefore, the text portion in the corrected image regains its horizontal distribution.

[0154] This application addresses the shortcomings of existing document image correction techniques, such as reliance on physical document boundaries. It proposes a solution combining text line line detection with a majority voting mechanism. By detecting lines in the text line edge image obtained from edge detection, a set of lines representing the text line direction is obtained. A majority voting mechanism is then used to aggregate the angle information of these lines, determining the direction represented by the target quantized angle interval with the highest number of votes. This direction serves as the main direction of the document image, eliminating reliance on physical document boundaries and effectively overcoming local interference in the document image. Geometric correction of the document image based on its main direction significantly improves the stability and reliability of the correction results.

[0155] The document image correction method described in the above embodiments of this application solves the overall orientation problem of the document through global rotation transformation. However, in reality, the following scenarios may also exist:

[0156] 1. Inconsistent orientation within a document: For example, in a composite layout document, the orientation of different areas (such as text areas and table areas) is inconsistent. (Example:) Figure 3a As shown.

[0157] 2. Nonlinear geometric deformation, such as curved books or wrinkled documents, causes the IDE to be physically bent.

[0158] To further address the aforementioned issues, this embodiment provides a zonal fine correction process based on the methods described in the previous embodiments. Zonal fine correction achieves refined correction across the entire image area through local deformation.

[0159] In this embodiment, the document image correction method may further include a partition fine correction process, which includes the following steps:

[0160] S200. Divide the corrected image obtained in step S130 of the above embodiment into multiple partitions and determine the local principal direction of each partition.

[0161] When partitioning the corrected image, adaptive partitioning can be performed based on the image's content. For images without a clear layout structure, a grid partitioning method can also be used to divide the corrected image into M×N partitions. For example... Figure 3e This example illustrates a schematic diagram of the partitioning result of a corrected image using a grid.

[0162] After partitioning the corrected image, the method described in the previous embodiment can be applied again in each partition to obtain the local principal direction θ_local_i of each partition through text line line detection and direction estimation.

[0163] S210. Based on the deviation between the local main direction of each partition and the set reference direction, calculate the vertical displacement of the control point of each partition, where the control point of each partition is the center point of the partition.

[0164] In this step, the deviation between the local principal direction θ_local_i of each partition and the set reference direction is calculated. The set reference direction can be an absolute horizontal direction or an absolute vertical direction, depending on the user's desired distribution direction of text lines in the corrected document image. In this embodiment, taking an absolute horizontal direction of 0° as an example, the deviation between the local principal direction θ_local_i of each partition and the set reference direction is:

[0165] Δθ_i = 0° - θ_local_i

[0166] For each partition, the coordinates of the partition's center point are used as the partition's control point. The vertical displacement v_i is calculated for each partition's control point. This vertical displacement is derived from the angle difference Δθ_i, and the calculation formula is as follows:

[0167]

[0168] in, This indicates that the angle difference is converted into slope.

[0169] This indicates the vertical position of the partition center relative to the center of the document image.

[0170] κ represents an adjustable proportionality coefficient that controls the overall deformation intensity.

[0171] S220. Based on the control points of each partition and the vertical displacement of the control points, a nonlinear displacement field function is constructed. The nonlinear displacement field function represents the correspondence between each pixel in the corrected image and the pixels in the global corrected image.

[0172] In this embodiment, the TPS transformation of a thin template strip is used as an example. Based on the control points of each partition and the vertical displacement of the control points, a globally smooth displacement field function can be generated:

[0173] f(x, y) = (u(x,y), v(x,y))

[0174] This function ensures accurate displacement at control points and a smooth transition between control points. The function outputs the horizontal and vertical displacements (u,v) required to correct any point (x,y) on the image.

[0175] S230. Apply the nonlinear displacement field function to perform global correction on the correction image to obtain the global correction image.

[0176] By resampling the image, a nonlinear displacement field function is applied to the corrected image to achieve pixel-level fine correction. For each pixel (x_dst, y_dst) in the corrected image obtained in step S130, the corresponding position (x_src, y_src) in the global corrected image is found using the displacement field f(x,y). The coordinate mapping relationship is as follows:

[0177] x_src = x_dst + u(x_dst, y_dst)

[0178] y_src = y_dst + v(x_dst, y_dst)

[0179] Since (x_src, y_src) are mostly non-integer coordinates, in this embodiment, an interpolation algorithm (such as bicubic interpolation or other interpolation algorithms) can be further used to calculate the pixel value at the corresponding position and assign it to the global correction image. The interpolation process is equivalent to performing a smooth and continuous non-linear deformation on the image, and finally outputs a global correction image after all local partitions have been finely corrected.

[0180] Reference Figure 3f This example demonstrates a globally corrected image compared to the initial corrected image. Figure 3d The tables in the globally corrected image are also corrected to a horizontal distribution.

[0181] The partitioning fine correction method provided in this embodiment can effectively eliminate surface distortions that traditional methods cannot handle, restore the text content to a flat state, and greatly improve visual readability.

[0182] The document image correction method provided in the above embodiments of this application has the following advantages:

[0183] Fully automated: It achieves zero-intervention end-to-end processing from image input to result output.

[0184] Strong robustness: The majority voting mechanism has inherent fault tolerance to local noise, abnormal text lines and non-textual interference.

[0185] Wide applicability: It can effectively handle borderless documents, complex layouts (such as text containing tables), and non-linear deformations (such as curved books).

[0186] High execution performance: The voting strategy employed in this application has a computational complexity far lower than traditional methods that require searching and optimizing complex quadrilateral contours. In actual mobile deployments, the processing efficiency of this application is significantly better than traditional solutions, meeting the requirements for real-time processing.

[0187] This application also provides an electronic device in its embodiments. (See reference...) Figure 4The diagram illustrates a structural schematic suitable for implementing the electronic device in the embodiments of this application. The electronic device in the embodiments of this application may include, but is not limited to, fixed terminals such as mobile phones, tablets, learning machines, document scanners, and so on. Figure 4 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0188] like Figure 4 As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 1, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 2 or a program loaded from a storage device 8 into a random access memory (RAM) 3, to implement the document image correction method of the foregoing embodiments of this application. When the electronic device is powered on, the RAM 3 also stores various programs and data required for the operation of the electronic device. The processing unit 1, ROM 2, and RAM 3 are interconnected via a bus 4. An input / output (I / O) interface 5 is also connected to the bus 4.

[0189] Typically, the following devices can be connected to I / O interface 5: input devices 6 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 7 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 8 including, for example, memory cards, hard drives, etc.; and communication devices 9. Communication device 9 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 4 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0190] This application also provides a computer program product including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the document image correction methods provided in this application.

[0191] This application also provides a computer-readable storage medium carrying one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the document image correction methods provided in this application.

[0192] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0193] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0194] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0195] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

[0196] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

Claims

1. A document image correction method characterized by, The method comprises: obtaining a document image to be processed, performing edge detection on the document image to obtain a text line edge image; performing straight line detection on the text line edge image to obtain a straight line set representing text lines; voting for each quantized angle interval according to the angles of the straight lines in the straight line set, and determining a target quantized angle interval with the highest number of votes, and taking the direction represented by the target quantized angle interval as the main direction of the document image; performing geometric correction on the document image based on the main direction of the document image to obtain a corrected image.

2. The method of claim 1, wherein, The process of voting for each quantized angle interval according to the angles of the straight lines in the straight line set comprises: determining the voting weight of a straight line based on the length and / or fitting goodness of each straight line in the straight line set, the voting weight being positively correlated with the length and fitting goodness of the straight line, and the fitting goodness representing the degree of fit between the detected straight line and the actual edge points in the text line edge image; voting for each quantized angle interval according to the angles of the straight lines in the straight line set, and multiplying the voting result of each straight line by the corresponding voting weight.

3. The method of claim 1, wherein, The process of performing straight line detection on the text line edge image to obtain a straight line set representing text lines comprises: detecting a first straight line set from the text line edge image using a set straight line detection algorithm; screening the detected first straight line set according to prior knowledge of the text line distribution direction of the document image to obtain a screened straight line set representing text lines.

4. The method of claim 1, wherein, The process of performing geometric correction on the document image based on the main direction of the document image to obtain a corrected image comprises: calculating the required rotation correction angle of the document image based on the main direction of the document image and the prior knowledge of the text line distribution direction of the document image; constructing a 2D affine transformation matrix based on the rotation correction angle and taking the center of the document image as the rotation reference point, and performing rotation correction on the document image according to the 2D affine transformation matrix to obtain a corrected image.

5. The method of claim 4, wherein, Before constructing a 2D affine transformation matrix based on the rotation correction angle and taking the center of the document image as the rotation reference point, the method further comprises: determining whether the absolute value of the rotation correction angle is less than a preset threshold, if yes, directly taking the document image as the corrected image, and if no, performing the step of constructing a 2D affine transformation matrix based on the rotation correction angle and taking the center of the document image as the rotation reference point.

6. The method of claim 4, wherein, Before performing edge detection on the document image, the method further comprises: performing grayscale processing on the document image, and then performing edge detection on the grayscale image.

7. The method according to any one of claims 1 to 6, characterized in that, After performing geometric correction on the document image based on the main direction of the document image to obtain a corrected image, the method further comprises: dividing the corrected image into multiple partitions and determining the local main direction of each partition; calculating the vertical displacement amount of the control point of each partition based on the deviation between the local main direction of each partition and a set reference direction, the control point of each partition being the center point of the partition; Based on the control points of each partition and the vertical displacement amount of the control points, a nonlinear displacement field function is constructed, the nonlinear displacement field function representing a corresponding relationship between each pixel point in the correction image and a pixel point in a global correction image; A global correction image is obtained by applying the nonlinear displacement field function to the correction image for global correction.

8. An electronic device, comprising: Comprise: a memory and a processor; the memory is used to store a program; the processor is used to execute the program, and realize each step of the document image correction method according to any one of claims 1-7.

9. A readable storage medium, having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize each step of the document image correction method according to any one of claims 1-7.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to realize each step of the document image correction method according to any one of claims 1-7. The computer program is executed by the processor to realize each step of the document image correction method according to any one of claims 1-7.

Images