Text Image Correction Method, Device, Computer System, and Readable Storage Medium

Abstract

The present disclosure provides a text image correction method, which can be used in the fields of big data, artificial intelligence technology or other fields. Among them, the method includes: performing a straight line detection process on the text image to be processed to obtain the tilt angle of the text image to be processed; performing tilt correction on the text image to be processed according to the tilt angle to obtain a preliminary corrected image; performing a distortion fitting process on the preliminary corrected image to obtain the distortion variable of the preliminary corrected image; and performing distortion correction on the preliminary corrected image based on the distortion variable. The present disclosure also provides a text image correction device, a computer system, a readable storage medium, and a computer program product.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technologies, and more particularly, to a method, apparatus, computer system, readable storage medium, and computer program product for text image correction. Background Art

[0002] In the era of artificial intelligence, the retention of paper materials such as receipts, certificates, and vouchers mostly involves operations such as photographing, scanning, and remote transmission, and thus increasingly relies on text image data. In addition, intelligent recognition, extraction, and processing also require text image data as a basis.

[0003] In the process of implementing the concept of the present disclosure, the inventors found that at least the following problems exist in the related art: There are often phenomena such as blurring, tilting, or distortion in text images, resulting in large errors in automatic recognition and processing. Summary of the Invention

[0004] In view of this, the present disclosure provides a method, apparatus, computer system, readable storage medium, and computer program product for text image correction.

[0005] One aspect of the present disclosure provides a method for text image correction, including:

[0006] Performing a straight line detection process on a text image to be processed to obtain the tilt angle of the text image to be processed;

[0007] Performing tilt correction on the text image to be processed according to the tilt angle to obtain a preliminary corrected image;

[0008] Performing a distortion fitting process on the preliminary corrected image to obtain the distortion variable of the preliminary corrected image; and

[0009] Performing distortion correction on the preliminary corrected image based on the distortion variable.

[0010] According to an embodiment of the present disclosure, before performing a straight line detection process on a text image to be processed to obtain the tilt angle of the text image to be processed, the method further includes:

[0011] Performing preprocessing on the text image to be processed to obtain a binary image;

[0012] Performing dilation and erosion processing on the binary image to obtain a binary connected domain image;

[0013] Performing edge detection processing on the binary connected domain image to obtain a text contour image;

[0014] Performing cropping processing on the text contour image to obtain a plurality of cropped images for performing a straight line detection process on the plurality of cropped images.

[0015] According to an embodiment of the present disclosure, performing a straight-line detection process on a text image to be processed to obtain the tilt angle of the text image to be processed includes:

[0016] Performing a straight-line detection process on each of the multiple cropped images to obtain multiple straight-line slopes, where each straight-line slope corresponds to one cropped image; and

[0017] Based on the multiple straight-line slopes, determining the tilt angle.

[0018] According to an embodiment of the present disclosure, performing tilt correction on the text image to be processed according to the tilt angle to obtain a preliminary correction image includes:

[0019] Determining the relationship between the tilt angle and a preset threshold;

[0020] In the case where the tilt angle is not equal to the preset threshold, performing the operation of performing tilt correction on the text image to be processed according to the tilt angle to obtain a preliminary correction image;

[0021] In the case where the tilt angle is equal to the preset threshold, using the binary connected component image as the preliminary correction image.

[0022] According to an embodiment of the present disclosure, performing a distortion fitting process on the preliminary correction image to obtain the distortion variable of the preliminary correction image includes:

[0023] Obtaining the pixel points of multiple preliminary correction images;

[0024] Based on the multiple pixel points, generating a distortion representation matrix, where each element in the distortion representation matrix is the distortion result of a pixel point, and the distortion result is determined based on the pixel values of two adjacent and symmetric pixel points of the pixel point;

[0025] Based on the distortion representation matrix, obtaining the sparsity of the distortion representation matrix; and

[0026] Based on the sparsity of the distortion representation matrix, obtaining the distortion variable of the preliminary correction image.

[0027] According to an embodiment of the present disclosure, obtaining the distortion variable of the preliminary correction image based on the sparsity of the distortion representation matrix includes:

[0028] Constructing a fitting correction function;

[0029] Based on the sparsity of the distortion representation matrix, adjusting the parameter values in the fitting correction function until the sparsity of the distortion representation matrix converges;

[0030] Using the fitting correction function corresponding to when the sparsity of the distortion representation matrix converges as the final fitting correction function; and

[0031] Based on the final fitting correction function, the distortion variables of the preliminary correction image are obtained.

[0032] Another aspect of the present disclosure provides a text image correction device, including:

[0033] A straight line detection processing module, configured to perform straight line detection processing on the text image to be processed, and obtain the inclination angle of the text image to be processed;

[0034] An inclination correction module, configured to perform inclination correction on the text image to be processed according to the inclination angle, and obtain a preliminary correction image;

[0035] A distortion fitting processing module, configured to perform distortion fitting processing on the preliminary correction image, and obtain the distortion variables of the preliminary correction image; and

[0036] A distortion correction module, configured to perform distortion correction on the preliminary correction image based on the distortion variables.

[0037] Another aspect of the present disclosure provides a computer system, including:

[0038] One or more processors;

[0039] A memory, configured to store one or more programs,

[0040] wherein, when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the above-mentioned text image correction method.

[0041] Another aspect of the present disclosure provides a computer-readable storage medium, on which executable instructions are stored, and when the instructions are executed by a processor, the processor is caused to implement the above-mentioned text image correction method.

[0042] Another aspect of the present disclosure provides a computer program product, including a computer program, where the computer program includes computer-executable instructions, and the instructions are used to implement the above-mentioned text image correction method when executed.

[0043] According to the embodiments of the present disclosure, because the technical means of performing straight line detection processing on the text image to be processed to obtain the inclination angle of the text image to be processed; performing inclination correction on the text image to be processed according to the inclination angle to obtain a preliminary correction image; performing distortion fitting processing on the preliminary correction image to obtain the distortion variables of the preliminary correction image; and performing distortion correction on the preliminary correction image based on the distortion variables are adopted, inclination correction and distortion correction are performed on the text image; therefore, at least partially, the technical problem in the prior art that the automatic recognition and processing errors are large due to the phenomena of blurring, inclination or distortion often existing in the text image is overcome, and furthermore, the technical effect of improving the recognition accuracy of the text image is achieved. Description of the Drawings

[0044] Through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, the above and other objects, features, and advantages of the present disclosure will become clearer. In the drawings:

[0045] Figure 1 An exemplary system architecture in which the text image correction method and apparatus of the present disclosure can be applied is schematically shown;

[0046] Figure 2 A flowchart of the text image correction method according to an embodiment of the present disclosure is schematically shown;

[0047] Figure 3 A schematic diagram showing the occurrence of tilt distortion in an embodiment of the present disclosure is schematically shown;

[0048] Figure 4 Schematically shows Figure 3 A schematic diagram after text image correction is performed;

[0049] Figure 5 A flowchart of the text image correction method according to another embodiment of the present disclosure is schematically shown;

[0050] Figure 6 A flowchart of the text image correction method according to another embodiment of the present disclosure is schematically shown;

[0051] Figure 7 A block diagram of the text image correction apparatus according to an embodiment of the present disclosure is schematically shown; and

[0052] Figure 8 A block diagram of a computer system for the text image correction method according to an embodiment of the present disclosure is schematically shown. Detailed implementation manners

[0053] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the following detailed description, for the sake of explanation, many specific details are set forth in order to provide a comprehensive understanding of the embodiments of the present disclosure. However, obviously, one or more embodiments can also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily confusing the concepts of the present disclosure.

[0054] The terms used herein are merely for describing specific embodiments and are not intended to limit the present disclosure. The terms "including", "comprising", etc. used herein indicate the presence of the described features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0055] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those of ordinary skill in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0056] In cases where expressions similar to "at least one of A, B, and C, etc." are used, generally, it should be interpreted according to the meaning commonly understood by those of ordinary skill in the art (for example, "a system having at least one of A, B, and C" should include, but is not limited to, a system having only A, only B, only C, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.). In cases where expressions similar to "at least one of A, B, or C, etc." are used, generally, it should be interpreted according to the meaning commonly understood by those of ordinary skill in the art (for example, "a system having at least one of A, B, or C" should include, but is not limited to, a system having only A, only B, only C, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.).

[0057] Embodiments of the present disclosure provide a method for correcting a text image. The method includes: performing a straight-line detection process on the text image to be processed to obtain the tilt angle of the text image to be processed; performing tilt correction on the text image to be processed according to the tilt angle to obtain a preliminary corrected image; performing a distortion fitting process on the preliminary corrected image to obtain the distortion variable of the preliminary corrected image; and performing distortion correction on the preliminary corrected image based on the distortion variable.

[0058] According to the embodiments of the present disclosure, by performing tilt correction and distortion correction on a text image through the text image correction method, errors occurring in the text analysis and recognition processes can be reduced, and the recognition accuracy can be improved.

[0059] Figure 1 Exemplary system architecture 100 to which the text image correction method and apparatus according to the embodiments of the present disclosure can be applied is schematically shown. It should be noted that Figure 1 What is shown is only an example of the system architecture to which the embodiments of the present disclosure can be applied, to help those of ordinary skill in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments, or scenarios.

[0060] As Figure 1As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0061] Users can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications may be installed on the terminal devices 101, 102, 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (for example only).

[0062] The terminal devices 101, 102, 103 may be various electronic devices with a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, and desktop computers, etc.

[0063] The server 105 may be a server that provides various services, such as a background management server that supports the websites browsed by users using the terminal devices 101, 102, 103 (for example only). The background management server may analyze and process data such as received user requests, etc., and feedback the processing results (such as web pages, information, or data, etc. obtained or generated according to user requests) to the terminal devices.

[0064] It should be noted that the text image correction method provided by the embodiments of the present disclosure can generally be executed by the server 105. Correspondingly, the text image correction device provided by the embodiments of the present disclosure can generally be set in the server 105. The text image correction method provided by the embodiments of the present disclosure can also be executed by a server or a server cluster different from the server 105 and capable of communicating with the terminal devices 101, 102, 103 and / or the server 105. Correspondingly, the text image correction device provided by the embodiments of the present disclosure can also be set in a server or a server cluster different from the server 105 and capable of communicating with the terminal devices 101, 102, 103 and / or the server 105. Or, the text image correction method provided by the embodiments of the present disclosure can also be executed by the terminal devices 101, 102, or 103, or can also be executed by other terminal devices different from the terminal devices 101, 102, or 103. Correspondingly, the text image correction device provided by the embodiments of the present disclosure can also be set in the terminal devices 101, 102, or 103, or set in other terminal devices different from the terminal devices 101, 102, or 103.

[0065] For example, the text image to be processed may originally be stored in any one of the terminal devices 101, 102, or 103 (e.g., terminal device 101, but not limited thereto), or stored on an external storage device and can be imported into the terminal device 101. Then, the terminal device 101 may send the text image to be processed to other terminal devices, servers, or server clusters, and the text image correction method provided by the embodiments of the present disclosure may be executed by other servers or server clusters that receive the text image to be processed.

[0066] It should be understood that Figure 1 the numbers of terminal devices, networks, and servers in

[0067] It should be noted that the text image correction method, text image correction device, computer system, computer-readable storage medium, and computer program product of the present disclosure can be used in the fields of big data and artificial intelligence technologies, and can also be used in any field other than the fields of big data and artificial intelligence technologies. The application fields of the text image correction method, text image correction device, computer system, computer-readable storage medium, and computer program product of the present disclosure are not limited.

[0068] Figure 2 Schematically shows a flowchart of a text image correction method according to an embodiment of the present disclosure.

[0069] As Figure 2 shown, the method includes operations S210 to S240.

[0070] In operation S210, perform a straight line detection process on the text image to be processed to obtain the tilt angle of the text image to be processed.

[0071] In operation S220, perform tilt correction on the text image to be processed according to the tilt angle to obtain a preliminary corrected image.

[0072] In operation S230, perform a distortion fitting process on the preliminary corrected image to obtain the distortion variable of the preliminary corrected image.

[0073] In operation S240, perform distortion correction on the preliminary corrected image based on the distortion variable.

[0074] According to an embodiment of the present disclosure, the text image to be processed may be an image containing text, but is not limited thereto, and may also be an image containing both text and other non-text information, such as information in tables, graphs, etc. According to an embodiment of the present disclosure, the text image to be processed may be an electronic text image such as a receipt, customer certificate, voucher, etc. in the banking and financial fields.

[0075] Figure 3 Schematically shows a schematic diagram of the tilt and twist occurring in an embodiment of the present disclosure.

[0076] As Figure 3 shown, for fields such as banks and finance that have extremely high requirements for digital accuracy, problems such as difficult text recognition and easy occurrence of recognition errors are caused by the tilt and twist of text images. The tilt and twist of text images become various obstacles to image applications, which is not conducive to the further analysis and processing of text images. Using manual investigation and processing is both time-consuming and laborious, and certain errors are also likely to occur.

[0077] According to an embodiment of the present disclosure, the text image to be processed can be first subjected to tilt correction, and then to twist correction, finally achieving the effect of tilt and twist correction. In the embodiment of the present disclosure, the horizontal baseline is used as the correction line for subsequent twist correction. Therefore, by adopting the correction method of first tilt correction and then twist correction, the achieved correction effect is good and the correction speed is fast.

[0078] Figure 4 Schematically shows a schematic diagram after Figure 3 performing text image correction.

[0079] As Figure 4 shown, according to an embodiment of the present disclosure, through the tilt correction and twist correction of the text image, the clarity and recognizability of the text image are improved, thereby improving the accuracy and precision of subsequent automatic recognition and analysis of the text image.

[0080] Next, with reference to Figures 5 to 6 , in combination with specific embodiments, Figure 2 the method shown is further described.

[0081] Figure 5 Schematically shows a flowchart of a text image correction method according to another embodiment of the present disclosure.

[0082] As Figure 5 shown, before performing straight line detection processing on the text image to be processed to obtain the tilt angle of the text image to be processed, the text image correction method includes operations S510 to S560.

[0083] In operation S510, preprocess the text image to be processed to obtain a binary image.

[0084] According to an embodiment of the present disclosure, preprocessing the text image to be processed may include grayscale processing and binary processing. Among them, the text image to be processed can be first subjected to grayscale processing to obtain a grayscale image, and then the grayscale image is used for binary processing to obtain a binary image.

[0085] According to an embodiment of the present disclosure, a grayscale image can be converted into a grayscale matrix form and processed using the following formula (1) to obtain a binary image.

[0086] Formula (1):

[0087] Where P(n) represents the grayscale value of the nth pixel in the grayscale matrix, s refers to the first s pixels of the nth pixel; T(n) represents the corresponding pixel value after binarization, and the average grayscale value of the grayscale image is used as the initial value for iteration.

[0088] In operation S520, perform dilation and erosion processing on the binary image to obtain a binary connected component image.

[0089] According to an embodiment of the present disclosure, by performing dilation and erosion processing on the binary image, the influence of the disconnection of the text in the text image can be reduced, which is beneficial to subsequent line detection processing.

[0090] In operation S530, perform edge detection processing on the binary connected component image to obtain a text contour image.

[0091] According to an embodiment of the present disclosure, the edge detection processing can be implemented through the following operations. Define a convolution kernel G with a size of 3×3. At this time, let the convolution kernel Use the convolution kernel to slide from left to right and from top to bottom on the binary connected component image to perform convolution operations, set the weight to 1 / 9. If the convolution result is -1, then g(x + 1, y + 1) = 1; if the convolution result is not -1, then g(x + 1, y + 1) = 0. At this time, convert the pixel values in the middle part of the binary connected component image to 1, which can reduce interference during line detection processing.

[0092] In operation S540, perform cropping processing on the text contour image to obtain multiple cropped images.

[0093] According to an embodiment of the present disclosure, when performing cropping processing on the text contour image, it can be horizontally equally divided into multiple parts.

[0094] According to an alternative embodiment of the present disclosure, the cropping processing of the present disclosure can be cropped into three horizontally equal parts.

[0095] In operation S550, perform line detection processing on each of the multiple cropped images respectively to obtain multiple line slopes, where each line slope corresponds to one cropped image.

[0096] According to an embodiment of the present disclosure, multiple slope values are obtained by using the Hough transform line detection to process the cropped image.

[0097] According to an alternative embodiment of the present disclosure, the pixel points with a pixel value of 1 in the three cropped images obtained by horizontally trisecting the text contour image can be sequentially placed into the corresponding point sets S1, S2, and S3 to reduce the influence brought by the vertical direction. A straight line can be represented in the polar coordinate system, such as the formula: After simplification, we get r = x cosθ + y sinθ, where r represents the distance from the straight line to the origin, and θ represents the angle between the straight line and the x-axis. Thus, for any point with coordinates (x0, y0), all the straight lines passing through this point satisfy r(θ) = x0 cosθ + y0 sinθ.

[0098] Randomly take pixel points from the point set S1 and project them into the polar coordinate parameter space. Calculate the corresponding θ values at each r. The number of pixel points that can be mapped to this straight line is represented by the accumulator acc(r, θ). For each obtained corresponding (r, θ), acc(r, θ) is incremented by 1, and the pixel point is deleted from the point set S1.

[0099] According to an embodiment of the present disclosure, the threshold thr of the accumulator acc(r, θ) can be set to the length w of the image. If acc(r, θ) ≥ w, the corresponding (r, θ) determines a straight line, and the remaining pixel points in the point set S1 located on this straight line are deleted, and this accumulator is cleared. If acc(r, θ) < w, the mapping of pixel points continues; if acc(r, θ) < w and the point set S1 is already empty, the threshold thr of the accumulator acc(r, θ) is reduced by 2, and the mapping of pixel points is repeated again; until the (r, θ) of the point set S1 corresponds to a straight line; similarly, the straight line (r, θ) pairs of the point sets S2 and S3 are obtained.

[0100] In operation S560, based on multiple straight line slopes, the inclination angle is determined.

[0101] According to an embodiment of the present disclosure, for the determination of this inclination angle, the average value of multiple straight line slopes can be calculated, and the inclination angle is determined based on the average value of this straight line slope.

[0102] According to other embodiments of the present disclosure, according to the inclination angle, performing skew correction on the text image to be processed to obtain a preliminary corrected image may include the following operations.

[0103] Determine the relationship between the inclination angle and a preset threshold; in the case where the inclination angle is not equal to the preset threshold, perform the operation of performing skew correction on the text image to be processed according to the inclination angle to obtain a preliminary corrected image; and in the case where the inclination angle is equal to the preset threshold, use the binary connected component image as the preliminary corrected image.

[0104] According to an embodiment of the present disclosure, the tilt angle can be compared with a preset threshold, and based on the relationship between the two, it can be determined whether the text image to be processed is tilted. In an embodiment of the present disclosure, the preset threshold can be set to 0°. When the tilt angle is not equal to 0°, it indicates that the text image to be processed is tilted. In this case, tilt correction needs to be performed on the text image to be processed according to the tilt angle, and the image after tilt correction is used as the preliminary correction image. When the tilt angle is equal to the preset threshold, it indicates that the text image to be processed is not tilted, and the binary connected domain image can be directly used as the preliminary correction image for subsequent distortion judgment and correction.

[0105] Figure 6 Schematically shows a flowchart of a text image correction method according to another embodiment of the present disclosure.

[0106] As Figure 6 shown, performing a distortion fitting process on the preliminary correction image, and obtaining the distortion variables of the preliminary correction image includes operations S610 to S640.

[0107] In operation S610, obtain the pixel points of multiple preliminary correction images.

[0108] In operation S620, based on multiple pixel points, generate a distortion representation matrix, where each element in the distortion representation matrix is the distortion result of a pixel point, and the distortion result is determined based on the pixel values of two adjacent and symmetric pixel points of the pixel point.

[0109] In operation S630, based on the distortion representation matrix, obtain the sparsity of the distortion representation matrix.

[0110] In operation S640, based on the sparsity of the distortion representation matrix, obtain the distortion variables of the preliminary correction image.

[0111] According to an embodiment of the present disclosure, to obtain the pixel points of multiple preliminary correction images, it can be based on the midline of the preliminary correction image. Obtain m line segments with a pixel value of 0, generate two horizontal lines y1 = k1 and y2 = k2 according to the one-third and two-thirds points of the line segment, and perform random sampling (x1, y1), (x2, y2)... (x n , y n ) on both sides of the midline.

[0112] According to an embodiment of the present disclosure, the distortion representation matrix can be set as where c 2mn represents the value of the pixel point (x n , y n ) in the vertical direction and the value of the pixel point (x n , y n - 1), and the value of the pixel point (x n , y n+1) The result of the exclusive OR of the values of the pixel points. If the values of the upper and lower pixel points of the sampling point are different, it indicates that there may be distortion at that location, and the exclusive OR value is 1. If the values of the upper and lower pixel points of the sampling point are the same, the exclusive OR value is 0. Finally, a matrix K of 0s and 1s is obtained. 2m×n .

[0113] According to an embodiment of the present disclosure, the sparsity of the distortion characterization matrix K 2m×n can be expressed as where m×n represents the number of elements of the distortion characterization matrix, and ∑|c ij | represents the sum of the absolute values of all elements. The value of the sparsity is in the range of [0, 1]. The larger the value, the sparser it is.

[0114] According to an embodiment of the present disclosure, the operation S640 for obtaining the distortion variable of the preliminary corrected image based on the sparsity of the distortion characterization matrix may include the following operations.

[0115] Construct a fitting correction function; adjust the parameter values in the fitting correction function based on the sparsity of the distortion characterization matrix until the sparsity of the distortion characterization matrix converges; use the fitting correction function corresponding to the convergence of the sparsity of the distortion characterization matrix as the final fitting correction function; and obtain the distortion variable of the preliminary corrected image based on the final fitting correction function.

[0116] According to an embodiment of the present disclosure, the fitting correction function may be a high-order function, such as Formula 2 and Formula 3.

[0117] Formula 2: x(u, v) = u×(1 + k1×u 2 + k2×v 2 );

[0118] Formula 3: y(u, v) = v×(1 + k1×u 2 + k2×v 2 );

[0119] where k1 is a parameter for controlling the distortion correction in the horizontal direction, k2 is a parameter for controlling the distortion correction in the vertical direction, u represents a variable related to the horizontal direction, and v represents a variable related to the vertical direction.

[0120] According to an embodiment of the present disclosure, taking the sparsity of the distortion characterization matrix K 2m×n as its constraint condition, starting from 0, continuously adjust the values of k1 and k2 of the function, and at the same time calculate the sparsity of the corresponding matrix K 2m×n . When the sparsity reaches convergence (it can be when the sparsity reaches the maximum), the values of k1 and k2 are obtained, and the fitted distortion characterization matrix can be calculated according to Formula 2 and Formula 3.

[0121] According to an embodiment of the present disclosure, the embodiments of the present disclosure combine the characteristics of distorted text and connected components, use sparsity as an effect index for distortion correction, and further improve the effect of distortion correction through the operation and optimization of relevant parameters.

[0122] In summary, the text image correction method using the embodiments of the present disclosure combines the skew correction and distortion correction of the text image, corrects the image in multiple aspects, and more effectively improves the accuracy of subsequent automatic recognition and analysis of the text image.

[0123] Figure 7 A block diagram of a text image correction device according to an embodiment of the present disclosure is schematically shown.

[0124] As Figure 7 shown, the text image correction device 700 includes a straight line detection processing module 710, a skew correction module 720, a distortion fitting processing module 730, and a distortion correction module 740.

[0125] The straight line detection processing module 710 is configured to perform straight line detection processing on the text image to be processed to obtain the skew angle of the text image to be processed.

[0126] The skew correction module 720 is configured to perform skew correction on the text image to be processed according to the skew angle to obtain a preliminary corrected image.

[0127] The distortion fitting processing module 730 is configured to perform distortion fitting processing on the preliminary corrected image to obtain the distortion variable of the preliminary corrected image; and

[0128] The distortion correction module 740 is configured to perform distortion correction on the preliminary corrected image based on the distortion variable.

[0129] According to an embodiment of the present disclosure, before performing straight line detection processing on the text image to be processed to obtain the skew angle of the text image to be processed, the text image correction device 700 further includes a preprocessing module, a dilation and erosion module, an edge detection processing module, and a cropping processing module. Among them, the preprocessing module is configured to perform preprocessing on the text image to be processed to obtain a binary image. The dilation and erosion module is configured to perform dilation and erosion processing on the binary image to obtain a binary connected component image. The edge detection processing module is configured to perform edge detection processing on the binary connected component image to obtain a text contour image. The cropping processing module is configured to perform cropping processing on the text contour image to obtain a plurality of cropped images for performing straight line detection processing on the plurality of cropped images.

[0130] According to an embodiment of the present disclosure, the straight line detection processing module 710 includes a straight line slope obtaining unit and a skew angle determining unit.

[0131] A straight-line slope obtaining unit is configured to perform straight-line detection processing on each of multiple cropped images respectively to obtain multiple straight-line slopes, where each straight-line slope corresponds to one cropped image.

[0132] An inclination angle determination unit is configured to determine an inclination angle based on the multiple straight-line slopes.

[0133] According to an embodiment of the present disclosure, the inclination correction module 720 includes an inclination determination unit, an inclination correction unit, and a preliminary correction image determination unit.

[0134] The inclination determination unit is configured to determine the relationship between the inclination angle and a preset threshold.

[0135] The inclination correction unit is configured to, when the inclination angle is not equal to the preset threshold, perform an operation of performing inclination correction on the text image to be processed according to the inclination angle to obtain a preliminary correction image.

[0136] The preliminary correction image determination unit is configured to, when the inclination angle is equal to the preset threshold, use the binary connected component image as the preliminary correction image.

[0137] According to an embodiment of the present disclosure, the distortion fitting processing module 730 includes an acquisition unit, a generation unit, a sparsity obtaining unit, and a distortion variable obtaining unit.

[0138] The acquisition unit is configured to acquire pixel points of multiple preliminary correction images.

[0139] The generation unit is configured to generate a distortion representation matrix based on the multiple pixel points, where each element in the distortion representation matrix is a distortion result of a pixel point, and the distortion result is determined based on the pixel point values of two adjacent and symmetric pixel points of the pixel point.

[0140] The sparsity obtaining unit is configured to obtain the sparsity of the distortion representation matrix based on the distortion representation matrix.

[0141] The distortion variable obtaining unit is configured to obtain the distortion variable of the preliminary correction image based on the sparsity of the distortion representation matrix.

[0142] According to an embodiment of the present disclosure, the distortion variable obtaining unit includes a construction subunit, an adjustment subunit, a fitting subunit, and an obtaining subunit.

[0143] The construction subunit is configured to construct a fitting correction function.

[0144] The adjustment subunit is configured to adjust the parameter values in the fitting correction function based on the sparsity of the distortion representation matrix until the sparsity of the distortion representation matrix converges.

[0145] A fitting subunit, configured to use the fitting correction function corresponding to when the sparsity of the distortion representation matrix converges as the final fitting correction function.

[0146] An obtaining subunit, configured to obtain the distortion variable of the preliminary corrected image based on the final fitting correction function.

[0147] According to embodiments of the present disclosure, any plurality of modules, sub-modules, units, and sub-units, or at least part of the functions of any of them, can be implemented in one module. Any one or more of the modules, sub-modules, units, and sub-units according to embodiments of the present disclosure can be split into multiple modules for implementation. Any one or more of the modules, sub-modules, units, and sub-units according to embodiments of the present disclosure can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on a substrate, a system in a package, an application specific integrated circuit (ASIC), or can be implemented by any other reasonable way of integrating or packaging circuits in hardware or firmware, or can be implemented in any one of the three implementation manners of software, hardware, and firmware, or in an appropriate combination of any several of them. Alternatively, one or more of the modules, sub-modules, units, and sub-units according to embodiments of the present disclosure can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding functions can be executed.

[0148] For example, any combination of the straight line detection processing module 710, the tilt correction module 720, the distortion fitting processing module 730, and the distortion correction module 740 can be combined and implemented in one module / unit / sub-unit, or any one of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present disclosure, at least one of the straight line detection processing module 710, the tilt correction module 720, the distortion fitting processing module 730, and the distortion correction module 740 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on substrate, a system on package, an application specific integrated circuit (ASIC), or any other reasonable way of integrating or packaging circuits, etc., implemented by hardware or firmware, or implemented in any one of the three implementation manners of software, hardware, and firmware, or in any appropriate combination of several of them. Alternatively, at least one of the straight line detection processing module 710, the tilt correction module 720, the distortion fitting processing module 730, and the distortion correction module 740 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0149] It should be noted that the part of the text image correction device in the embodiments of the present disclosure corresponds to the part of the text image correction method in the embodiments of the present disclosure. For the description of the text image correction device part, please refer to the text image correction method part specifically, and details will not be repeated here.

[0150] Figure 8 A block diagram of a computer system suitable for implementing the method described above according to an embodiment of the present disclosure is schematically shown. Figure 8 The computer system shown is only an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0151] As Figure 8As shown, the computer system 800 according to an embodiment of the present disclosure includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage section 808 into a random access memory (RAM) 803. The processor 801 can include, for example, a general-purpose microprocessor (e.g., CPU), an instruction set processor, and / or a related chipset, and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), and so on. The processor 801 can also include on-board memory for caching purposes. The processor 801 can include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0152] In the RAM 803, various programs and data required for the operation of the system 800 are stored. The processor 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. The processor 801 performs various operations of the method flow according to an embodiment of the present disclosure by executing the program in the ROM 802 and / or the RAM 803. It should be noted that the program can also be stored in one or more memories other than the ROM 802 and the RAM 803. The processor 801 can also perform various operations of the method flow according to an embodiment of the present disclosure by executing the program stored in the one or more memories.

[0153] According to an embodiment of the present disclosure, the system 800 can further include an input / output (I / O) interface 805, and the input / output (I / O) interface 805 is also connected to the bus 804. The system 800 can further include one or more of the following components connected to the I / O interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed so that a computer program read from it can be installed into the storage section 808 as needed.

[0154] According to an embodiment of the present disclosure, the method flow according to the embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes program codes for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the processor 801, the above functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the above-described system, device, apparatus, module, unit, etc. can be implemented by computer program modules.

[0155] The present disclosure also provides a computer-readable storage medium, which may be included in the device / device / system described in the above embodiment; or may exist alone without being assembled into the device / device / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0156] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include but is not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, device, or device.

[0157] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the above-described ROM 802 and / or RAM 803 and / or one or more memories other than ROM 802 and RAM 803.

[0158] An embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program includes program codes for executing the method provided by the embodiment of the present disclosure. When the computer program product runs on an electronic device, the program codes are used to enable the electronic device to implement the text image correction method provided by the embodiment of the present disclosure.

[0159] When the computer program is executed by the processor 801, the above functions defined in the system / apparatus of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the above-described system, apparatus, module, unit, etc. can be implemented by computer program modules.

[0160] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and downloaded and installed through the communication part 809, and / or installed from the removable medium 811. The program code included in the computer program may be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0161] According to embodiments of the present disclosure, the program code for executing the computer program provided by the embodiments of the present disclosure may be written in any combination of one or more programming languages. Specifically, these computing programs may be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages include but are not limited to, for example, Java, C++, Python, the "C" language, or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (for example, by using an Internet service provider to connect through the Internet).

[0162] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur in a different order than that noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and combinations of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or by a combination of dedicated hardware and computer instructions. Those skilled in the art will understand that the features recited in the various embodiments and / or claims of the present disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly recited in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features recited in the various embodiments and / or claims of the present disclosure can be combined and / or combined in various ways. All such combinations and / or combinations fall within the scope of the present disclosure.

[0163] The embodiments of the present disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments have been described separately above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of the present disclosure is defined by the appended claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art can make various substitutions and modifications, and all such substitutions and modifications should fall within the scope of the present disclosure.

Claims

1. A text image correction method, comprising: Performing a straight line detection process on the text image to be processed to obtain the inclination angle of the text image to be processed; Performing inclination correction on the text image to be processed according to the inclination angle to obtain a preliminary correction image; Performing a distortion fitting process on the preliminary correction image to obtain the distortion variable of the preliminary correction image; and Performing distortion correction on the preliminary correction image based on the distortion variable; The performing a distortion fitting process on the preliminary correction image to obtain the distortion variable of the preliminary correction image includes: Obtaining pixel points of multiple preliminary correction images; Generating a distortion representation matrix based on the multiple pixel points, wherein each element in the distortion representation matrix is the distortion result of a pixel point, and the distortion result is determined based on the pixel values of two adjacent and symmetric pixel points of the pixel point; Obtaining the sparsity of the distortion representation matrix based on the distortion result and the number of elements of each element of the distortion representation matrix; Construct a fitting correction function, where the fitting correction function includes: and , is a parameter for controlling the distortion correction in the horizontal direction, is a parameter for controlling the distortion correction in the vertical direction, u represents a variable related to the horizontal direction, v represents a variable related to the vertical direction, and (x(u, v), y(u, v)) represents a pixel point; Adjusting the parameter value in the fitting correction function based on the sparsity of the distortion representation matrix until the sparsity of the distortion representation matrix converges; Taking the fitting correction function corresponding to the convergence of the sparsity of the distortion representation matrix as the final fitting correction function; and Obtaining the distortion variable of the preliminary correction image based on the final fitting correction function.

2. The text image correction method according to claim 1, wherein, Before performing the straight line detection process on the text image to be processed to obtain the inclination angle of the text image to be processed, the method further includes: Performing preprocessing on the text image to be processed to obtain a binary image; Performing dilation and erosion processing on the binary image to obtain a binary connected domain image; Performing edge detection processing on the binary connected domain image to obtain a text contour image; Performing cropping processing on the text contour image to obtain multiple cropped images for performing straight line detection processing on the multiple cropped images.

3. The text image correction method according to claim 2, wherein, The performing a straight line detection process on the text image to be processed to obtain the inclination angle of the text image to be processed includes: Performing straight line detection processing on each of the multiple cropped images respectively to obtain multiple straight line slopes, wherein each straight line slope corresponds to one of the cropped images; and Determining the inclination angle based on the multiple straight line slopes.

4. According to the text image correction method described in claim 2, the performing inclination correction on the text image to be processed according to the inclination angle to obtain a preliminary correction image includes: Determining the relationship between the inclination angle and a preset threshold; In the case where the inclination angle is not equal to the preset threshold, performing the operation of performing inclination correction on the text image to be processed according to the inclination angle to obtain a preliminary correction image; In the case where the inclination angle is equal to the preset threshold, taking the binary connected domain image as the preliminary correction image.

5. A text image correction device, comprising: A straight line detection processing module for performing straight line detection processing on a text image to be processed to obtain the inclination angle of the text image to be processed; An inclination correction module for correcting the inclination of the text image to be processed according to the inclination angle to obtain a preliminary corrected image; A distortion fitting processing module for performing distortion fitting processing on the preliminary corrected image to obtain a distortion variable of the preliminary corrected image; And A distortion correction module for correcting the distortion of the preliminary corrected image based on the distortion variable; Wherein, the distortion fitting processing module includes: An acquisition unit for acquiring pixel points of a plurality of the preliminary corrected images; A generation unit for generating a distortion characterization matrix based on the plurality of pixel points, wherein each element in the distortion characterization matrix is a distortion result of a pixel point, and the distortion result is determined based on the pixel values of two adjacent and symmetric pixel points of the pixel point; A sparsity obtaining unit for obtaining the sparsity of the distortion characterization matrix based on the distortion result and the number of elements of each element of the distortion characterization matrix; A distortion variable obtaining unit for obtaining the distortion variable of the preliminary corrected image based on the sparsity of the distortion characterization matrix; Wherein, the distortion variable obtaining unit includes: A construction subunit for constructing a fitting correction function, where the fitting correction function includes: and , is a parameter for controlling the distortion correction in the horizontal direction, is a parameter for controlling the distortion correction in the vertical direction, u represents a variable related to the horizontal direction, v represents a variable related to the vertical direction, and (x(u, v), y(u, v)) represents a pixel point; An adjustment subunit for adjusting the parameter value in the fitting correction function based on the sparsity of the distortion characterization matrix until the sparsity of the distortion characterization matrix converges; A fitting subunit for using the fitting correction function corresponding to the convergence of the sparsity of the distortion characterization matrix as the final fitting correction function; An obtaining subunit for obtaining the distortion variable of the preliminary corrected image based on the final fitting correction function.

6. A computer system, comprising: One or more processors; A memory for storing one or more programs, Wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the text image correction method according to any one of claims 1 to 4.

7. A computer-readable storage medium having executable instructions stored thereon, and when the instructions are executed by a processor, the processor implements the text image correction method according to any one of claims 1 to 4.

8. A computer program product, comprising a computer program, the computer program comprising computer-executable instructions, and the instructions are used to implement the text image correction method according to any one of claims 1 to 4 when executed.

Images