Text correction method, device, equipment and storage medium

Abstract

The present invention discloses a text correction method, device, equipment and storage medium, the method comprising: obtaining an original text image to be corrected, the original text image containing at least one text line, the text line being text data divided into lines; adding a mask to each of the text lines; in response to detecting that the mask of the first text line is a curved mask, correcting the first text line. The present invention can solve the problem of curved text lines in the original text image, ensure that the corrected text lines are no longer curved, and facilitate subsequent text recognition based on a character segmentation algorithm or a deep learning algorithm based on sequence recognition, thereby improving the accuracy of text recognition, and compared with the technical solution in the related art that can only solve the global tilt or global curvature of the text page based on the overall characteristics of the text page, the technical solution of the present invention can realize text correction in units of text lines, which can further improve the quality of text correction.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a text correction method, device, equipment and storage medium. Background Art

[0002] Computer vision technology can detect and recognize text information in images and convert it into computer text. It has been widely used in various enterprise and consumer application scenarios, and can significantly improve office efficiency.

[0003] In practical applications, due to deformation of the photographed object, such as bent books or wrinkled pages, or because the shooting angle of the image acquisition device and the photographed object is not completely horizontal, the horizontal text lines in the captured text image will become curved. The curved text lines will bring difficulties to the subsequent text recognition based on character segmentation algorithms or deep learning algorithms based on sequence recognition, resulting in an increase in the text recognition error rate. Summary of the invention

[0004] In view of this, the present invention proposes a text correction method, device, equipment and storage medium to solve the above technical problems.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is:

[0006] According to a first aspect of an embodiment of the present invention, there is provided a text correction method, comprising:

[0007] Acquire an original text image to be corrected, wherein the original text image contains at least one text line, and the text line is text data divided into lines;

[0008] Adding a mask to each of the text lines;

[0009] In response to detecting that the mask of the first text line is a curved mask, the first text line is corrected.

[0010] In one embodiment, the correcting the first text line includes:

[0011] determining an outer contour parameter of the first text line based on the mask of the first text line;

[0012] Based on the external contour parameters, the pixels of the first text line are copied to a pre-created target text image to obtain a corrected text line corresponding to the first text line, and the target text image is a blank text image newly created based on the original text image.

[0013] In one embodiment, adding a mask to each of the text lines comprises:

[0014] The original text image is input into a pre-trained text detection model to obtain a masked text image, in which each text line in the masked text image has a mask.

[0015] In one embodiment, the method further comprises detecting that the mask of the first text line is a curved mask based on:

[0016] Detecting coordinates of points on an outer contour of the first text line based on a contour of the mask;

[0017] The mask of the first text line is determined to be a curved mask based on the coordinates of the points on the outer contour.

[0018] In one embodiment, determining the mask of the first text line as a curved mask based on the coordinates of the points on the outer contour includes:

[0019] Acquire a preset number of sampling points from the external contour along the X-axis direction, wherein the sampling points are all upper edge points of the external contour, or the sampling points are all lower edge points of the external contour;

[0020] Calculate the slope of the line between two adjacent sampling points based on the coordinates of the sampling points;

[0021] In response to the derivative of the slope being greater than or equal to a set derivative threshold, it is determined that the mask of the first text line is a curved mask.

[0022] In one embodiment, determining the mask of the first text line as a curved mask based on the coordinates of the points on the outer contour includes:

[0023] Determine a minimum circumscribed rectangle of the outer contour based on coordinates of points on the outer contour;

[0024] Calculate the ratio of the area of ​​the region enclosed by the outer contour to the area of ​​the minimum circumscribed rectangle;

[0025] In response to the ratio being less than or equal to a set ratio threshold, it is determined that the mask of the first text line is a curved mask.

[0026] In one embodiment, the determining of the outer contour parameters of the first text line based on the mask of the first text line includes:

[0027] Determine the distance between each point on the outer contour of the first text line and the origin of the coordinate system based on the contour of the mask;

[0028] Determine a preset positioning point on the outer contour based on the distance;

[0029] The Y-axis coordinate of the preset positioning point is determined as the outer contour parameter of the first text line.

[0030] In one embodiment, the determining of the outer contour parameters of the first text line based on the mask of the first text line includes:

[0031] Detecting a difference between upper and lower boundaries of an outer contour of the first text line based on a contour of the mask;

[0032] determining an average contour height of the outer contour based on the difference;

[0033] The average contour height is determined as an external contour parameter of the first text line.

[0034] In one embodiment, the determining of the outer contour parameters of the first text line based on the mask of the first text line includes:

[0035] Detecting the Y-axis coordinate of each point on the upper boundary of the outer contour of the first text line and the Y-axis coordinate of the highest point based on the contour of the mask;

[0036] The difference between the Y-axis coordinate of each point and the Y-axis coordinate of the highest point is determined as an external contour parameter of the first text line.

[0037] In one embodiment, copying the pixels of the first text line to a pre-created target text image based on the external contour parameters includes:

[0038] determining, based on the outer contour parameters, pixel coordinate adjustment parameters for copying pixels of the first text line into the target text image;

[0039] Based on the pixel coordinate adjustment parameter and the coordinates of the pixels of the first text line, the pixels of the first text line are copied to the target text image.

[0040] In one embodiment, copying the pixels of the first text line to a pre-created target text image based on the external contour parameters includes:

[0041] Based on the external contour parameters, forming a corrected text line contour corresponding to the curved text line in the target text image;

[0042] Based on a predetermined mapping relationship between pixels of a first text line in the original text image and an outline of a corrected text line in the target text image, the pixels of the first text line are copied to an area surrounded by the outline of the corrected text line.

[0043] In one embodiment, the method further comprises:

[0044] Detecting coordinates of points on an outer contour of the first text line based on a contour of the mask;

[0045] The operation of copying the pixels of the first text line to a pre-created target text image based on the external contour parameters is performed with the X-axis coordinate of the leftmost point and the Y-axis coordinate of the highest point of the external contour as base points.

[0046] In one embodiment, the method further comprises:

[0047] Based on a preset text recognition method, obtaining a preset line spacing of the first text line;

[0048] The line spacing of the corrected text line is determined based on the preset line spacing.

[0049] In one embodiment, the external contour parameters include at least one of the following:

[0050] The Y-axis coordinate of a preset positioning point on the outer contour of the first text line;

[0051] an average contour height of an outer contour of the first text line;

[0052] The difference between the Y-axis coordinate of each point on the upper boundary of the outer contour of the first text line and the Y-axis coordinate of the highest point on the boundary.

[0053] In one embodiment, when the external contour parameter includes the Y-axis coordinate of the preset positioning point, copying the pixels of the first text line to a pre-created target text image based on the external contour parameter to obtain a corrected text line corresponding to the first text line includes:

[0054] Performing translation alignment on the corrected text line based on the Y-axis coordinate of the preset positioning point;

[0055] When the external contour parameter includes the average contour height, copying the pixels of the first text line to a pre-created target text image based on the external contour parameter to obtain a corrected text line corresponding to the first text line includes:

[0056] determining a contour height of the corrected text line based on the average contour height;

[0057] When the external contour parameter includes the difference value, copying the pixels of the first text line to a pre-created target text image based on the external contour parameter to obtain a corrected text line corresponding to the first text line includes:

[0058] The Y-axis coordinate of a mapping point corresponding to each point on the contour of the corrected text line is determined based on the difference.

[0059] In one embodiment, the method further comprises:

[0060] In response to detecting that the mask of the second text line is a non-curved mask, the second text line is copied into the target text image.

[0061] According to a second aspect of an embodiment of the present invention, there is provided a text correction device, comprising:

[0062] An image acquisition module, used for acquiring an original text image to be corrected, wherein the original text image contains at least one text line, and the text line is text data divided into lines;

[0063] A mask adding module, used for adding a mask to each of the text lines;

[0064] The text correction module is used for correcting the first text line in response to detecting that the mask of the first text line is a curved mask.

[0065] In one embodiment, the text correction module includes:

[0066] a parameter determination unit, configured to determine an outer contour parameter of the first text line based on the mask of the first text line;

[0067] A text correction unit is used to copy the pixels of the first text line to a pre-created target text image based on the external contour parameters to obtain a corrected text line corresponding to the first text line, and the target text image is a blank text image newly created based on the original text image.

[0068] In one embodiment, the mask adding module is further used to input the original text image into a pre-trained text detection model to obtain a masked text image, wherein each text line in the masked text image has a mask.

[0069] In one embodiment, the apparatus further comprises a bent mask detection module;

[0070] The curved mask detection module comprises:

[0071] a contour coordinate detection unit, configured to detect coordinates of points on an outer contour of the first text line based on a contour of the mask;

[0072] The curved mask detection unit is configured to determine that the mask of the first text line is a curved mask based on the coordinates of the points on the outer contour.

[0073] In one embodiment, the curved mask detection unit is further used for:

[0074] Acquire a preset number of sampling points from the external contour along the X-axis direction, wherein the sampling points are all upper edge points of the external contour, or the sampling points are all lower edge points of the external contour;

[0075] Calculate the slope of the line between two adjacent sampling points based on the coordinates of the sampling points;

[0076] In response to the derivative of the slope being greater than or equal to a set derivative threshold, it is determined that the mask of the first text line is a curved mask.

[0077] In one embodiment, the curved mask detection unit is further used for:

[0078] Determine a minimum circumscribed rectangle of the outer contour based on coordinates of points on the outer contour;

[0079] Calculate the ratio of the area of ​​the region enclosed by the outer contour to the area of ​​the minimum circumscribed rectangle;

[0080] In response to the ratio being less than or equal to a set ratio threshold, it is determined that the mask of the first text line is a curved mask.

[0081] In one embodiment, the parameter determination unit is further configured to:

[0082] Determine the distance between each point on the outer contour of the first text line and the origin of the coordinate system based on the contour of the mask;

[0083] Determine a preset positioning point on the outer contour based on the distance;

[0084] The Y-axis coordinate of the preset positioning point is determined as the outer contour parameter of the first text line.

[0085] In one embodiment, the parameter determination unit is further configured to:

[0086] Detecting a difference between upper and lower boundaries of an outer contour of the first text line based on a contour of the mask;

[0087] determining an average contour height of the outer contour based on the difference;

[0088] The average contour height is determined as an external contour parameter of the first text line.

[0089] In one embodiment, the parameter determination unit is further configured to:

[0090] Detecting the Y-axis coordinate of each point on the upper boundary of the outer contour of the first text line and the Y-axis coordinate of the highest point based on the contour of the mask;

[0091] The difference between the Y-axis coordinate of each point and the Y-axis coordinate of the highest point is determined as an external contour parameter of the first text line.

[0092] In one embodiment, the text correction unit is further used for:

[0093] determining, based on the outer contour parameters, pixel coordinate adjustment parameters for copying pixels of the first text line into the target text image;

[0094] Based on the pixel coordinate adjustment parameter and the coordinates of the pixels of the first text line, the pixels of the first text line are copied to the target text image.

[0095] In one embodiment, the text correction unit is further used for:

[0096] Based on the external contour parameters, forming a corrected text line contour corresponding to the curved text line in the target text image;

[0097] Based on a predetermined mapping relationship between pixels of a first text line in the original text image and an outline of a corrected text line in the target text image, the pixels of the first text line are copied to an area surrounded by the outline of the corrected text line.

[0098] In one embodiment, the text correction module further includes:

[0099] a contour coordinate detection unit, configured to detect coordinates of points on an outer contour of the first text line based on a contour of the mask;

[0100] The text correction unit is also used to perform the operation of copying the pixels of the first text line to a pre-created target text image based on the external contour parameters, taking the X-axis coordinate of the leftmost point and the Y-axis coordinate of the highest point of the external contour as base points.

[0101] In one embodiment, the device further comprises: a line spacing determination module;

[0102] The line spacing determination module comprises:

[0103] A preset line spacing acquisition unit, used to acquire the preset line spacing of the first text line based on a preset text recognition method;

[0104] A text line spacing determination unit is used to determine the line spacing of the corrected text line based on the preset line spacing.

[0105] In one embodiment, the external contour parameters include at least one of the following:

[0106] The Y-axis coordinate of a preset positioning point on the outer contour of the first text line;

[0107] an average contour height of an outer contour of the first text line;

[0108] The difference between the Y-axis coordinate of each point on the upper boundary of the outer contour of the first text line and the Y-axis coordinate of the highest point on the boundary.

[0109] In one embodiment, when the external contour parameter includes the Y-axis coordinate of the preset positioning point, the text correction unit is further used to perform translation alignment on the correction text line based on the Y-axis coordinate of the preset positioning point;

[0110] When the external contour parameter includes the average contour height, the text correction unit is further used to determine the contour height of the corrected text line based on the average contour height;

[0111] When the external contour parameter includes the difference value, the text correction unit is further configured to determine the Y-axis coordinate of a mapping point corresponding to each point on the contour of the corrected text line based on the difference value.

[0112] In one embodiment, the device further comprises:

[0113] The text copying module is used for copying the second text line into the target text image in response to detecting that the mask of the second text line is a non-curved mask.

[0114] According to a third aspect of an embodiment of the present invention, an electronic device is provided, the electronic device comprising:

[0115] a processor, and a memory for storing a computer program;

[0116] Wherein, the processor is configured to implement, when executing the computer program:

[0117] Acquire an original text image to be corrected, wherein the original text image contains at least one text line, and the text line is text data divided into lines;

[0118] Adding a mask to each of the text lines;

[0119] In response to detecting that the mask of the first text line is a curved mask, the first text line is corrected.

[0120] According to a fourth aspect of an embodiment of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, wherein when the program is processed by a processor, the following is implemented:

[0121] Acquire an original text image to be corrected, wherein the original text image contains at least one text line, and the text line is text data divided into lines;

[0122] Adding a mask to each of the text lines;

[0123] In response to detecting that the mask of the first text line is a curved mask, the first text line is corrected.

[0124] Compared with the prior art, the present invention obtains an original text image to be corrected, wherein the original text image contains at least one text line, and the text line is text data divided by lines, and a mask is added to each of the text lines, and then in response to detecting that the mask of the first text line is a curved mask, the first text line is corrected. This can solve the problem of curved text lines in the original text image and ensure that the corrected text lines are no longer curved, which is beneficial for subsequent text recognition based on character segmentation algorithms or deep learning algorithms based on sequence recognition, thereby improving the accuracy of text recognition. Compared with the technical solutions in the related art that can only solve the global tilt or global curvature of the text page based on the overall characteristics of the text page, the technical solution of the present invention can realize text correction in units of text lines, and can further improve the quality of text correction. BRIEF DESCRIPTION OF THE DRAWINGS

[0125] Figure 1A A flowchart of a text correction method according to an exemplary embodiment of the present invention is shown;

[0126] Figure 1B A schematic diagram showing a heat map of each text line in an original text image according to an exemplary embodiment of the present invention is shown;

[0127] Figure 2 A flow chart showing how to correct the first text line according to an exemplary embodiment of the present invention;

[0128] Figure 3 A flowchart showing how to detect that a mask of a first text line is a curved mask according to an exemplary embodiment of the present invention;

[0129] Figure 4A A flowchart showing how to determine that the mask of the first text line is a curved mask based on the coordinates of the points on the outer contour according to an exemplary embodiment of the present invention;

[0130] Figure 4B A schematic diagram showing a method of obtaining a preset number of sampling points on an external contour according to an exemplary embodiment of the present invention is shown;

[0131] Figure 5A A flowchart showing how to determine that the mask of the first text line is a curved mask based on the coordinates of the points on the outer contour according to an exemplary embodiment of the present invention;

[0132] Figure 5BA schematic diagram showing a minimum circumscribed rectangle of an outer contour according to an exemplary embodiment of the present invention is shown;

[0133] Fig. 6A A flow chart showing how to determine the outer contour parameters of the first text line based on the mask of the first text line according to an exemplary embodiment of the present invention;

[0134] Figure 6B A schematic diagram showing the Y-axis coordinates of a preset positioning point according to an exemplary embodiment of the present invention is shown;

[0135] Fig. 7A A flowchart showing how to determine the outer contour parameters of the first text line based on the mask of the first text line according to another exemplary embodiment of the present invention;

[0136] Figure 7B A schematic diagram showing the difference between the upper and lower boundaries of the outer contour of a text line according to an exemplary embodiment of the present invention;

[0137] Fig. 8A A flowchart showing how to determine the outer contour parameters of the first text line based on the mask of the first text line according to another exemplary embodiment of the present invention;

[0138] Figure 8B A schematic diagram showing the difference between the Y-axis coordinate of each point on the upper boundary of the outer contour of a text line and the Y-axis coordinate of the highest point according to an exemplary embodiment of the present invention;

[0139] Fig. 9 A flowchart showing how to copy pixels of the first text line to a pre-created target text image based on the outer contour parameters according to an exemplary embodiment of the present invention;

[0140] Fig.10 A flowchart showing how to copy pixels of the first text line to a pre-created target text image based on the outer contour parameters according to an exemplary embodiment of the present invention;

[0141] Fig.11A A flowchart showing how to copy pixels of the first text line into a pre-created target text image according to an exemplary embodiment of the present invention;

[0142] Fig. 11B A schematic diagram showing copying pixels of the first text line into a target text image according to an exemplary embodiment of the present invention is shown;

[0143] Fig.12 A flow chart showing how to determine the line spacing of the corrected text line according to an exemplary embodiment of the present invention;

[0144] Fig.13 shows a structural block diagram of a text correction device according to an exemplary embodiment of the present invention;

[0145] Fig.14 shows a structural block diagram of a text correction device according to another exemplary embodiment of the present invention;

[0146] Fig.15 A structural block diagram of an electronic device according to an exemplary embodiment of the present invention is shown. DETAILED DESCRIPTION

[0147] The present invention will be described in detail below in conjunction with the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, methodological, or functional changes made by a person skilled in the art based on these embodiments are all within the protection scope of the present invention.

[0148] The terms used in the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The singular forms "a", "the" and "the" used in the present invention and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0149] It should be understood that although the terms first, second, etc. may be used to describe various structures in the present invention, these structures should not be limited to these terms. These terms are only used to distinguish structures of the same type from each other.

[0150] Figure 1A FIG. 2 is a flowchart of a text correction method according to an exemplary embodiment of the present invention. The method of this embodiment can be applied to electronic devices with text processing functions (such as smart phones, tablet computers, laptops or desktop computers, etc.). Figure 1A As shown, the method includes the following steps S101-S104:

[0151] In step S101, an original text image to be corrected is obtained.

[0152] In this embodiment, when the electronic device needs to correct a text line in an original text image, the original text image to be corrected can be acquired.

[0153] The original text image contains at least one text line, and the text line is text data divided into lines.

[0154] In one embodiment, the original text image may include a text image obtained by photographing text in a related document (e.g., a book, a newspaper, etc.) using an image acquisition device such as a camera or a scanner. However, due to deformation of the photographed object, such as a bent book or wrinkled pages, or due to the fact that the image acquisition device and the photographed object are not completely horizontal in angle, the horizontal text lines in the photographed text image may be bent, and the bent text lines may cause difficulties in subsequent text recognition based on a character segmentation algorithm or a deep learning algorithm based on sequence recognition, resulting in an increase in the error rate of text recognition. Therefore, in this embodiment, after obtaining the original text image to be corrected, a subsequent process of correcting the text lines in the original text image may be executed to restore the bent text in the original text image to a horizontal text line.

[0155] In step S102, a mask is added to each of the text lines.

[0156] In this embodiment, after the original text image to be corrected is obtained, a mask may be added to each of the text lines.

[0157] It is worth noting that the type of the above mask can be set based on actual business needs. Exemplarily, the above mask can be a heat map.

[0158] Figure 1B FIG. 4 is a schematic diagram showing a heat map of each text line in an original text image according to an exemplary embodiment of the present invention; Figure 1B As shown, after the original text image 100 is obtained, the heat map of each text line shown in the text image 200 can be obtained.

[0159] The heat map can be used to display the text lines in the original text image in a special highlighted form. For example, after obtaining the original text image to be corrected, the original text image can be input into a pre-trained segmentation-based text detection model (e.g., DBNet model), and then the features of the original text image can be extracted based on the convolutional neural network in the model, and then the heat map of the area where the text line is located can be output, so that the heat map displayed on the text line in the original text image can be obtained.

[0160] In step S103, in response to detecting that the mask of the first text line is a curved mask, the first text line is corrected.

[0161] In this embodiment, after adding a mask to each of the text lines, it can be detected whether the mask of each of the text lines is a curved mask; if it is detected that the mask of one of the text lines (hereinafter referred to as the first text line) is a curved mask, the first text line can be regarded as a curved text line to be corrected, and then the first text line can be corrected.

[0162] For example, after adding a mask to each of the text lines, it is possible to detect whether the outline of the mask of each of the text lines is curved. Then, when it is detected that the outline of the mask of the first text line is curved, the first text line can be regarded as a curved text line to be corrected. It is understandable that the outline of the mask of the text line can reflect the characteristics of the outer contour of the text line, and thus it can be determined whether the text line is a curved text line by detecting the outline of the mask of the text line. In another embodiment, the above-mentioned method of correcting the first text line can refer to the following Figure 2 The above-mentioned method of detecting whether the mask of the first text line is a curved mask can also be referred to as follows Figure 3 The illustrated embodiment will not be described in detail here.

[0163] It can be seen from the above description that the method of this embodiment obtains an original text image to be corrected, wherein the original text image contains at least one text line, and the text line is text data divided by lines, and a mask is added to each of the text lines, and then in response to detecting that the mask of the first text line is a curved mask, the first text line is corrected. This can solve the problem of curved text lines in the original text image and ensure that the corrected text lines are no longer curved, which is beneficial for subsequent text recognition based on character segmentation algorithms or deep learning algorithms based on sequence recognition, thereby improving the accuracy of text recognition. Compared with the technical solutions in the related art that can only solve the global tilt or global curvature of the text page based on the overall characteristics of the text page, the technical solution of the present invention can realize text correction in units of text lines, which can further improve the quality of text correction.

[0164] Figure 2 FIG. 1 is a flowchart showing how to correct the first text line according to an exemplary embodiment of the present invention; based on the above embodiment, this embodiment takes how to correct the first text line as an example for exemplary description. Figure 2 As shown, the correction of the first text line in the above step S103 may include the following steps S201-S202:

[0165] In step S201, external contour parameters of the first text line are determined based on a mask of the first text line.

[0166] In this embodiment, after it is detected that the mask of the first text line is a curved mask, the external contour parameters of the first text line may be determined based on the mask of the first text line.

[0167] The outer contour parameters of the first text line may include at least one of the ordinate of a preset positioning point on the outer contour of the first text line in the original text image (i.e., the Y-axis coordinate of the coordinate system), the average contour height, and the difference between the ordinate of each point on the upper boundary of the contour and the ordinate of the highest point on the contour. Exemplarily, the coordinate system includes an X-axis and a Y-axis perpendicular to each other, wherein: the X-axis direction corresponds to the horizontal direction of the original text image, the Y-axis direction of the coordinate system corresponds to the vertical direction of the original text image, and the origin of the coordinate system may be set to the upper left corner vertex of the original text image.

[0168] In step S202, pixels of the first text line are copied to a pre-created target text image based on the external contour parameters to obtain a corrected text line corresponding to the first text line.

[0169] In this embodiment, after the outer contour parameters of the first text line are determined based on the mask of the first text line, the pixels of the first text line can be copied to a pre-created target text image based on the outer contour parameters to obtain a corrected text line corresponding to the first text line. The target text image is a blank text image newly created based on the original text image.

[0170] For example, when it is detected that the mask of the first text line is a curved mask, a blank text image can be newly created based on the original text image and used as a target text image for mapping the first text line in the original text image. Exemplarily, the size of the blank text image can be the same as the size of the original text image, so as to facilitate copying the pixels of the corrected first text line to the blank text image. Furthermore, the pixels of the first text line can be copied to the target text image based on the above-mentioned external contour parameters.

[0171] In another embodiment, in response to detecting that the mask of the second text line is a non-curved mask, the second text line is copied into the target text image.

[0172] That is to say, when it is detected that the mask of the second text line in the original text image is a non-curved mask, the second text line may not be corrected, but may be directly copied to a corresponding position in the target text image.

[0173] From the above description, it can be seen that this embodiment determines the external contour parameters of the first text line based on the mask of the first text line, and copies the pixels of the first text line to a pre-created target text image based on the external contour parameters to obtain a corrected text line corresponding to the first text line. This can correct the first text line and solve the problem of curved text lines in the original text image, ensuring that the corrected text lines are no longer curved, which is beneficial to subsequent text recognition based on character segmentation algorithms or deep learning algorithms based on sequence recognition, thereby improving the accuracy of text recognition.

[0174] Figure 3 FIG. 4 is a flowchart showing how to detect that a mask of a first text line is a curved mask according to an exemplary embodiment of the present invention; Figure 3 As shown, based on the above embodiment, this embodiment may further include detecting that the mask of the first text line is a curved mask based on the following steps S301-S302:

[0175] In step S301 , the coordinates of points on the outer contour of the first text line are detected based on the contour of the mask.

[0176] In this embodiment, after a mask is added to each of the text lines, the coordinates of points on the outer contour of the first text line may be detected based on the contour of the mask.

[0177] For example, after adding a mask to the first text line, the coordinates of each point in the mask can be obtained based on an image processing method in the related art, and then the coordinates of the points on the outer contour of the first text line can be screened out from the coordinates of these points. Exemplarily, the coordinates of the points on the outer contour of the first text line can be determined based on the coordinates of the points on the boundaries of the mask (e.g., the upper boundary, the lower boundary, the left boundary and / or the right boundary, etc.).

[0178] In step S302, the mask of the first text line is determined to be a curved mask based on the coordinates of the points on the outer contour.

[0179] In this embodiment, after the coordinates of points on the outer contour of the first text line are detected based on the contour of the mask, it can be determined that the mask of the first text line is a curved mask based on the coordinates of the points on the outer contour.

[0180] It can be understood that the coordinates of the points on the outer contour can reflect the characteristics of the mask of the first text line, and thus the mask of the first text line can be determined as a curved mask based on the coordinates of the points on the outer contour. In another embodiment, the above method of determining that the mask of the first text line is a curved mask based on the coordinates of the points on the outer contour can also refer to the following Figure 4A or Figure 5AThe illustrated embodiment will not be described in detail here.

[0181] Figure 4A A schematic diagram is shown of how to determine that the mask of the first text line is a curved mask based on the coordinates of the points on the outer contour according to an exemplary embodiment of the present invention; based on the above embodiment, this embodiment takes how to determine that the mask of the first text line is a curved mask based on the coordinates of the points on the outer contour as an example for exemplary description. Figure 4A As shown, the step S302 of determining that the mask of the first text line is a curved mask based on the coordinates of the points on the outer contour may further include the following steps S401-S403:

[0182] In step S401 , coordinates of a preset number of sampling points are acquired from the coordinates of points on the outer contour along the X-axis direction.

[0183] Figure 4B FIG. 4 shows a schematic diagram of obtaining a preset number of sampling points on an external contour according to an exemplary embodiment of the present invention; Figure 4B As shown, after the coordinates of the points on the outer contour of the text line are detected based on the heat map contour, the coordinates of a preset number of sampling points can be obtained from the coordinates of the points on the outer contour along the X-axis direction of the coordinate system (i.e., Figure 4B The preset number can be set based on actual needs, such as 5, etc., which is not limited in this embodiment.

[0184] In one embodiment, in order to ensure that the coordinates of the preset number of sampling points obtained above can accurately reflect the characteristics of the outer contour of the text line, a preset number of sampling points can be evenly selected from the points on the outer contour (that is, ensure that the selected sampling points are evenly distributed on the outer contour, rather than concentrated in a local area of ​​the outer contour), and the coordinates of the sampling points are obtained.

[0185] In one embodiment, the X-axis direction of the above-mentioned coordinate system corresponds to the horizontal direction of the original text image, the Y-axis direction of the coordinate system corresponds to the vertical direction of the original text image, and the origin of the coordinate system can be the upper left corner vertex of the original text image.

[0186] In step S402, the slope of the line between two adjacent sampling points is calculated based on the coordinates of the sampling points.

[0187] In this embodiment, after the coordinates of a preset number of sampling points are acquired from the coordinates of the points on the outer contour along the X-axis direction, the slope of the line between two adjacent sampling points may be calculated based on the coordinates of the sampling points.

[0188] For example, after obtaining the coordinates of the preset number of sampling points, the slope of the line between each two adjacent sampling points can be determined based on the slope calculation formula in the related art. 1 ,y 1 ) and (x 2 ,y 2 ), the calculation formula of the slope k of the line between the two sampling points can be referred to the following formula (1):

[0189] k=(y 1 -y 2 ) / (x 1 -x 2 ). (1)

[0190] In step S403, in response to the derivative of the slope being greater than or equal to a set derivative threshold, it is determined that the mask of the first text line is a curved mask.

[0191] In this embodiment, after calculating the slope of the line between two adjacent sampling points based on the coordinates of the sampling points, the derivative of the slope can be calculated, and then the derivative can be compared with the set derivative threshold. The derivative of the slope reflects the rate of change of the slope. The larger the derivative, the more curved the text line is. On this basis, when the derivative is greater than or equal to the set derivative threshold, it can be determined that the mask of the first text line is a curved mask.

[0192] As can be seen from the above description, this embodiment obtains the coordinates of a preset number of sampling points from the coordinates of the points on the external contour along the X-axis direction, and calculates the slope of the line between two adjacent sampling points based on the coordinates of the sampling points, and then determines that the mask of the first text line is a curved mask when the derivative of the slope is greater than or equal to the set derivative threshold. This can achieve the determination that the mask of the first text line is a curved mask based on the coordinates of the points on the external contour, and thus can lay an accurate foundation for subsequently correcting the first text line in response to detecting that the mask of the first text line is a curved mask.

[0193] Figure 5A A schematic diagram is shown of how to determine that the mask of the first text line is a curved mask based on the coordinates of the points on the outer contour according to another exemplary embodiment of the present invention; based on the above embodiment, this embodiment takes how to determine that the mask of the first text line is a curved mask based on the coordinates of the points on the outer contour as an example for exemplary description. Figure 5A As shown, the step S302 of determining that the mask of the first text line is a curved mask based on the coordinates of the points on the outer contour may further include the following steps S501-S503:

[0194] In step S501 , a minimum circumscribed rectangle of the external contour is determined based on the coordinates of the points on the external contour.

[0195] Figure 5B FIG. 4 is a schematic diagram showing a minimum circumscribed rectangle of an external contour according to an exemplary embodiment of the present invention; Figure 5B As shown, after the coordinates of the points on the outer contour of the text line are detected based on the heat map contour, the minimum circumscribed rectangle of the outer contour can be determined based on the coordinates of the points on the outer contour.

[0196] Among them, the minimum enclosing rectangle of the external contour refers to a rectangle that can enclose all the coordinates of the external contour and has the smallest area. The method for determining the above-mentioned minimum enclosing rectangle can refer to the technical solution in the relevant technology, and this embodiment does not limit this. For example, the leftmost, top, right, and bottom points of the above-mentioned external contour can be obtained in the original text image; then, a candidate rectangle is determined based on the four points; then, the above-mentioned external contour in the original text image is rotated by a preset angle, and the leftmost, top, right, and bottom points of the external contour are obtained again to determine another candidate rectangle based on the four points; repeat this process until the above-mentioned external contour is rotated to its original position, and then the one with the smallest area among all candidate rectangles can be determined as the minimum enclosing rectangle of the external contour.

[0197] In step S502, the ratio of the area of ​​the region enclosed by the outer contour to the area of ​​the minimum circumscribed rectangle is calculated.

[0198] In this embodiment, after the minimum circumscribed rectangle of the external contour is determined based on the coordinates of the points on the external contour, the area of ​​the region enclosed by the external contour and the area of ​​the minimum circumscribed rectangle determined above can be calculated, and then the ratio of the two can be calculated.

[0199] The calculation method of the area of ​​the region enclosed by the above-mentioned external contour and the area of ​​the minimum circumscribed rectangle can refer to the explanation and description in the relevant technology, and this embodiment does not limit this.

[0200] In step S503, in response to the ratio being less than or equal to a set ratio threshold, it is determined that the mask of the first text line is a curved mask.

[0201] In this embodiment, the smaller the ratio, the more curved the text line is. Further, after calculating the ratio of the area of ​​the area enclosed by the outer contour to the area of ​​the minimum circumscribed rectangle, the ratio can be compared with a set ratio threshold, and then when the ratio is less than or equal to the set ratio threshold, the text line can be determined to be a curved text line.

[0202] The actual value of the above-mentioned set ratio threshold can be set based on actual needs, such as being set to 0.7, etc., and this embodiment does not limit this.

[0203] As can be seen from the above description, this embodiment determines the minimum circumscribed rectangle of the external contour based on the coordinates of the points on the external contour, and calculates the ratio of the area of ​​the area enclosed by the external contour to the area of ​​the minimum circumscribed rectangle, and then determines that the mask of the first text line is a curved mask in response to the ratio being less than or equal to the set ratio threshold. This can achieve the determination that the mask of the first text line is a curved mask based on the coordinates of the points on the external contour, and further can lay an accurate foundation for the subsequent correction of the first text line in response to detecting that the mask of the first text line is a curved mask.

[0204] Fig. 6A FIG. 4 is a flowchart showing how to determine the outer contour parameters of the first text line based on the mask of the first text line according to an exemplary embodiment of the present invention; Fig. 6A As shown, the determination of the outer contour parameters of the first text line based on the mask of the first text line in the above step S201 may include the following steps S601-S603:

[0205] In step S601, the distance between each point on the outer contour of the first text line and the origin of the coordinate system is determined based on the contour of the mask.

[0206] The type of the distance can be set based on actual needs, such as being set to Euclidean distance, which is not limited in this embodiment.

[0207] In step S602, a preset positioning point on the outer contour is determined based on the distance.

[0208] In one embodiment, the preset positioning point may be directly selected as the upper left corner vertex of the outer contour of the text line.

[0209] In yet another embodiment, the preset positioning point may also be determined based on the alignment of the text line.

[0210] For example, if the above-mentioned alignment method is left alignment, the above-mentioned preset positioning point can be selected as the upper left corner vertex of the outer contour of the text line; if the above-mentioned alignment method is right alignment, the above-mentioned preset positioning point can be selected as the upper right corner vertex of the outer contour of the text line; if the above-mentioned alignment method is center alignment, the above-mentioned preset positioning point can be selected as the midpoint of the upper edge of the outer contour of the text line.

[0211] Take the above preset positioning point as the upper left corner vertex of the outer contour of the text line as an example. Figure 6BFIG. 4 is a schematic diagram showing the Y-axis coordinates of a preset positioning point according to an exemplary embodiment of the present invention. Figure 6B As shown, after determining the distance between each point on the outer contour of the curved text line and the origin of the coordinate system, the point on the outer contour with the smallest distance from the origin of the coordinate system can be determined as the preset positioning point (i.e., Figure 6B The upper left corner vertex of each text line outline).

[0212] In step S603, the Y-axis coordinate of the preset positioning point is determined as the outer contour parameter of the first text line.

[0213] In this embodiment, after the preset positioning point on the outer contour is determined based on the distance, the Y-axis coordinate of the preset positioning point (ie, Figure 6B The coordinates in the direction indicated by the arrow in the figure are determined as the external contour parameters of the first text line.

[0214] On this basis, correcting the first text line may include:

[0215] The corrected text line is translated and aligned based on the Y-axis coordinate of the preset positioning point.

[0216] From the above description, it can be seen that this embodiment determines the distance between each point on the outer contour of the first text line and the origin of the coordinate system based on the contour of the mask, and determines the preset positioning point on the outer contour based on the distance, and then determines the Y-axis coordinate of the preset positioning point as the outer contour parameter of the first text line. The subsequent translation alignment of the corrected text line based on the Y-axis coordinate of the preset positioning point can be achieved, and text correction in units of text lines can be achieved, thereby improving the quality of text correction.

[0217] Fig. 7A FIG. 4 is a flowchart showing how to determine the outer contour parameters of the first text line based on the mask of the first text line according to another exemplary embodiment of the present invention; Fig. 7A As shown, the determination of the outer contour parameters of the first text line based on the mask of the first text line in the above step S201 may include the following steps S701-S703:

[0218] In step S701, the difference between the upper and lower boundaries of the outer contour of the first text line is detected based on the contour of the mask.

[0219] Figure 7B FIG. 1 is a schematic diagram showing the difference between the upper and lower boundaries of the outer contour of a text line according to an exemplary embodiment of the present invention; in this embodiment, the mask is still taken as a heat map as an example for explanation. Figure 7BAs shown, in this embodiment, the coordinates of the points on the outer contour of the first text line can be detected based on the heat map contour, and then the difference between the upper boundary and the lower boundary of the outer contour can be determined based on the coordinates (that is, Figure 7B The coordinate difference of the point in the direction indicated by the arrow).

[0220] In step S702, an average contour height of the outer contour is determined based on the difference.

[0221] In this embodiment, after the difference between the upper and lower boundaries of the outer contour of the first text line is detected based on the contour of the mask, the average contour height h_mean of the outer contour may be determined based on the difference.

[0222] In step S703, the average contour height is determined as an external contour parameter of the first text line.

[0223] In this embodiment, after the average contour height of the external contour is determined based on the difference, the average contour height may be determined as the external contour parameter of the first text line.

[0224] On this basis, correcting the first text line may include:

[0225] A contour height of the corrected text line is determined based on the average contour height.

[0226] It can be seen from the above description that this embodiment detects the difference between the upper and lower boundaries of the outer contour of the first text line based on the contour of the mask, and determines the average contour height of the outer contour based on the difference, and then determines the average contour height as the outer contour parameter of the first text line. The subsequent determination of the contour height of the corrected text line based on the average contour height can be achieved, and it can be ensured that the contour height of the corrected text line corresponding to the curved text line formed in the target text image meets the requirements.

[0227] Fig. 8A FIG. 4 is a flowchart showing how to determine the outer contour parameters of the first text line based on the mask of the first text line according to another exemplary embodiment of the present invention; Fig. 7A As shown, the step S201 of determining the outer contour parameters of the first text line based on the mask of the first text line may further include the following steps S801-S802:

[0228] In step S801, the Y-axis coordinate of each point on the upper boundary of the outer contour of the first text line and the Y-axis coordinate of the highest point are detected based on the contour of the mask.

[0229] In this embodiment, the coordinates of points on the outer contour of the first text line can be detected based on the contour of the mask, and then the Y-axis coordinate of the highest point on the upper boundary of the outer contour can be determined based on the coordinates, and then the difference h_diff between the Y-axis coordinate of each point on the upper boundary of the outer contour and the Y-axis coordinate of the highest point of the outer contour can be determined.

[0230] In step S802, the difference between the Y-axis coordinate of each point and the Y-axis coordinate of the highest point is determined as the external contour parameter of the first text line.

[0231] Figure 8B A schematic diagram showing the difference between the Y-axis coordinate of each point on the upper boundary of the outer contour of a text line and the Y-axis coordinate of the highest point according to an exemplary embodiment of the present invention is shown; Figure 8B As shown, after the Y-axis coordinate of each point on the upper boundary of the outer contour of the first text line and the Y-axis coordinate of the highest point are detected based on the contour of the mask, the difference between the Y-axis coordinate of each point and the Y-axis coordinate of the highest point can be determined (i.e., Figure 8B The coordinate difference of the point in the direction indicated by the arrow in the figure can be determined as the external contour parameter of the first text line.

[0232] On this basis, correcting the first text line may include:

[0233] Based on the Y-axis coordinate of each point on the upper boundary of the outer contour and the difference, the Y-axis coordinate of a mapping point on the contour of the corrected text line corresponding to each point is determined.

[0234] For example, when determining the Y-axis coordinate y of the i-th point on the upper boundary of the outer contour i And the corresponding difference h_diff i , the Y-axis coordinate y' of the mapping point corresponding to each point on the contour of the corrected text line can be determined based on the following formula (2): i :

[0235] y' i =y i -h_diff i ; (2)

[0236] As can be seen from the above description, this embodiment detects the Y-axis coordinate of each point on the upper boundary of the outer contour of the first text line and the Y-axis coordinate of the highest point based on the contour of the mask, and determines the difference between the Y-axis coordinate of each point and the Y-axis coordinate of the highest point as the outer contour parameter of the first text line, and then determines the Y-axis coordinate of the mapping point corresponding to each point on the contour of the corrected text line based on the difference, thereby realizing text correction in units of text lines and improving the quality of text correction.

[0237] Fig. 9 FIG. 4 is a flowchart showing how to copy pixels of the first text line to a pre-created target text image based on the external contour parameters according to an exemplary embodiment of the present invention; Fig. 9 As shown, the copying of the pixels of the first text line to the pre-created target text image based on the external contour parameters in the above step S202 may include the following steps S901-S902:

[0238] In step S901 , pixel coordinate adjustment parameters for copying pixels of the first text line into the target text image are determined based on the external contour parameters.

[0239] In this embodiment, after the outer contour parameters of the first text line are determined based on the mask of the first text line, pixel coordinate adjustment parameters for copying pixels of the first text line to the target text image may be determined based on the outer contour parameters.

[0240] For example, the outer contour parameters of the first text line may be determined based on the mask of the first text line, such as at least one of the following (1) to (3):

[0241] (1) the Y-axis coordinate of a preset positioning point on the outer contour of the first text line;

[0242] (2) an average contour height of the outer contour of the first text line;

[0243] (3) The difference between the Y-axis coordinate of each point on the upper boundary of the outer contour of the first text line and the Y-axis coordinate of the highest point on the boundary.

[0244] Furthermore, pixel coordinate adjustment parameters for copying pixels of the first text line to the target text image may be determined based on the external contour parameters.

[0245] For example, when the external contour parameter includes the Y-axis coordinate of the preset positioning point, the parameter for performing translation alignment on the corrected text line in the target text image can be determined based on the parameter;

[0246] When the outer contour parameter includes the average contour height, a parameter for calculating the contour height of the corrected text line in the target text image may be determined based on the parameter;

[0247] When the external contour parameter includes the difference value, the parameter for calculating the Y-axis coordinate of the mapping point corresponding to each point on the contour of the corrected text line in the target text image can be determined based on the parameter.

[0248] In step S902, based on the pixel coordinate adjustment parameter and the coordinates of the pixels of the first text line, the pixels of the first text line are copied to the target text image.

[0249] In this embodiment, after the pixel coordinate adjustment parameters for copying the pixels of the first text line into the target text image are determined based on the external contour parameters, the pixels of the first text line can be copied into the target text image based on the pixel coordinate adjustment parameters and the coordinates of the pixels of the first text line.

[0250] For example, after determining the above-mentioned parameters for translational alignment of the corrected text line in the target text image, the parameters for calculating the contour height of the corrected text line in the target text image, and the parameters for calculating the Y-axis coordinate of the mapping point corresponding to each point on the contour of the corrected text line in the target text image, the pixels of the first text line can be copied to the target text image based on these parameters and the coordinates of the pixels of the first text line.

[0251] Fig.10 FIG. 4 is a flowchart showing how to copy pixels of the first text line to a pre-created target text image based on the external contour parameters according to an exemplary embodiment of the present invention; Fig.10 As shown, the copying of the pixels of the first text line to the pre-created target text image based on the external contour parameters in the above step S202 may include the following steps S1001-S1002:

[0252] In step S1001, a corrected text line contour corresponding to the curved text line is formed in the target text image based on the external contour parameters.

[0253] In this embodiment, after the outer contour parameters of the first text line are determined based on the mask of the first text line, a corrected text line contour corresponding to the curved text line can be formed in the target text image based on the outer contour parameters, wherein the shape of the corrected text line contour is a rectangle.

[0254] For example, after the external contour parameters of the first text line are determined based on the mask of the first text line, the external contour parameters can be corrected first so that the shape of the text line contour corresponding to the corrected external contour parameters is a rectangle, and then based on the corrected external contour parameters, a corrected text line contour is formed in the target text image, thereby ensuring that the shape of the corrected text line contour is also a rectangle.

[0255] In step S1002, based on a predetermined mapping relationship between pixels of a first text line in the original text image and an outline of a corrected text line in the target text image, pixels of the first text line are copied to an area surrounded by the outline of the corrected text line.

[0256] In this embodiment, after a corrected text line contour corresponding to the curved text line is formed in the target text image based on the external contour parameters, the pixels of the first text line can be copied to the area surrounded by the contour of the corrected text line based on a predetermined mapping relationship between the pixels of the first text line in the original text image and the contour of the corrected text line in the target text image.

[0257] Fig.11A FIG. 4 shows a flowchart of how to copy pixels of the first text line to a pre-created target text image according to an exemplary embodiment of the present invention; FIG. Fig.11A As shown, in this embodiment Figure 2 Based on the embodiment shown, the method further includes performing the operation of copying the pixels of the first text line to the pre-created target text image based on the external contour parameters based on the following steps S1101-S1102:

[0258] In step S1101 , the coordinates of points on the outer contour of the first text line are detected based on the contour of the mask.

[0259] In this embodiment, after a mask is added to each of the text lines, the coordinates of points on the outer contour of the first text line may be detected based on the contour of the mask.

[0260] For example, after adding a mask to the first text line, the coordinates of each point in the mask can be obtained based on an image processing method in the related art, and then the coordinates of the points on the outer contour of the first text line can be screened out from the coordinates of these points. Exemplarily, the coordinates of the points on the outer contour of the first text line can be determined based on the coordinates of the points on the boundaries of the mask (e.g., the upper boundary, the lower boundary, the left boundary and / or the right boundary, etc.).

[0261] In step S1102, the operation of copying the pixels of the first text line to a pre-created target text image based on the external contour parameters is performed with the X-axis coordinate of the leftmost point and the Y-axis coordinate of the highest point of the external contour as base points.

[0262] In this embodiment, after the coordinates of the points on the outer contour of the first text line are detected based on the contour of the mask, the operation of copying the pixels of the first text line to a pre-created target text image based on the outer contour parameters described in the above step S202 can be performed with the X-axis coordinate of the leftmost point and the Y-axis coordinate of the highest point of the outer contour as base points.

[0263] Fig. 11B FIG. 4 shows a schematic diagram of copying pixels of the first text line into a target text image according to an exemplary embodiment of the present invention; FIG. Fig. 11B As shown, after the coordinates of the points on the outer contour of the first text line are detected based on the contour of the mask, the leftmost point of the first text line (ie, Fig. 11B The X coordinate of the point with the smallest X coordinate on the left boundary of the outer contour of each curved text line of the original text image 400) and the highest point (ie, Fig. 11B The Y-axis coordinate of the point with the smallest Y coordinate on the upper boundary of the outer contour of each curved text line in the original text image 400 is taken as the base point, and the pixels of the first text line are copied to Fig. 11B The target text image 500 is copied, for example, into the area surrounded by the outline of the corresponding corrected text line in the target text image 500.

[0264] As can be seen from the above description, this embodiment detects the coordinates of points on the outer contour of the first text line based on the contour of the mask, and uses the X-axis coordinate of the leftmost point and the Y-axis coordinate of the highest point of the outer contour as base points to map the pixels of the first text line to the target text image. This can achieve correction of curved text lines by mapping the pixels of the first text line to the target text image based on a predetermined mapping relationship between the original text image and the text lines in the target text image. This can achieve text correction in units of text lines, thereby improving the quality of text correction.

[0265] Fig.12 A flowchart of how to determine the line spacing of the corrected text line according to an exemplary embodiment of the present invention is shown; based on the above embodiment, this embodiment further includes determining the line spacing of the corrected text line based on the following steps S1201-S1202:

[0266] In step S1201, based on a preset text recognition method, a preset line spacing of the first text line is obtained.

[0267] In this embodiment, when the pixels of the first text line are copied to a pre-created target text image based on the external contour parameters, a preset line spacing of the first text line may also be obtained based on a preset text recognition method.

[0268] Among them, the text recognition method of the preset line spacing of the above-mentioned first text line can refer to the technical solutions in the relevant technology. For example, the above-mentioned original text image can be input into a pre-trained convolutional neural network model for line spacing detection, and then the features of the above-mentioned first text line in the original text image can be extracted based on the model, and then the line spacing of the first text line can be output.

[0269] In step S1202, the line spacing of the corrected text line is determined based on the preset line spacing.

[0270] In this embodiment, after the preset line spacing of the first text line is obtained, the line spacing of the corrected text line can be determined based on the preset line spacing.

[0271] For example, when obtaining the preset line spacing h of the first text line 0 Then, if the Y-axis coordinate of the preset positioning point of the contour of the j-th corrected text line is y j1 , and the contour height of the corrected text line is h j , then the Y-axis coordinate y of the preset positioning point of the contour of the j+1th corrected text line is j+1，1 The calculation formula can be shown as follows (3):

[0272] y j+1，1 =y j1 +h j +h 0 ; (3)

[0273] From the above description, it can be seen that this embodiment obtains the preset line spacing of the first text line and determines the line spacing of the corrected text line based on the preset line spacing, so as to accurately determine the line spacing of the corrected text line corresponding to the first text line in the target text image, thereby ensuring that the line spacing of the text lines in the target text image is the same as the line spacing of the text lines in the original text image, meeting the requirements of text correction quality.

[0274] Fig.13 FIG. 1 shows a structural block diagram of a text correction device according to an exemplary embodiment of the present invention; the device of this embodiment can be applied to electronic devices with text processing functions (such as smart phones, tablet computers, laptops or desktop computers, etc.). Fig.13 As shown, the device includes: an image acquisition module 110, a mask adding module 120 and a text correction module 130, wherein:

[0275] An image acquisition module 110 is used to acquire an original text image to be corrected, wherein the original text image includes at least one text line, and the text line is text data divided into lines;

[0276] A mask adding module 120, used for adding a mask to each of the text lines;

[0277] A text correction module 130, configured to correct the first text line in response to detecting that the mask of the first text line is a curved mask;

[0278] As can be seen from the above description, the device of this embodiment obtains an original text image to be corrected, wherein the original text image contains at least one text line, and the text line is text data divided by lines, and a mask is added to each of the text lines, and then in response to detecting that the mask of the first text line is a curved mask, the first text line is corrected. This can solve the problem of curved text lines in the original text image and ensure that the corrected text lines are no longer curved, which is beneficial for subsequent text recognition based on character segmentation algorithms or deep learning algorithms based on sequence recognition, thereby improving the accuracy of text recognition. Compared with the technical solutions in the related art that can only solve the global tilt or global curvature of the text page based on the overall characteristics of the text page, the technical solution of the present invention can realize text correction in units of text lines, which can further improve the quality of text correction.

[0279] Fig.14 FIG. 2 shows a structural block diagram of a text correction device according to another exemplary embodiment of the present invention; the device of this embodiment can be applied to electronic devices with text processing functions (such as smart phones, tablet computers, laptops or desktop computers, etc.). Among them, the image acquisition module 210, the mask adding module 220 and the text correction module 230 are similar to the aforementioned Fig.13 The functions of the image acquisition module 110, the mask adding module 120 and the text correction module 130 in the illustrated embodiment are the same and will not be described in detail herein. Fig.14 As shown, the text correction module 230 includes:

[0280] A parameter determination unit 231, configured to determine an outer contour parameter of the first text line based on the mask of the first text line;

[0281] The text correction unit 232 is used to copy the pixels of the first text line to a pre-created target text image based on the external contour parameters to obtain a corrected text line corresponding to the first text line, and the target text image is a blank text image newly created based on the original text image.

[0282] In one embodiment, the mask adding module 220 is further used to input the original text image into a pre-trained text detection model to obtain a masked text image, in which each text line in the masked text image has a mask.

[0283] In one embodiment, the above apparatus may further include a bent mask detection module 240;

[0284] The bent mask detection module 240 may include:

[0285] A contour coordinate detection unit 241, configured to detect coordinates of points on the outer contour of the first text line based on the contour of the mask;

[0286] The curved mask detection unit 242 is configured to determine that the mask of the first text line is a curved mask based on the coordinates of the points on the outer contour.

[0287] In one embodiment, the curved mask detection unit 242 is further configured to:

[0288] Acquire a preset number of sampling points from the external contour along the X-axis direction, wherein the sampling points are all upper edge points of the external contour, or the sampling points are all lower edge points of the external contour;

[0289] Calculate the slope of the line between two adjacent sampling points based on the coordinates of the sampling points;

[0290] In response to the derivative of the slope being greater than or equal to a set derivative threshold, it is determined that the mask of the first text line is a curved mask.

[0291] In one embodiment, the curved mask detection unit 242 is further configured to:

[0292] Determine a minimum circumscribed rectangle of the outer contour based on coordinates of points on the outer contour;

[0293] Calculate the ratio of the area of ​​the region enclosed by the outer contour to the area of ​​the minimum circumscribed rectangle;

[0294] In response to the ratio being less than or equal to a set ratio threshold, it is determined that the mask of the first text line is a curved mask.

[0295] In one embodiment, the parameter determination unit 231 is further configured to:

[0296] Determine the distance between each point on the outer contour of the first text line and the origin of the coordinate system based on the contour of the mask;

[0297] Determine a preset positioning point on the outer contour based on the distance;

[0298] The Y-axis coordinate of the preset positioning point is determined as the outer contour parameter of the first text line.

[0299] In one embodiment, the parameter determination unit 231 is further configured to:

[0300] Detecting a difference between upper and lower boundaries of an outer contour of the first text line based on a contour of the mask;

[0301] determining an average contour height of the outer contour based on the difference;

[0302] The average contour height is determined as an external contour parameter of the first text line.

[0303] In one embodiment, the parameter determination unit 231 is further configured to:

[0304] Detecting the Y-axis coordinate of each point on the upper boundary of the outer contour of the first text line and the Y-axis coordinate of the highest point based on the contour of the mask;

[0305] The difference between the Y-axis coordinate of each point and the Y-axis coordinate of the highest point is determined as an external contour parameter of the first text line.

[0306] In one embodiment, the text correction unit 232 is further configured to:

[0307] determining, based on the outer contour parameters, pixel coordinate adjustment parameters for copying pixels of the first text line into the target text image;

[0308] Based on the pixel coordinate adjustment parameter and the coordinates of the pixels of the first text line, the pixels of the first text line are copied to the target text image.

[0309] In one embodiment, the text correction unit 232 is further configured to:

[0310] Based on the external contour parameters, forming a corrected text line contour corresponding to the curved text line in the target text image;

[0311] Based on a predetermined mapping relationship between pixels of a first text line in the original text image and an outline of a corrected text line in the target text image, the pixels of the first text line are copied to an area surrounded by the outline of the corrected text line.

[0312] In one embodiment, the text correction module 230 further includes:

[0313] A contour coordinate detection unit 233, configured to detect coordinates of points on the outer contour of the first text line based on the contour of the mask;

[0314] The text correction unit 232 is further configured to perform the operation of copying the pixels of the first text line to a pre-created target text image based on the external contour parameters, taking the X-axis coordinate of the leftmost point and the Y-axis coordinate of the highest point of the external contour as base points.

[0315] In one embodiment, the apparatus may further include: a line spacing determination module 250;

[0316] The line spacing determination module 250 may include:

[0317] A preset line spacing acquisition unit 251, configured to acquire a preset line spacing of the first text line based on a preset text recognition method;

[0318] The text line spacing determination unit 252 is used to determine the line spacing of the corrected text line based on the preset line spacing.

[0319] In one embodiment, the above-mentioned external contour parameters may include at least one of the following:

[0320] The Y-axis coordinate of a preset positioning point on the outer contour of the first text line;

[0321] an average contour height of an outer contour of the first text line;

[0322] The difference between the Y-axis coordinate of each point on the upper boundary of the outer contour of the first text line and the Y-axis coordinate of the highest point on the boundary.

[0323] In one embodiment, when the external contour parameter includes the Y-axis coordinate of the preset positioning point, the text correction unit 232 is further configured to perform translation alignment on the correction text line based on the Y-axis coordinate of the preset positioning point;

[0324] When the external contour parameter includes the average contour height, the text correction unit 232 is further configured to determine the contour height of the corrected text line based on the average contour height;

[0325] When the external contour parameter includes the difference value, the text correction unit 232 is further configured to determine the Y-axis coordinate of a mapping point corresponding to each point on the contour of the corrected text line based on the difference value.

[0326] In one embodiment, the device further comprises:

[0327] The text copying module 260 is configured to copy the second text line into the target text image in response to detecting that the mask of the second text line is a non-curved mask.

[0328] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Ordinary technicians in this field can understand and implement it without paying creative work.

[0329] The text correction device of the present invention can be applied to electronic devices. The device embodiment can be implemented by software, hardware, or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, the processor of the device reads the corresponding computer program instructions in the non-volatile memory into the memory and runs them. From the hardware level, if Fig.15 As shown, it is a hardware structure diagram of the electronic device in which the text correction device of the present invention is located. Fig.15 In addition to the processor, network interface, memory and non-volatile memory shown, the device in the embodiment where the apparatus is located may generally include other hardware, such as a forwarding chip responsible for processing messages, etc.; from a hardware structure perspective, the device may also be a distributed device, which may include multiple interface cards to expand message processing at the hardware level.

[0330] The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the program is processed by a processor, the following is implemented:

[0331] Acquire an original text image to be corrected, wherein the original text image contains at least one text line, and the text line is text data divided into lines;

[0332] Adding a mask to each of the text lines;

[0333] In response to detecting that the mask of the first text line is a curved mask, the first text line is corrected.

[0334] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed by the present invention. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the claims of the present invention.

[0335] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A text correction method, characterized in that: include: Acquire an original text image to be corrected, wherein the original text image contains at least one text line, and the text line is text data divided into lines; Adding a mask to each of the text lines; In response to detecting that the mask of the first text line is a curved mask, correcting the first text line; The correcting the first text line includes: determining an outer contour parameter of the first text line based on the mask of the first text line; Based on the external contour parameters, the pixels of the first text line are copied to a pre-created target text image to obtain a corrected text line corresponding to the first text line, wherein the target text image is a blank text image newly created based on the original text image; The step of copying the pixels of the first text line to a pre-created target text image based on the external contour parameters includes: Based on the external contour parameters, forming a corrected text line contour corresponding to the first text line in the target text image; Based on a predetermined mapping relationship between pixels of a first text line in the original text image and an outline of a corrected text line in the target text image, the pixels of the first text line are copied to an area surrounded by the outline of the corrected text line.

2. The method according to claim 1, characterized in that The adding a mask to each of the text lines comprises: The original text image is input into a pre-trained text detection model to obtain a masked text image, in which each text line in the masked text image has a mask.

3. The method according to claim 1, characterized in that The method further includes detecting that the mask of the first text line is a curved mask based on: Detecting coordinates of points on an outer contour of the first text line based on a contour of the mask; The mask of the first text line is determined to be a curved mask based on the coordinates of the points on the outer contour.

4. The method according to claim 3, characterized in that The step of determining, based on the coordinates of the points on the outer contour, that the mask of the first text line is a curved mask comprises: Acquire a preset number of sampling points from the external contour along the X-axis direction, wherein the sampling points are all upper edge points of the external contour, or the sampling points are all lower edge points of the external contour; Calculate the slope of the line between two adjacent sampling points based on the coordinates of the sampling points; In response to the derivative of the slope being greater than or equal to a set derivative threshold, it is determined that the mask of the first text line is a curved mask.

5. The method according to claim 3, characterized in that: The step of determining, based on the coordinates of the points on the outer contour, that the mask of the first text line is a curved mask comprises: Determine a minimum circumscribed rectangle of the outer contour based on coordinates of points on the outer contour; Calculate the ratio of the area of ​​the region enclosed by the outer contour to the area of ​​the minimum circumscribed rectangle; In response to the ratio being less than or equal to a set ratio threshold, it is determined that the mask of the first text line is a curved mask.

6. The method according to claim 1, characterized in that The determining of the outer contour parameters of the first text line based on the mask of the first text line includes: Determine the distance between each point on the outer contour of the first text line and the origin of the coordinate system based on the contour of the mask; Determine a preset positioning point on the outer contour based on the distance; The Y-axis coordinate of the preset positioning point is determined as the outer contour parameter of the first text line.

7. The method according to claim 1, characterized in that The determining of the outer contour parameters of the first text line based on the mask of the first text line includes: Detecting a difference between upper and lower boundaries of an outer contour of the first text line based on a contour of the mask; determining an average contour height of the outer contour based on the difference; The average contour height is determined as an external contour parameter of the first text line.

8. The method according to claim 1, characterized in that The determining of the outer contour parameters of the first text line based on the mask of the first text line includes: Detecting the Y-axis coordinate of each point on the upper boundary of the outer contour of the first text line and the Y-axis coordinate of the highest point based on the contour of the mask; The difference between the Y-axis coordinate of each point and the Y-axis coordinate of the highest point is determined as an external contour parameter of the first text line.

9. The method according to claim 1, characterized in that: The step of copying the pixels of the first text line to a pre-created target text image based on the external contour parameters includes: determining, based on the outer contour parameters, pixel coordinate adjustment parameters for copying pixels of the first text line into the target text image; Based on the pixel coordinate adjustment parameter and the coordinates of the pixels of the first text line, the pixels of the first text line are copied to the target text image.

10. The method according to claim 1, characterized in that The method further comprises: Detecting coordinates of points on an outer contour of the first text line based on a contour of the mask; The operation of copying the pixels of the first text line to a pre-created target text image based on the external contour parameters is performed with the X-axis coordinate of the leftmost point and the Y-axis coordinate of the highest point of the external contour as base points.

11. The method according to claim 1, characterized in that: The method further comprises: Based on a preset text recognition method, obtaining a preset line spacing of the first text line; The line spacing of the corrected text line is determined based on the preset line spacing.

12. The method according to claim 1, characterized in that The external contour parameters include at least one of the following: The Y-axis coordinate of a preset positioning point on the outer contour of the first text line; an average contour height of an outer contour of the first text line; The difference between the Y-axis coordinate of each point on the upper boundary of the outer contour of the first text line and the Y-axis coordinate of the highest point on the boundary.

13. The method according to claim 12, characterized in that When the external contour parameter includes the Y-axis coordinate of the preset positioning point, the step of copying the pixels of the first text line to a pre-created target text image based on the external contour parameter to obtain a corrected text line corresponding to the first text line includes: Performing translation alignment on the corrected text line based on the Y-axis coordinate of the preset positioning point; When the external contour parameter includes the average contour height, copying the pixels of the first text line to a pre-created target text image based on the external contour parameter to obtain a corrected text line corresponding to the first text line includes: determining a contour height of the corrected text line based on the average contour height; When the external contour parameter includes the difference value, copying the pixels of the first text line to a pre-created target text image based on the external contour parameter to obtain a corrected text line corresponding to the first text line includes: The Y-axis coordinate of a mapping point corresponding to each point on the contour of the corrected text line is determined based on the difference.

14. The method according to claim 1, characterized in that The method further comprises: In response to detecting that the mask of the second text line is a non-curved mask, the second text line is copied into the target text image.

15. A text correction device, characterized in that: include: An image acquisition module, used for acquiring an original text image to be corrected, wherein the original text image contains at least one text line, and the text line is text data divided into lines; A mask adding module, used for adding a mask to each of the text lines; A text correction module, configured to correct the first text line in response to detecting that the mask of the first text line is a curved mask; The text correction module comprises: a parameter determination unit, configured to determine an outer contour parameter of the first text line based on the mask of the first text line; A text correction unit, configured to copy pixels of the first text line to a pre-created target text image based on the external contour parameters to obtain a corrected text line corresponding to the first text line, wherein the target text image is a blank text image newly created based on the original text image; The text correction unit is also used for: Based on the external contour parameters, forming a corrected text line contour corresponding to the first text line in the target text image; Based on a predetermined mapping relationship between pixels of a first text line in the original text image and an outline of a corrected text line in the target text image, the pixels of the first text line are copied to an area surrounded by the outline of the corrected text line.

16. An electronic device, characterized in that: The electronic device comprises: a processor, and a memory for storing a computer program; Wherein, the processor is configured to implement, when executing the computer program: Acquire an original text image to be corrected, wherein the original text image contains at least one text line, and the text line is text data divided into lines; Adding a mask to each of the text lines; In response to detecting that the mask of the first text line is a curved mask, correcting the first text line; The correcting the first text line includes: determining an outer contour parameter of the first text line based on the mask of the first text line; Based on the external contour parameters, the pixels of the first text line are copied to a pre-created target text image to obtain a corrected text line corresponding to the first text line, wherein the target text image is a blank text image newly created based on the original text image; The step of copying the pixels of the first text line to a pre-created target text image based on the external contour parameters includes: Based on the external contour parameters, forming a corrected text line contour corresponding to the first text line in the target text image; Based on a predetermined mapping relationship between pixels of a first text line in the original text image and an outline of a corrected text line in the target text image, the pixels of the first text line are copied to an area surrounded by the outline of the corrected text line.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is processed by the processor, it realizes: Acquire an original text image to be corrected, wherein the original text image contains at least one text line, and the text line is text data divided into lines; Adding a mask to each of the text lines; In response to detecting that the mask of the first text line is a curved mask, correcting the first text line; The correcting the first text line includes: determining an outer contour parameter of the first text line based on the mask of the first text line; Based on the external contour parameters, the pixels of the first text line are copied to a pre-created target text image to obtain a corrected text line corresponding to the first text line, wherein the target text image is a blank text image newly created based on the original text image; The step of copying the pixels of the first text line to a pre-created target text image based on the external contour parameters includes: Based on the external contour parameters, forming a corrected text line contour corresponding to the first text line in the target text image; Based on a predetermined mapping relationship between pixels of a first text line in the original text image and an outline of a corrected text line in the target text image, the pixels of the first text line are copied to an area surrounded by the outline of the corrected text line.

Images