A method and device for recognizing text of a distorted document and a storage medium

Abstract

Embodiments of the present application provide a method and device for recognizing text in a distorted document and a storage medium. The method comprises: segmenting text lines in a text image to be recognized according to a preset segmentation model to obtain a mask image of the text lines; performing distortion analysis on the text lines according to the mask image to determine a mapping relationship model between the text lines and horizontal text lines; correcting the text lines in a curved distribution according to the mapping relationship model to obtain text lines in a straight line distribution after correction; and recognizing text in the text lines after correction.

Description

Technical Field

[0001] This invention relates to the field of cloud-intelligence integration, and in particular to a method, apparatus and storage medium for text recognition of distorted documents. Background Technology

[0002] Image recognition is of great significance in the field of intelligent recognition. With the advancement of technology and social development, the demand for automatic text recognition in images is also increasing rapidly. During the process of acquiring images through scanning, photography, and other means, image distortion can occur due to factors such as the acquisition angle and the physical characteristics of the document source itself, posing challenges to the recognition process.

[0003] Existing technical solutions require splitting, correcting, and merging multiple regions of distorted document images, resulting in incomplete matching of edge pixels. Furthermore, this approach requires straightening each line of text in the document during recognition, consuming significant computational resources and exhibiting low performance. Summary of the Invention

[0004] This invention provides a method, apparatus, and storage medium for text recognition in distorted documents.

[0005] The technical solution of this invention is implemented as follows:

[0006] This invention provides a method for text recognition in distorted documents, comprising:

[0007] The text lines in the text image to be identified are segmented according to the preset segmentation model, and the mask image of the text lines is obtained.

[0008] Based on the mask image, a distortion analysis is performed on the text lines to determine the mapping relationship model between the text lines and the horizontal text lines;

[0009] Based on the mapping relationship model, the text lines that are distributed in a curve are corrected to obtain text lines that are distributed in a straight line after correction;

[0010] Identify the text in the corrected text line.

[0011] In the above scheme, the step of performing distortion analysis on the text lines based on the mask image to determine the mapping relationship model between the text lines and the horizontal text lines includes:

[0012] Based on the mask image, a distortion analysis is performed on the text line to determine the inflection point of the curve corresponding to the text line and the pixel value of the pixel in the neighborhood of the inflection point.

[0013] The edge points of the polygon are determined based on the pixel values ​​of the mask image;

[0014] Map the edge points on the curve to the positions corresponding to the straight line to determine the mapping relationship model between the text line and the horizontal text line.

[0015] In the above scheme, determining the edge points of the polygon based on the pixel values ​​of the mask image includes:

[0016] The pixel values ​​of the pixels in the neighborhood of the inflection point are divided into a first set and a second set; wherein the pixel value of the pixels in the first set is greater than a first preset value, the pixel value of the pixels in the second set is less than a second preset value, and the first preset value is greater than the second preset value;

[0017] The pixel located at the boundary position between the pixels in the second set and the pixels in the first set is determined as the edge point.

[0018] In the above scheme, determining the pixel located at the boundary position between the pixels in the second set and the pixels in the first set as the edge point includes:

[0019] Calculate the sum of the pixel values ​​of the pixels in the first set to obtain the first sum value;

[0020] And calculate the sum of the pixel values ​​of the pixels in the second set to obtain the second sum value;

[0021] Based on the magnitude of the first sum and the second sum, the edge point is determined to be the upper edge inflection point or the lower edge inflection point of the text line.

[0022] In the above scheme, determining whether the edge point is the upper or lower edge inflection point of the text line based on the magnitude of the first sum and the second sum includes:

[0023] If the text in the text line is dark and the background of the text is light, when the first sum is greater than the second sum, the edge point is determined to be the upper edge inflection point;

[0024] or,

[0025] If the text in the text line is dark and the background of the text is light, then when the first sum is less than the second sum, the edge point is determined to be the lower edge inflection point.

[0026] In the above scheme, determining whether the edge point is the upper or lower edge inflection point of the text line based on the magnitude of the first sum and the second sum includes:

[0027] If the text in the text line is light-colored and the background of the text is black, when the first sum is greater than the second sum, the edge point is determined to be the lower edge inflection point;

[0028] or,

[0029] If the text in the text line is light-colored and the background of the text is black, then when the first sum is less than the second sum, the edge point is determined to be the upper edge inflection point.

[0030] In the above scheme, the step of correcting the text lines that are distributed in a curve according to the mapping relationship model includes:

[0031] Based on the mapping relationship model, the pixel mapping points corresponding to the edge inflection points of the polygon are determined. The pixel mapping points represent the corresponding points on the straight line of the text line in the text image to be identified after passing through the mapping relationship model.

[0032] Based on the pixel mapping points, determine the correction reference points of the document to be identified;

[0033] The document to be identified is corrected using a thin-plate spline function based on the correction reference point.

[0034] In the above scheme, the step of correcting the document to be identified using a thin-plate spline function based on the correction reference point includes:

[0035] Based on the edge inflection point and the correction reference point, the interpolation function for the edge inflection point and the correction reference point is determined by the thin plate spline function;

[0036] The document to be recognized is corrected according to the interpolation function to obtain the text lines that are linearly distributed after correction.

[0037] In the above scheme, the identification of text in the corrected text line includes:

[0038] Based on the corrected text lines, establish a set of text lines;

[0039] Determine the text line image matrix based on the set of text lines;

[0040] The text line image matrix is ​​identified by a text recognition model to obtain the corrected text in the text line.

[0041] In the above scheme, the preset segmentation model includes: DBnet image segmentation network;

[0042] The method further includes: training the DBnet image segmentation network, and obtaining the segmentation model based on the trained DBnet image segmentation network.

[0043] This invention also provides a text recognition device for distorted documents, the device comprising: a segmentation module, a determination module, a correction module, and a recognition module; wherein,

[0044] The segmentation module is used to segment the text lines in the text image to be recognized according to a preset segmentation model, and obtain the mask image of the text lines;

[0045] The determination module is used to perform distortion analysis on the text lines based on the mask image and determine the mapping relationship model between the text lines and the horizontal text lines;

[0046] The correction module is used to correct the text lines that are distributed in a curve according to the mapping relationship model, so as to obtain text lines that are distributed in a straight line after correction.

[0047] The recognition module is used to identify the text in the corrected text lines.

[0048] In the above scheme, the determining module is specifically used for:

[0049] Based on the mask image, a distortion analysis is performed on the text line to determine the inflection point of the curve corresponding to the text line and the pixel value of the pixel in the neighborhood of the inflection point.

[0050] The edge points of the polygon are determined based on the pixel values ​​of the mask image;

[0051] Map the edge points on the curve to the positions corresponding to the straight line to determine the mapping relationship model between the text line and the horizontal text line.

[0052] In the above scheme, the correction module is specifically used for:

[0053] The pixel values ​​of the pixels in the neighborhood of the inflection point are divided into a first set and a second set; wherein the pixel value of the pixels in the first set is greater than a first preset value, the pixel value of the pixels in the second set is less than a second preset value, and the first preset value is greater than the second preset value;

[0054] The pixel located at the boundary position between the pixels in the second set and the pixels in the first set is determined as the edge point.

[0055] In the above scheme, the correction module is specifically used for:

[0056] Calculate the sum of the pixel values ​​of the pixels in the first set to obtain the first sum value;

[0057] And calculate the sum of the pixel values ​​of the pixels in the second set to obtain the second sum value;

[0058] Based on the magnitude of the first sum and the second sum, the edge point is determined to be the upper edge inflection point or the lower edge inflection point of the text line.

[0059] In the above scheme, the correction module is specifically used for:

[0060] If the text in the text line is dark and the background of the text is light, when the first sum is greater than the second sum, the edge point is determined to be the upper edge inflection point;

[0061] or,

[0062] If the text in the text line is dark and the background of the text is light, then when the first sum is less than the second sum, the edge point is determined to be the lower edge inflection point.

[0063] In the above scheme, the correction module is specifically used for:

[0064] If the text in the text line is light-colored and the background of the text is black, when the first sum is greater than the second sum, the edge point is determined to be the lower edge inflection point;

[0065] or,

[0066] If the text in the text line is light-colored and the background of the text is black, then when the first sum is less than the second sum, the edge point is determined to be the upper edge inflection point.

[0067] In the above scheme, the correction module is further used for:

[0068] Based on the mapping relationship model, the pixel mapping points corresponding to the edge inflection points of the polygon are determined. The pixel mapping points represent the corresponding points on the straight line of the text line in the text image to be identified after passing through the mapping relationship model.

[0069] Based on the pixel mapping points, determine the correction reference points of the document to be identified;

[0070] The document to be identified is corrected using a thin-plate spline function based on the correction reference point.

[0071] In the above scheme, the correction module is specifically used for:

[0072] Based on the edge inflection point and the correction reference point, the interpolation function for the edge inflection point and the correction reference point is determined by the thin plate spline function;

[0073] The document to be recognized is corrected according to the interpolation function to obtain the text lines that are linearly distributed after correction.

[0074] In the above scheme, the identification module is specifically used for:

[0075] Based on the corrected text lines, establish a set of text lines;

[0076] Determine the text line image matrix based on the set of text lines;

[0077] The text line image matrix is ​​identified by a text recognition model to obtain the corrected text in the text line.

[0078] In the above scheme, the preset segmentation model includes: DBnet image segmentation network;

[0079] The apparatus further includes a preprocessing unit; the preprocessing unit is used to train the DBnet image segmentation network and obtain the segmentation model based on the trained DBnet image segmentation network.

[0080] This invention also provides a text recognition device for distorted documents, the device comprising: a processor and a memory for storing a computer program capable of running on the processor;

[0081] Wherein, when the processor is used to run the computer program, it executes the steps of the text recognition method for distorted documents according to the embodiments of the present invention.

[0082] This invention also provides a computer storage medium, characterized in that the computer storage medium stores computer-executable instructions; after being executed by a processor, the computer-executable instructions are able to implement the steps of the text recognition method for distorted documents according to this invention.

[0083] In this embodiment of the invention, during the correction process, the inflection points of the curves corresponding to the text lines are determined based on the text line mask image. Then, the mapping relationship between the edge points on the curves and their corresponding positions on the straight lines is determined based on the inflection point information to distort the document for correction. This eliminates the need to split the image into multiple polygons, thus achieving correction for all text lines. This results in less overall time consumption, resource conservation, and more complete character correction, effectively improving recognition accuracy. Furthermore, this method allows for batch recognition of corrected text lines, reducing recognition time and improving performance. Attached Figure Description

[0084] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0085] Figure 1 This is a flowchart illustrating the text recognition method for distorted documents provided by the present invention;

[0086] Figure 2 This is a flowchart illustrating a method for segmenting text lines using a segmentation model provided in this invention.

[0087] Figure 3 This is a schematic diagram illustrating the result of an image annotation method provided in an embodiment of the present invention;

[0088] Figure 4 This is a schematic diagram of the results of text line distortion analysis provided in an embodiment of the present invention;

[0089] Figure 5 This is a schematic diagram illustrating the implementation process of a polygon edge point determination method provided in an embodiment of the present invention.

[0090] Figure 6 This is a schematic diagram of a method for determining edge points provided in an embodiment of the present invention;

[0091] Figure 7 This is a schematic diagram of the pixel mapping relationship of a text line provided in an embodiment of the present invention;

[0092] Figure 8 This is a schematic diagram of a document correction result provided in an embodiment of the present invention;

[0093] Figure 9 This is a schematic diagram of an image matrix construction result provided in an embodiment of the present invention;

[0094] Figure 10 A schematic diagram of the structure of a text recognition device for distorted documents provided in an embodiment of the present invention;

[0095] Figure 11 This is a schematic diagram of another text recognition device for distorted documents provided in an embodiment of the present invention. Detailed Implementation

[0096] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on the present invention. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0097] Unless otherwise specified, the embodiments and features described herein can be combined arbitrarily. The steps illustrated in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0098] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to limit the invention.

[0099] This invention provides a method for text recognition in distorted documents. Figure 1 A flowchart illustrating the text recognition method for distorted documents provided by this invention; as shown. Figure 1 As shown, the method includes:

[0100] Step S101: Segment the text lines in the text image to be recognized according to the preset segmentation model to obtain the mask image of the text lines;

[0101] Step S102: Based on the mask image, perform distortion analysis on the text lines to determine the mapping relationship model between the text lines and the horizontal text lines;

[0102] Step S103: According to the mapping relationship model, the text lines that are distributed in a curve are corrected to obtain text lines that are distributed in a straight line after correction;

[0103] Step S104: Identify the text in the corrected text line.

[0104] In step S101, the mask image of the text line is a two-dimensional matrix of a binary image.

[0105] Specifically, such as Figure 2 As shown, Figure 2 This is a flowchart illustrating a text line segmentation method provided in this application. The method uses a preset segmentation model to predict text line regions in the text image to be identified. The prediction results are then binarized, and a mask of the text line regions after binarization is output, i.e., the mask of the text lines. Here, binarization involves setting the grayscale value of pixels in the image to 0 or 255, effectively giving the entire image a distinct black and white effect. The text line mask image obtained after segmentation by the segmentation model is a two-dimensional matrix with two values, 0 and 255; where 0 represents the background and 255 represents the text region. Figure 2 The black portion represents the background with a grayscale value of 0, while the white portion represents the text area with a grayscale value of 255. The mask image based on the text lines is a two-dimensional matrix of a binary image, where the pixel values ​​are equal to the grayscale values.

[0106] For example, this two-dimensional matrix includes only "0" and "1" elements. Furthermore, in a two-dimensional matrix composed of only "0" and "1" elements, "0" can represent pixels not covered by text, and "1" can represent pixels covered by text; or, "1" can represent pixels not covered by text, and "0" can represent pixels covered by text. If the text is black, and "0" can represent pixels not covered by text, and "1" can represent pixels covered by text, then pixels with a value of "0" or a difference from "0" equal to a preset value (e.g., 1, 2, or 3) can be considered as the pixels containing the text, and can be uniformly converted to "1" in the mask image. Of course, this is just an example; there are various specific implementation methods.

[0107] Furthermore, the preset segmentation model includes: DBnet image segmentation network;

[0108] Before step S101, the method further includes: training the DBnet image segmentation network and obtaining the segmentation model based on the trained DBnet image segmentation network.

[0109] The DBnet image segmentation network is a real-time scene text detection network that uses differentiable binarization. It is used to convert the probability map generated by the segmentation method into text regions. Through the optimization of the DBnet segmentation network, the binarization threshold can be adaptively set to completely distinguish between foreground and background pixels, that is, to distinguish between text lines and background pixels.

[0110] Specifically, the process of training the DBnet image segmentation network also includes document image preparation, i.e., preparation of the training set.

[0111] In one embodiment, the document image preparation may include: image acquisition and image annotation. During the image acquisition stage, images can be acquired from books, contracts, official documents, etc.; during the acquisition process, distorted document images are obtained by altering the physical deformation of the paper, camera position, and lighting conditions, and these distorted document images are used as training sample data; during the image annotation process, text lines can be surrounded using line segments, marker points, etc., to annotate the image data, such as... Figure 3 As shown.

[0112] Specifically, in the text line segmentation training process, the images from the document image preparation stage are used as the training set for the segmentation model. A binary classification cross-entropy function is used as the loss function, and the DBnet segmentation network is trained through multiple iterations to obtain the preset segmentation model. DBnet is then used as the segmentation network to predict whether each pixel in the input image is a text line region.

[0113] In this embodiment, during the process of recognizing the text line region in the text image to be recognized by the DBnet segmentation network, adaptive binarization is performed on each pixel. The binarization threshold is learned by the network. By incorporating the binarization step into the network for training, the final output image becomes very robust to the threshold.

[0114] In step S102, the distortion analysis of the text lines involves performing dilation and erosion smoothing on the mask image of the text lines identified by the segmentation model, based on which the mask is then used. The approxPolyDP function of OpenCV is then used to perform polygon fitting on the identified text line region to obtain the inflection points of the curve corresponding to the text lines and the pixel values ​​of the pixels in the neighborhood of the inflection points. The result is shown in the figure below. Figure 4 As shown.

[0115] The neighborhood of the inflection point refers to any range of pixels centered on the inflection point. Here, the range of the neighborhood is not limited.

[0116] Furthermore, step S102 also includes: performing distortion analysis on the text line based on the mask image to determine the inflection point of the curve corresponding to the text line and the pixel value of the pixel point in the neighborhood of the inflection point; determining the edge points of the polygon based on the pixel value of the mask image; mapping the edge points on the curve to the positions corresponding to the straight line to determine the mapping relationship model between the text line and the horizontal text line.

[0117] Specifically, the edge points on the curve are mapped to positions corresponding to the straight line based on the length of the broken line between the edge points, and a mapping relationship model is established based on the points on the straight line corresponding to the edge points on the curve; here, the length of the broken line between the edge points is approximately the length of the horizontal straight line of the flat document.

[0118] Based on the established mapping relationship model, the edge points of the curves corresponding to the segmented text lines are mapped, and then the text lines that are distributed in curves are corrected to obtain text lines that are distributed in straight lines after correction.

[0119] In some embodiments, such as Figure 5 As shown, step S102 includes:

[0120] Step S1021: Divide the pixel values ​​of the pixels in the neighborhood of the inflection point into a first set and a second set; wherein the pixel values ​​of the pixels in the first set are greater than a first preset value, the pixel values ​​of the pixels in the second set are less than a second preset value, and the first preset value is greater than the second preset value.

[0121] The pixel located at the boundary position between the pixels in the second set and the pixels in the first set is determined as the edge point.

[0122] An inflection point, also known as a curve reversal point, is a point where the curve changes direction from upward to downward. For example, a pixel whose left side is an ascending curve and its right side is a descending curve can be considered an inflection point in this invention. It can also be determined by the slope of the tangent line corresponding to the curve where the inflection point is located; in other words, the inflection point is the point where the tangent line crosses the curve (i.e., the boundary between the concave and convex arcs of a continuous curve). For example, a tangent line with two different slopes, or a tangent line with a slope of 0 or infinity, is an inflection point.

[0123] In one embodiment, the pixel values ​​of the pixels in the neighborhood of the inflection point are divided into a first set and a second set; the first preset value and the second preset value can be customized according to the text line mask image obtained after segmentation and binarization of the image by the segmentation model. In some embodiments, the pixels in the first set can be pixels of the text region, and the pixels in the second set can be pixels of the background region; when the text is white and the background of the text is black, the first preset value can be any pixel value less than 255, and the second preset value can be any pixel value greater than 0; in another embodiment, the pixels in the first set can be pixels of the background region, and the pixels in the second set can be pixels of the text region; when the text is black and the background of the text is white, the first preset value can be any pixel value greater than 0, and the second preset value can be any pixel value less than 255; the pixels at the boundary position between the pixels in the second set and the first set are the edge points of the curve corresponding to the text line.

[0124] The pixels in the first set can be pixels of the text area, and the pixels in the second set can be pixels of the background area; or the pixels in the first set can also be pixels of the background area, and the pixels in the second set can also be pixels of the text area; no specific limitation is made here.

[0125] Step S1022: Calculate the sum of the pixel values ​​of the pixels in the first set to obtain the first sum value;

[0126] And calculate the sum of the pixel values ​​of the pixels in the second set to obtain the second sum value;

[0127] Based on the magnitude of the first sum and the second sum, the edge point is determined to be the upper edge inflection point or the lower edge inflection point of the text line.

[0128] In one embodiment, the edge point is determined to be either an upper edge point or a lower edge point by comparing the sum of the pixel values ​​of the pixels in the first set and the second set. The state of the edge point represents the distortion state of the text line; when the edge point is an upper edge point, the text area is below the edge point; when the edge point is a lower edge point, the text area is above the edge point.

[0129] Step S10231: If the text in the text line is dark and the background of the text is light, when the first sum is greater than the second sum, the edge point is determined to be the upper edge inflection point;

[0130] or,

[0131] If the text in the text line is dark and the background of the text is light, then when the first sum is less than the second sum, the edge point is determined to be the lower edge inflection point.

[0132] Step S10232: If the text in the text line is light-colored and the background of the text is black, when the first sum is greater than the second sum, the edge point is determined to be the lower edge inflection point;

[0133] or,

[0134] If the text in the text line is light-colored and the background of the text is black, then when the first sum is less than the second sum, the edge point is determined to be the upper edge inflection point.

[0135] Specifically, the upper edge inflection point is the inflection point of the curve corresponding to the upper edge of the text line; the lower edge inflection point is the inflection point of the curve corresponding to the lower edge of the text line.

[0136] In this embodiment, the text in the text line is light-colored and the background of the text is black, such as... Figure 6 As shown, Figure 6 This invention provides a method for determining edge points: selecting the pixel values ​​of 8 pixels in the neighborhood of the curve inflection point corresponding to the text line, and determining whether the polygon edge point is an upper edge point or a lower edge point by statistically analyzing the changes in the pixel values ​​of the 8 pixels in the neighborhood of the polygon inflection point, and determining the distortion state of the text line based on the upper edge polyline of the polygon.

[0137] Specifically, taking the top edge point as an example, the pixel values ​​p0, p1, p2, p3, p4, p5, p6, and p7 of the eight pixels in the neighborhood of the inflection point are divided into two sets: p0, p1, p2, p3, and p4 are background pixels, and p0, p1, p2, p3, and p4 are grouped into the first set; p6, p7, and p8 are text line pixels, and p6, p7, and p8 are grouped into the second set. In an image where the text is white and the background is black, if sum(p6, p7, p8) is greater than sum(p0, p1, p2, p3, p4), then p is a top edge sampling point. If sum(p0, p1, p2) is greater than sum(p3, p4, p5, p6, p7), then p is a bottom edge sampling point.

[0138] In some embodiments, step 103 includes:

[0139] Based on the mapping relationship model, the pixel mapping points corresponding to the edge inflection points of the polygon are determined. The pixel mapping points represent the corresponding points of the text lines in the text image to be recognized on the straight line through the mapping relationship model. Based on the pixel mapping points, the correction reference points of the document to be recognized are determined. Based on the correction reference points, the document to be recognized is corrected using a thin plate spline function.

[0140] In this embodiment, the pixel mapping relationship of the text lines is as follows: Figure 7 As shown, based on the established pixel mapping relationship model, the edge inflection points of the determined polygons are mapped onto straight lines, thus determining the pixel mapping points of the edge inflection points. The sampling point set a = {a0, a1, a2, a3, a4, ..., an} is mapped onto straight lines A = {A0, A1, A2, A3, A4, ..., An}, where a0a1 = A0A1, a1a2 = A1A2, ..., an-1an = An-1An.

[0141] Here, the inflection point is mapped to the corresponding pixel on the straight line, which is the correction reference point of the document to be recognized. Based on the correction reference point, the entire document to be recognized is corrected using the Thin Plate Spline (TPS) function, while recognizing multiple lines of text simultaneously.

[0142] Further, the step of correcting the document to be recognized using a thin-plate spline function based on the correction reference point includes: determining an interpolation function for the edge inflection point and the correction reference point using the thin-plate spline function; and correcting the document to be recognized using the interpolation function to obtain text lines that are linearly distributed after correction.

[0143] Specifically, TPS is often used to perform non-rigid deformation of shapes to find the smooth surface with the minimum distortion through all reference points. Based on the finite set of points A of the original shape and the corresponding target point set B after deformation, a TPS is fitted to obtain the interpolation function. In this way, for points not in the point set, we can obtain the target point through interpolation, thereby completing the deformation of the entire surface.

[0144] In this embodiment, the document correction result is as follows: Figure 8 As shown, the reference point pairs of the polygon's edge inflection points and their mapping points on the straight line are obtained by using a thin plate spline function to obtain the interpolation function of the reference point pairs, and then the interpolation function of each pixel transformation is fitted; for points not in the point set, the target position is calculated by using the interpolation function, and then all pixels can be transformed by the interpolation function to correct the entire document to be recognized, resulting in text lines that are straight-lined after correction, thus obtaining a flat document.

[0145] It should be noted that Thin Plate Spline (TPS) functions can simulate more deformations. Based on reference point pairs of edge inflection points and their mapping points on a straight line, the entire document to be recognized is corrected using TPS. TPS performs overall correction of distorted document images without needing to split the image into multiple polygons, thus achieving correction of the entire document, resulting in less overall time and resource savings. Furthermore, unlike conventional rigid deformation methods that rely solely on atomic operations such as translation, rotation, scaling, inversion, and shearing deformation for distortion correction, TPS utilizes the flexible transformation characteristics of thin plate spline functions to correct distorted documents, resulting in more complete character correction and more effectively improving recognition accuracy.

[0146] In some embodiments, step 104 includes:

[0147] Based on the corrected text lines, a text line set is established; based on the text line set, a text line image matrix is ​​determined; the text line image matrix is ​​recognized by a character recognition model to obtain the characters in the corrected text lines.

[0148] In this embodiment, a horizontal text line set is established based on the corrected text lines, and a text line image matrix is ​​constructed. The entire matrix is ​​then fed into the ResNeXt-Blstm text recognition model for batch recognition to obtain a text information matrix. Using the technical solution proposed in this proposal, the total time consumption is reduced, resources are saved, and the recognition accuracy is effectively improved.

[0149] Specifically, a horizontal set of text lines, L = {L1, L2, L3, ..., Ln}, is established based on the corrected text lines, where L represents the coordinates of the text lines, and L1 can be [(x0,y0), (x1,y1), (x2,y2), (x3,y3), (x4,y4)]. Here, (x0,y0) corresponds to the coordinates of the top-left pixel of the text line region, (x1,y1) corresponds to the coordinates of the top-right pixel of the text line region, (x2,y2) corresponds to the coordinates of the bottom-right pixel of the text line region, (x3,y3) corresponds to the coordinates of the top-left pixel of the text line region, and (x4,y4) corresponds to the coordinates of the bottom-left pixel of the text line region. The image region of each text line can be extracted using the coordinates of the text lines.

[0150] Furthermore, based on the established horizontal text row set, a text image matrix is ​​constructed.

[0151] Specifically, first, the height of each text line is fixed at 32, and the grayscale image of the text is scaled proportionally to obtain the maximum width Wmax, creating a base image. The base image has dimensions of 1 * Wmax * 32, with all pixel values ​​set to 0. Then, the grayscale image of each text line is pasted onto the base image to form a fixed-size text line image. The text lines are then concatenated to obtain a text line image matrix of size nImage * 1 * Wmax * 32, where nImage represents the number of text lines.

[0152] Furthermore, the entire matrix is ​​fed into the ResNeXt-Blstm text recognition model for batch recognition to obtain a text information matrix, the size of which is nImage*1. The matrix is ​​then split to obtain the recognition result for each row. In one embodiment, the resulting text row image matrix is ​​shown in the figure below. Figure 9 As shown.

[0153] By constructing a text line image matrix from the corrected text lines, batch text recognition of documents is performed. This method reduces the time required for recognizing documents with multiple text lines, effectively solving the problem of long recognition times for multi-line text and resulting in better performance.

[0154] like Figure 10 The diagram shown is a structural schematic of a text recognition device for distorted documents provided in an embodiment of the present invention. The device includes: a segmentation module 1001, a determination module 1002, a correction module 1003, and a recognition module 1004.

[0155] The segmentation module 1001 is used to segment the text lines in the text image to be recognized according to a preset segmentation model, and obtain a mask image of the text lines.

[0156] The determining module 1002 is used to perform distortion analysis on the text lines based on the mask image to determine the mapping relationship model between the text lines and the horizontal text lines;

[0157] The correction module 1003 is used to correct the text lines that are distributed in a curve according to the mapping relationship model, so as to obtain text lines that are distributed in a straight line after correction.

[0158] The recognition module 1004 is used to recognize the text in the corrected text line.

[0159] Specifically, the determining module 1002 is used to perform distortion analysis on the text line based on the mask image, determine the inflection point of the curve corresponding to the text line and the pixel value of the pixel point in the neighborhood of the inflection point; determine the edge points of the polygon based on the pixel value of the mask image; map the edge points on the curve to the position corresponding to the straight line, and determine the mapping relationship model between the text line and the horizontal text line.

[0160] Specifically, the correction module 1003 is used to divide the pixel values ​​of the pixels in the neighborhood of the inflection point into a first set and a second set; wherein the pixel value of the pixels in the first set is greater than a first preset value, the pixel value of the pixels in the second set is less than a second preset value, and the first preset value is greater than the second preset value; and the pixels located at the boundary position between the pixels in the second set and the pixels in the first set are determined as the edge points.

[0161] Specifically, the correction module 1003 is used to: calculate the sum of pixel values ​​of pixels in the first set to obtain a first sum; calculate the sum of pixel values ​​of pixels in the second set to obtain a second sum; and determine the edge point as the upper edge inflection point or the lower edge inflection point of the text line based on the magnitude of the first sum and the second sum.

[0162] Specifically, the correction module 1003 is specifically used to: if the text in the text line is dark and the background of the text is light, when the first sum is greater than the second sum, determine the edge point as the upper edge inflection point;

[0163] or,

[0164] If the text in the text line is dark and the background of the text is light, then when the first sum is less than the second sum, the edge point is determined to be the lower edge inflection point.

[0165] Specifically, the correction module 1003 is used to: if the text in the text line is light-colored and the background of the text is black, when the first sum is greater than the second sum, determine the edge point as the lower edge inflection point;

[0166] or,

[0167] If the text in the text line is light-colored and the background of the text is black, then when the first sum is less than the second sum, the edge point is determined to be the upper edge inflection point.

[0168] Specifically, the correction module 1003 is further configured to: determine the pixel mapping points corresponding to the edge inflection points of the polygon according to the mapping relationship model, wherein the pixel mapping points represent the corresponding points of the text lines in the text image to be recognized on the straight line through the mapping relationship model; determine the correction reference points of the document to be recognized according to the pixel mapping points; and correct the document to be recognized by means of a thin plate spline function according to the correction reference points.

[0169] Specifically, the correction module 1003 is used to: determine the interpolation function of the edge inflection point and the correction reference point using the thin plate spline function; and correct the document to be recognized according to the interpolation function to obtain the text lines that are linearly distributed after correction.

[0170] Specifically, the recognition module 1004 is used to: establish a set of text lines based on the corrected text lines; determine a text line image matrix based on the set of text lines; and recognize the text line image matrix through a text recognition model to obtain the text in the corrected text lines.

[0171] Specifically, the preset segmentation model includes a DBnet image segmentation network; the device further includes a preprocessing device 1000; the preprocessing device is used to train the DBnet image segmentation network and obtain the segmentation model based on the trained DBnet image segmentation network.

[0172] To implement the method of the embodiments of the present invention, the embodiments of the present invention provide a text recognition device for distorted documents, specifically, as follows: Figure 11 As shown, the device 110 includes a processor 1101 and a memory 1102 for storing computer programs that can run on the processor;

[0173] When the processor 1101 runs the computer program, it performs the following actions: segmenting the text lines in the text image to be identified according to a preset segmentation model to obtain a mask image of the text lines; performing distortion analysis on the text lines according to the mask image to obtain a mapping relationship model between the text lines and horizontal text lines; correcting the text lines that are distributed in a curve according to the mapping relationship model to obtain text lines that are distributed in a straight line after correction; and identifying the text in the corrected text lines.

[0174] In one embodiment, the processor 1101 is further configured to, when running the computer program, perform: based on the mask image, perform distortion analysis on the text line to determine the inflection point of the curve corresponding to the text line and the pixel value of the pixel point in the neighborhood of the inflection point; based on the pixel value of the mask image, determine the edge points of the polygon; map the edge points on the curve to the positions corresponding to the straight line to determine the mapping relationship model between the text line and the horizontal text line.

[0175] In one embodiment, the processor 1101 is further configured to, when running the computer program, execute: dividing the pixel values ​​of pixels in the neighborhood of the inflection point into a first set and a second set; wherein the pixel values ​​of pixels in the first set are greater than a first preset value, the pixel values ​​of pixels in the second set are less than a second preset value, and the first preset value is greater than the second preset value; and determining the pixel located at the boundary position between the pixels in the second set and the pixels in the first set as the edge point.

[0176] In one embodiment, the processor 1101 is further configured to, when running the computer program, perform the following: calculate the sum of pixel values ​​of pixels in the first set to obtain a first sum; calculate the sum of pixel values ​​of pixels in the second set to obtain a second sum; and determine the edge point as the upper edge inflection point or the lower edge inflection point of the text line based on the magnitude of the first sum and the second sum.

[0177] In one embodiment, the processor 1101 is further configured to, when running the computer program, execute: determining the edge point as the upper edge inflection point or the lower edge inflection point of the text line based on the magnitude of the first sum and the second sum, including: if the text in the text line is dark and the background of the text is light, determining the edge point as the upper edge inflection point when the first sum is greater than the second sum; or, if the text in the text line is dark and the background of the text is light, determining the edge point as the lower edge inflection point when the first sum is less than the second sum.

[0178] In one embodiment, the processor 1101 is further configured to, when running the computer program, execute: determining the edge point as the upper or lower edge inflection point of the text line based on the magnitude of the first sum and the second sum, including: if the text in the text line is light-colored and the background of the text is black, determining the edge point as the lower edge inflection point when the first sum is greater than the second sum; or, if the text in the text line is light-colored and the background of the text is black, determining the edge point as the upper edge inflection point when the first sum is less than the second sum.

[0179] In one embodiment, the processor 1101 is further configured to, when running the computer program, perform the following: determining the pixel mapping points corresponding to the edge inflection points of the polygon according to the mapping relationship model, wherein the pixel mapping points represent the corresponding points of the text lines in the text image to be recognized on the straight line through the mapping relationship model; determining the correction reference points of the document to be recognized according to the pixel mapping points; and correcting the document to be recognized by means of a thin plate spline function according to the correction reference points.

[0180] In one embodiment, the processor 1101 is further configured to, when running the computer program, perform: determining an interpolation function for the edge inflection point and the correction reference point using the thin-plate spline function; correcting the document to be recognized according to the interpolation function to obtain the text lines that are linearly distributed after correction.

[0181] In one embodiment, the processor 1101 is further configured to, when running the computer program, perform the following: establish a set of text lines based on the corrected text lines; determine a text line image matrix based on the set of text lines; and identify the text line image matrix through a character recognition model to obtain the characters in the corrected text lines.

[0182] In one embodiment, the processor 1101 is further configured to, when running the computer program, execute: the preset segmentation model, including: a DBnet image segmentation network; the processor 1101 further includes: training the DBnet image segmentation network, and obtaining the segmentation model based on the trained DBnet image segmentation network.

[0183] It should be noted that the text recognition device for distorted documents and the text recognition method for distorted documents provided in the above embodiments belong to the same concept. For details of their specific implementation process, please refer to the method embodiments, which will not be repeated here.

[0184] Of course, in practical applications, such as Figure 11 As shown, the device 110 may further include at least one network interface 1103. The various components in the distorted document text recognition device 110 are coupled together via a bus system 1104. It is understood that the bus system 1104 is used to implement communication between these components. In addition to a data bus, the bus system 1104 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 11 All buses are labeled as bus system 1104. The number of processors 1101 can be at least one. Network interface 1103 is used for wired or wireless communication between the document distortion text recognition device 110 and other devices.

[0185] The memory 1102 in this embodiment of the invention is used to store various types of data to support the operation of the text recognition device 110 for distorting documents.

[0186] The methods disclosed in the above embodiments of the present invention can be applied to processor 1101, or implemented by processor 1101. Processor 1101 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 1101 or by instructions in the form of software. The processor 1101 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 1101 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of the present invention can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory 1102. Processor 1101 reads the information in memory 1002 and completes the steps of the aforementioned method in conjunction with its hardware.

[0187] In an exemplary embodiment, the text recognition device 110 for distorting documents may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned method.

[0188] In an exemplary embodiment, the present invention also provides a computer-readable storage medium, such as a memory 1102 including a computer program, which can be executed by the processor 1101 of the text recognition device 110 for distorting documents to complete the steps described in the aforementioned method.

[0189] Specifically, embodiments of the present invention also provide a computer-readable storage medium storing a computer program thereon. When the computer program is run by a processor, it performs the following: segmenting text lines in a text image to be identified according to a preset segmentation model to obtain a mask image of the text lines; performing distortion analysis on the text lines according to the mask image to determine a mapping relationship model between the text lines and horizontal text lines; correcting the text lines that are distributed in a curve according to the mapping relationship model to obtain text lines that are distributed in a straight line after correction; and identifying the text in the corrected text lines.

[0190] In one embodiment, when the computer program is run by the processor, it performs the following: based on the mask image, it performs distortion analysis on the text lines to determine the inflection points of the curve corresponding to the text lines and the pixel values ​​of the pixels in the neighborhood of the inflection points; based on the pixel values ​​of the mask image, it determines the edge points of the polygon; and it maps the edge points on the curve to the positions corresponding to the straight lines to determine the mapping relationship model between the text lines and the horizontal text lines.

[0191] In one embodiment, when the computer program is run by the processor, it executes the following: dividing the pixel values ​​of pixels in the neighborhood of the inflection point into a first set and a second set; wherein the pixel values ​​of pixels in the first set are greater than a first preset value, the pixel values ​​of pixels in the second set are less than a second preset value, and the first preset value is greater than the second preset value; and determining the pixel located at the boundary position between the pixels in the second set and the pixels in the first set as the edge point.

[0192] In one embodiment, when the computer program is run by the processor, it performs the following: calculating the sum of pixel values ​​of pixels in the first set to obtain a first sum; calculating the sum of pixel values ​​of pixels in the second set to obtain a second sum; and determining the edge point as the upper or lower edge inflection point of the text line based on the magnitude of the first and second sums.

[0193] In one embodiment, when the computer program is run by the processor, it executes the following: determining whether the edge point is the upper or lower edge inflection point of the text line based on the magnitude of the first sum and the second sum, including: if the text in the text line is dark and the background of the text is light, determining the edge point as the upper edge inflection point when the first sum is greater than the second sum; or, if the text in the text line is dark and the background of the text is light, determining the edge point as the lower edge inflection point when the first sum is less than the second sum.

[0194] In one embodiment, when the computer program is run by the processor, it executes the following: determining whether the edge point is the upper or lower edge inflection point of the text line based on the magnitude of the first sum and the second sum, including: if the text in the text line is light-colored and the background of the text is black, determining the edge point as the lower edge inflection point when the first sum is greater than the second sum; or, if the text in the text line is light-colored and the background of the text is black, determining the edge point as the upper edge inflection point when the first sum is less than the second sum.

[0195] In one embodiment, when the computer program is run by the processor, it performs the following: determining the pixel mapping points corresponding to the edge inflection points of the polygon according to the mapping relationship model, wherein the pixel mapping points represent the corresponding points of the text lines in the text image to be recognized on the straight line through the mapping relationship model; determining the correction reference points of the document to be recognized according to the pixel mapping points; and correcting the document to be recognized using a thin-plate spline function according to the correction reference points.

[0196] In one embodiment, when the computer program is run by the processor, it performs the following: determining an interpolation function for the edge inflection point and the correction reference point using the thin-plate spline function; and correcting the document to be recognized according to the interpolation function to obtain the text lines that are linearly distributed after correction.

[0197] In one embodiment, when the computer program is run by the processor, it performs the following: establishing a set of text lines based on the corrected text lines; determining a text line image matrix based on the set of text lines; and recognizing the text line image matrix using a character recognition model to obtain the characters in the corrected text lines.

[0198] In one embodiment, when the computer program is run by the processor, it executes: the preset segmentation model, including: the DBnet image segmentation network; the processor 1101 further includes: training the DBnet image segmentation network, and obtaining the segmentation model based on the trained DBnet image segmentation network.

[0199] The following is a specific example illustrating the text recognition method for distorted documents according to an embodiment of the present invention:

[0200] Step 1: Document Image Preparation. The prepared document image dataset is used for training the DBnet segmentation network. Image preparation includes image acquisition and image annotation.

[0201] Step 1.1: Image Acquisition. In the image acquisition stage, images are primarily obtained from books, contracts, official documents, etc. During the acquisition process, rich training sample data is obtained by changing the physical deformation of the paper, camera position, lighting conditions, etc.

[0202] Step 1.2: Image annotation. Use polyline segments to enclose text lines to annotate the image data.

[0203] Step 2: Text line segmentation model training. DBnet is used as the segmentation network to predict whether each pixel in the input image is a text line region. During training, binary cross-entropy is used as the loss function, and the text line segmentation model is obtained after multiple iterations. This algorithm model is then used to predict the mask image of the text lines.

[0204] Step 3: Mask Image Distortion Analysis. The edge variations of the mask image represent the distortion state of the text lines. After smoothing the text line mask image through dilation and erosion, polygon fitting is performed using OpenCV's approxPolyDP function to obtain the correction reference points for the distorted document.

[0205] Step 4: Text Line Distortion Analysis. By statistically analyzing the changes in the 0 and 1 pixel values ​​of the 8 neighboring pixels of the inflection point of the mask polygon, the inflection point sampling of the upper edge of the polygon is obtained. The upper edge text region is located below the edge inflection point. The pixel values ​​p0, p1, p2, p3, p4, p5, p6, and p7 of the 8 neighboring pixels of the inflection point are used to divide the text region pixels p5, p6, and p7 into a second set, and the background region pixels p0, p1, p2, p3, and p4 into a first set. If sum(p5, p6, p7) is greater than sum(p0, p1, p2, p3, p4), then p is an upper edge sampling point.

[0206] Step 5: Using the pixel mapping of the text lines in the document as the reference points for document correction, perform distortion correction on the entire document. The Thin Plate Spline (TPS) function is used to correct the entire document, and the pixel mapping points from the distorted text lines to the horizontal text lines are used as N reference points for TPS correction.

[0207] Step 6: Document Recognition Module. Establish a horizontal text line set L = {L1, L2, L3, ..., Ln}. By constructing a text line image matrix, the recognition performance of multi-line text is effectively improved. First, fix the text line height at 32, scale the text images proportionally to obtain the maximum width Wmax; then pad text line images with a width smaller than Wmax with zeros. Next, construct the text line image matrix, and feed the entire matrix into the ResNeXt-Blstm text recognition model for batch recognition to obtain the text information matrix.

[0208] It should be noted that the computer-readable storage medium provided in the embodiments of the present invention may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a device including one or any combination of the above-mentioned memories.

[0209] The above description is merely an embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of the present invention are included within the scope of protection of the present invention.

Claims

1. A method for text recognition in distorted documents, characterized in that, include: The text lines in the text image to be identified are segmented according to the preset segmentation model, and the mask image of the text lines is obtained. Based on the mask image, a distortion analysis is performed on the text lines to determine the mapping relationship model between the text lines and the horizontal text lines; Based on the mapping relationship model, the text lines that are distributed in a curve are corrected to obtain text lines that are distributed in a straight line after correction; Identify the text within the corrected text line; The step of performing distortion analysis on the text lines based on the mask image to determine the mapping relationship model between the text lines and the horizontal text lines includes: Based on the mask image, a distortion analysis is performed on the text line to determine the inflection point of the curve corresponding to the text line and the pixel value of the pixel in the neighborhood of the inflection point. The edge points of the polygon are determined based on the pixel values ​​of the mask image; The edge points on the curve are mapped to positions corresponding to the straight line based on the length of the polyline between the edge points, thus determining the mapping relationship model between the text line and the horizontal text line; The inflection point is the point where the curve changes direction upward or downward.

2. The method according to claim 1, characterized in that, Based on the pixel values ​​of the mask image, the edge points of the polygon are determined, including: The pixel values ​​of the pixels in the neighborhood of the inflection point are divided into a first set and a second set; wherein the pixel value of the pixels in the first set is greater than a first preset value, the pixel value of the pixels in the second set is less than a second preset value, and the first preset value is greater than the second preset value; The pixel located at the boundary position between the pixels in the second set and the pixels in the first set is determined as the edge point.

3. The method according to claim 2, characterized in that, The step of determining the pixel located at the boundary position between the pixels in the second set and the pixels in the first set as the edge point includes: Calculate the sum of the pixel values ​​of the pixels in the first set to obtain the first sum value; And calculate the sum of the pixel values ​​of the pixels in the second set to obtain the second sum value; Based on the magnitude of the first sum and the second sum, the edge point is determined to be the upper edge inflection point or the lower edge inflection point of the text line.

4. The method according to claim 3, characterized in that, The step of determining whether the edge point is the upper or lower edge inflection point of the text line based on the magnitude of the first sum and the second sum includes: If the text in the text line is dark and the background of the text is light, when the first sum is greater than the second sum, the edge point is determined to be the upper edge inflection point; or, If the text in the text line is dark and the background of the text is light, then when the first sum is less than the second sum, the edge point is determined to be the lower edge inflection point.

5. The method according to claim 3, characterized in that, The step of determining whether the edge point is the upper or lower edge inflection point of the text line based on the magnitude of the first sum and the second sum includes: If the text in the text line is light-colored and the background of the text is black, when the first sum is greater than the second sum, the edge point is determined to be the lower edge inflection point; or, If the text in the text line is light-colored and the background of the text is black, then when the first sum is less than the second sum, the edge point is determined to be the upper edge inflection point.

6. The method according to claim 1, characterized in that, The step of correcting the text lines that exhibit a curved distribution based on the mapping relationship model includes: Based on the mapping relationship model, the pixel mapping points corresponding to the edge inflection points of the polygon are determined. The pixel mapping points represent the corresponding points on the straight line of the text line in the text image to be identified after passing through the mapping relationship model. Based on the pixel mapping points, determine the correction reference points of the document to be identified; The document to be identified is corrected using a thin-plate spline function based on the correction reference point.

7. The method according to claim 6, characterized in that, The step of correcting the document to be identified using a thin-plate spline function based on the correction reference point includes: Based on the edge inflection point and the correction reference point, the interpolation function for the edge inflection point and the correction reference point is determined by the thin plate spline function; The document to be recognized is corrected according to the interpolation function to obtain the text lines that are linearly distributed after correction.

8. The method according to claim 1, characterized in that, The text in the identified and corrected text line includes: Based on the corrected text lines, establish a set of text lines; Determine the text line image matrix based on the set of text lines; The text line image matrix is ​​identified by a text recognition model to obtain the corrected text in the text line.

9. The method according to claim 1, characterized in that, The preset segmentation model includes: DBnet image segmentation network; The method further includes: training the DBnet image segmentation network, and obtaining the segmentation model based on the trained DBnet image segmentation network.

10. A text recognition device for distorted documents, characterized in that, The device includes: a segmentation module, a determination module, a correction module, and an identification module; wherein, The segmentation module is used to segment the text lines in the text image to be recognized according to a preset segmentation model, and obtain the mask image of the text lines; The determination module is used to perform distortion analysis on the text lines based on the mask image, determine the mapping relationship model between the text lines and the horizontal text lines, and determine the inflection point of the curve corresponding to the text lines and the pixel value of the pixels in the neighborhood of the inflection point. The correction module is used to correct the text lines that are distributed in a curve according to the mapping relationship model, so as to obtain text lines that are distributed in a straight line after correction. The recognition module is used to identify the text in the corrected text lines; The determining module is specifically used for: Based on the mask image, a distortion analysis is performed on the text line to determine the inflection point of the curve corresponding to the text line and the pixel value of the pixel in the neighborhood of the inflection point. The edge points of the polygon are determined based on the pixel values ​​of the mask image; The edge points on the curve are mapped to positions corresponding to the straight line based on the length of the broken line between the edge points, thus determining the mapping relationship model between the text line and the horizontal text line; the inflection point is the point that changes the upward or downward direction of the curve.

11. The apparatus according to claim 10, characterized in that, The correction module is specifically used for: The pixel values ​​of the pixels in the neighborhood of the inflection point are divided into a first set and a second set; wherein the pixel value of the pixels in the first set is greater than a first preset value, the pixel value of the pixels in the second set is less than a second preset value, and the first preset value is greater than the second preset value; The pixel located at the boundary position between the pixels in the second set and the pixels in the first set is determined as the edge point.

12. The apparatus according to claim 11, characterized in that, The correction module is specifically used for: Calculate the sum of the pixel values ​​of the pixels in the first set to obtain the first sum value; And calculate the sum of the pixel values ​​of the pixels in the second set to obtain the second sum value; Based on the magnitude of the first sum and the second sum, the edge point is determined to be the upper edge inflection point or the lower edge inflection point of the text line.

13. The apparatus according to claim 12, characterized in that, The correction module is specifically used for: If the text in the text line is dark and the background of the text is light, then when the first sum is greater than the second sum, the edge point is determined to be the upper edge inflection point; or, If the text in the text line is dark and the background of the text is light, then when the first sum is less than the second sum, the edge point is determined to be the lower edge inflection point.

14. The apparatus according to claim 12, characterized in that, The correction module is specifically used for: If the text in the text line is light-colored and the background of the text is black, when the first sum is greater than the second sum, the edge point is determined to be the lower edge inflection point; or, If the text in the text line is light-colored and the background of the text is black, then when the first sum is less than the second sum, the edge point is determined to be the upper edge inflection point.

15. The apparatus according to claim 10, characterized in that, The correction module is further used for: Based on the mapping relationship model, the pixel mapping points corresponding to the edge inflection points of the polygon are determined. The pixel mapping points represent the corresponding points on the straight line of the text line in the text image to be identified after passing through the mapping relationship model. Based on the pixel mapping points, determine the correction reference points of the document to be identified; The document to be identified is corrected using a thin-plate spline function based on the correction reference point.

16. The apparatus according to claim 15, characterized in that, The correction module is specifically used for: Based on the edge inflection point and the correction reference point, the interpolation function for the edge inflection point and the correction reference point is determined by the thin plate spline function; The document to be recognized is corrected according to the interpolation function to obtain the text lines that are linearly distributed after correction.

17. The apparatus according to claim 10, characterized in that, The identification module is specifically used for: Based on the corrected text lines, establish a set of text lines; Determine the text line image matrix based on the set of text lines; The text line image matrix is ​​identified by a text recognition model to obtain the corrected text in the text line.

18. The apparatus according to claim 10, characterized in that, The preset segmentation model includes: DBnet image segmentation network; The apparatus further includes a preprocessing unit; the preprocessing unit is used to train the DBnet image segmentation network and obtain the segmentation model based on the trained DBnet image segmentation network.

19. A text recognition device for distorted documents, characterized in that, The device includes: a processor and a memory for storing computer programs capable of running on the processor; When the processor is used to run the computer program, it performs the steps of the method according to any one of claims 1 to 9.

20. A computer storage medium, characterized in that, The computer storage medium stores computer-executable instructions; when executed by a processor, the computer-executable instructions can implement the method as described in any one of claims 1 to 9.

Images