Method, device, medium and equipment for rectifying direction of twisted text line

Abstract

Embodiments of the present application disclose a method, device, medium and equipment for rectifying the direction of a twisted text line. The method comprises the following steps: obtaining an image to be processed; obtaining reference points on a twisted text curve corresponding to a twisted text line in the image to be processed, and generating a reference point set according to the twisted text curve; obtaining a target circular curve corresponding to the twisted text line according to the coordinates of the reference points in the reference point set, so as to represent the distribution characteristics of the twisted text curve by a more standard and normative circular curve, facilitate subsequent calculation, and improve the accuracy of direction rectification; and calculating the rotation angle of a target character in the twisted text line according to the target circular curve and a preset reference direction, rotating the target character according to the rotation angle, and obtaining a twisted text line in a positive direction, so as to facilitate user reading and improve the accuracy of text recognition on the twisted text line.

Description

Technical Field

[0001] This application relates to the field of text recognition technology, and more specifically, to a method, apparatus, computer-readable storage medium, and electronic device for correcting the direction of distorted text lines. Background Technology

[0002] Among related technologies, optical character recognition (OCR) engines can achieve good recognition results for text lines that are distributed in the positive direction (conforming to the reading and writing standards, with text characters arranged from top to bottom and from left to right). They have good robustness and can effectively deal with image quality problems such as exposure, blurring, and degradation.

[0003] However, for randomly oriented distorted text lines, such as those on the circular cross-section of products like steel coils and wheel hubs, the text lines are distributed in a ring shape. Because the circular cross-sections of the products are arranged irregularly, the angle of the text lines is uncertain. As a result, the OCR recognition engines of related technologies usually have low recognition accuracy and cannot achieve good recognition results. Summary of the Invention

[0004] To address the aforementioned technical problems, embodiments of this application provide a method, apparatus, computer-readable storage medium, and electronic device for correcting the orientation of distorted text lines, thereby improving the user's reading experience or text recognition performance.

[0005] According to one aspect of the embodiments of this application, a method for correcting the direction of distorted text lines is provided. The method includes: acquiring an image to be processed; acquiring reference points located on the distorted text curve according to the distorted text curve corresponding to the distorted text lines in the image to be processed, and generating a set of reference points; obtaining a target circular curve corresponding to the distorted text lines according to the coordinates of the reference points in the set of reference points; calculating the rotation angle of the target characters in the distorted text lines according to the target circular curve and a preset reference direction, and rotating the target characters according to the rotation angle to obtain distorted text lines distributed in a positive direction.

[0006] According to one aspect of the embodiments of this application, a direction correction device for distorted text lines is provided. The device includes: an original text acquisition module configured to acquire an image to be processed, the image to be processed including labeled text and unlabeled text; an entity relationship acquisition module configured to extract at least two entities contained in the labeled text and the association relationship between the at least two entities based on the annotation information of the labeled text, to obtain an entity relationship set; a calculation module configured to calculate the text similarity and semantic similarity between the unlabeled text and each entity contained in the entity relationship set, and obtain at least two entities contained in the unlabeled text and the association relationship between the at least two entities based on the text similarity and semantic similarity; and a graph construction module configured to construct a knowledge graph based on the at least two entities contained in the unlabeled text and the association relationship between the at least two entities.

[0007] According to one aspect of the embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the above-described method for correcting the direction of distorted text lines.

[0008] According to one aspect of the embodiments of this application, an electronic device is provided, including one or more processors; and a storage device for storing one or more programs, which, when executed by the electronic device, cause the electronic device to implement the above-described method for correcting the orientation of distorted text lines.

[0009] In the technical solution provided by the embodiments of this application, an image to be processed is acquired; reference points located on the distorted text curve corresponding to the distorted text line in the image to be processed are obtained, and a set of reference points is generated; the target circular curve corresponding to the distorted text line is obtained according to the coordinates of the reference points in the set of reference points, so as to represent the distribution characteristics of the distorted text curve with a more standardized circular curve, which facilitates subsequent calculations and improves the accuracy of orientation correction; then, according to the target circular curve and the preset reference direction, the rotation angle of the target character in the distorted text line is calculated, and the target character is rotated according to the rotation angle to obtain a distorted text line with a positive orientation, which is convenient for users to read and improves the accuracy of text recognition when performing text recognition on the distorted text line in the future.

[0010] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0011] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0012] Figure 1 A flowchart of a text recognition method provided in related technologies;

[0013] Figure 2 An image to be processed is provided for an embodiment of the application;

[0014] Figure 3 This is a schematic diagram illustrating an exemplary application environment in which the technical solutions of the embodiments of this application can be applied;

[0015] Figure 4 This is a flowchart illustrating a method for correcting the direction of distorted text lines, as shown in an embodiment of this application;

[0016] Figure 5 This is a schematic diagram illustrating the calculation of the target circular curve as shown in an embodiment of this application;

[0017] Figure 6 This is a flowchart illustrating a method for correcting the direction of distorted text lines, as shown in another embodiment of this application;

[0018] Figure 7 This is a schematic diagram illustrating the generation of a set of control points according to an embodiment of this application;

[0019] Figure 8 This is an embodiment of the present application illustrating an image corresponding to a distorted text line;

[0020] Figure 9a This is a schematic diagram illustrating the working principle of the ABCNet model for text detection, as shown in one embodiment of this application.

[0021] Figure 9b This is a comparison diagram illustrating text detection results in one embodiment of this application;

[0022] Figure 10 This is a flowchart illustrating a method for correcting the direction of distorted text lines, as shown in another embodiment of this application;

[0023] Figure 11a This is a schematic diagram illustrating an embodiment of this application of obtaining the direction of a distorted text line;

[0024] Figure 11b This is a schematic diagram illustrating another embodiment of the present application of obtaining the direction of a distorted text line;

[0025] Figure 12a This is a schematic diagram illustrating the principle of rotating a distorted text line according to an embodiment of this application;

[0026] Figure 12b To Figure 12a The image obtained by rotating the distorted text lines in the image;

[0027] Figure 13 This is a flowchart illustrating a method for correcting the direction of distorted text lines, as shown in another embodiment of this application;

[0028] Figure 14 This is a schematic diagram illustrating the rotation of a target character according to an embodiment of this application;

[0029] Figure 15 To Figure 12b The image obtained by rotating the image and then correcting the distortion is the image obtained from the image.

[0030] Figure 16 This is a flowchart illustrating a method for correcting the direction of distorted text lines, as shown in another embodiment of this application;

[0031] Figure 17 This is a schematic diagram illustrating the straightening of distorted text lines according to an embodiment of this application;

[0032] Figure 18 This is a flowchart illustrating a method for correcting the direction of distorted text lines, as shown in another embodiment of this application;

[0033] Figure 19 A flowchart of a text recognition method provided in related technologies;

[0034] Figure 20a A distorted text line provided for embodiments of this disclosure;

[0035] Figure 20b In accordance with relevant technologies Figure 20a The image obtained by correcting the distortion of the distorted text lines shown;

[0036] Figure 21 A flowchart of another method for correcting the direction of distorted text lines provided in this disclosure embodiment;

[0037] Figure 22 This is a block diagram illustrating a distorted text line orientation correction device according to an embodiment of this application;

[0038] Figure 23 This is a schematic diagram of the structure of a computer system suitable for implementing the electronic devices of the present application embodiments. Detailed Implementation

[0039] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments identical to those described in this application. Rather, they are merely examples of apparatuses and methods identical to some aspects of this application as detailed in the appended claims.

[0040] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented as application programs, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0041] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0042] It should be noted that "multiple" as mentioned in this application refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0043] In industrial settings, objects such as steel coils and wheel hubs are typically identified by laser engraving or spraying lettering on their circular cross-sections. These markings are crucial for product quality traceability, manufacturing process data management, and ultimately, the creation of large-scale data on manufactured products. In actual production, operators collect these markings through visual inspection and manual recording.

[0044] In related technologies, OCR algorithms are typically used to perform text recognition on markings in the circular cross-sections of such objects. For details, see [link to relevant documentation]. Figure 1 , Figure 1 This is a flowchart illustrating text recognition in related technologies. The input image is a circular cross-section containing the image to be detected. Character detection is then performed on the image to determine the potential text regions within the circular cross-section. Finally, character recognition is performed on these determined image regions, yielding the output: the text content of the detected text. Essentially, it involves recognizing text within an image, extracting and recognizing the text even against a complex image background.

[0045] However, as Figure 2 In the industrial scenarios depicted, the placement of objects such as steel coils and wheel hubs is generally unrestricted, resulting in uncertain tilt angles for the text to be detected, which is distributed in a ring on the circular cross-section. Text recognition algorithms perform poorly on this type of ring-shaped text. In other words, the accuracy of text recognition for ring-shaped text is low in related technologies.

[0046] To address the aforementioned problems, this disclosure provides a method for correcting the direction of distorted text lines, a device for correcting the direction of distorted text lines, a computer-readable storage medium, and an electronic device.

[0047] Optionally, in this embodiment, the above-described method for correcting the direction of distorted text lines can be applied to, for example... Figure 3 The hardware environment shown consists of server 310 and terminal 320. Figure 3 This is a schematic diagram of the hardware environment for a method for correcting the direction of distorted text lines according to an embodiment of this application, as shown below. Figure 3 As shown, the server 310 connects to the terminal 320 via a network, which includes, but is not limited to, a wide area network, a metropolitan area network, or a local area network.

[0048] In this context, server 310 refers to the server corresponding to the method for correcting the direction of distorted text lines; the server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. This application does not limit this.

[0049] The terminal 320 can be an electronic device such as a camera, smartphone, tablet, laptop, or vehicle terminal, and this application does not limit it in this regard.

[0050] The method for correcting the direction of distorted text lines in this application embodiment can be executed by the server 310, the terminal 320, or jointly by the server 310 and the terminal 320. Alternatively, the terminal 320 can execute the method for correcting the direction of distorted text lines in this application embodiment by a client installed on it.

[0051] Please see Figure 4 , Figure 4 This is a flowchart illustrating a method for correcting the direction of distorted text lines, as shown in one embodiment of this application. Figure 4As shown, the method for correcting the direction of distorted text lines includes at least steps S410 to S440, which are described in detail below:

[0052] Step S410: Obtain the image to be processed.

[0053] The image to be processed in the distorted text line direction correction method of this application embodiment can be an ordinary image with text content, such as some newspapers and magazines, especially some images with distorted text lines. Taking the product logo image with distorted text lines as an example, the distorted text lines contained in the product logo image are often the product logo contained in the image. For example, in industrial scenarios, laser marking and spraying of product logos are performed on the circular cross-sections of products such as steel coils and wheel hubs for product quality traceability, manufacturing process data management, etc. For products of different shapes, the product logo contained in the product logo image can also be displayed in different distribution shapes; for example, if the shape of the steel coil is curved (such as circular or elliptical), then the product logo engraved on the steel coil can follow the shape of the steel coil and present a curved distribution, that is, each character contained in the product logo is displayed according to the position and angle of the curve, and because the circular cross-section of the steel coil is irregularly placed, the angle of the product logo is uncertain, and the distribution direction of each character contained in the product logo is uncertain.

[0054] The method of acquiring the image to be processed can vary depending on the specific scenario. For example, the image to be processed can be pre-saved into the memory of the computer device, so acquiring the image to be processed means acquiring the image to be processed stored in the computer device's memory. Alternatively, the user can directly input the image to be processed, and when the computer device needs to acquire the image to be processed, it receives the user's data input operation to obtain the image to be processed. Another method is for the computer device to connect to other devices via a network and acquire the corresponding image to be processed from those other devices. For example, if the computer device is connected to a camera device via a network, it can send an image acquisition command to the camera device, causing the camera device to acquire the image to be processed for the current scene according to the image acquisition command and return the image to be processed to the computer device. The computer device can also acquire the image to be processed through other methods, such as web scraping. This application does not limit the method of acquiring the image to be processed.

[0055] Furthermore, the acquired image to be processed can be pre-processed image data or unprocessed image data. If the acquired image to be processed is unprocessed image data, the method for correcting the direction of distorted text lines further includes a step of preprocessing the acquired image to be processed. For example, preprocessing the acquired image to be processed may include converting the acquired image to grayscale, binarizing, normalizing, and smoothing the image. Grayscale conversion can filter out interference information carried by color images; binarization can further separate the text part and the background part; normalization unifies the text in the video frame to the same size for subsequent matching, and it includes position normalization, size normalization, and stroke thickness normalization; smoothing makes the edges of the text smoother.

[0056] The acquired image to be processed is preprocessed to avoid errors in subsequent processing caused by defects in the image itself.

[0057] Step S420: Based on the distorted text curve corresponding to the distorted text line in the image to be processed, obtain the reference points located on the distorted text curve and generate a reference point set.

[0058] It should be noted that the distorted text lines in the image to be processed consist of multiple characters, and the characters contained in the distorted text lines are not on the same straight line. The distorted text curve is used to describe the distribution characteristics of the characters contained in the distorted text lines.

[0059] For example, the upper boundary of a distorted text line in the image to be processed can be detected to obtain the lower boundary, thus obtaining the distorted text curve corresponding to the distorted text line. The upper boundary refers to the area above the reading direction of the characters in the distorted text line, and the lower boundary refers to the area below the reading direction of the characters in the distorted text line. The boundary refers to the intersection of the character area and the non-character area. Figure 5 As shown, the image to be processed includes distorted text lines. By detecting the upper boundary of the distorted text lines, the upper boundary curve A is obtained, and the upper boundary curve A is used as the distorted text curve corresponding to the distorted text lines. It is understood that the lower boundary of the distorted text lines can also be detected, and the lower boundary curve can be used as the distorted text curve corresponding to the distorted text lines; this application does not limit this.

[0060] In this exemplary embodiment, the coordinates (x, y) of each reference point can be obtained by randomly selecting a preset number N reference points on the distorted text curve. i ,y i ), where i is an integer greater than 0 and less than or equal to N, x i The x-coordinate of reference point i is represented by y. i Let i represent the ordinate of the reference point, and then we can obtain the set of reference points.

[0061] For example, such as Figure 5 As shown, a reference point can be selected at a preset distance on the distorted text curve to obtain reference points 1, 2, 3, and 4. Then, based on the coordinates of each reference point, the set of reference points is {(x1,y1), (x2,y2), (x3,y3), (x4,y4)}.

[0062] By sampling the distribution characteristics of the characters contained in the distorted text line, reference points located on the distorted text curve are obtained, and a set of reference points is generated.

[0063] Step S430: Obtain the target circular curve corresponding to the distorted text line based on the coordinates of the reference points in the reference point set.

[0064] It should be noted that the target circular curve refers to the circular arc curve used to characterize the character distribution characteristics of distorted text lines.

[0065] Understandably, since distorted text lines may be irregularly distributed, relying solely on the distorted text curve cannot accurately determine their distribution direction. Directly correcting based on the distorted text curve corresponding to each distorted text line may result in inaccurate correction. In contrast, the distribution of the target circular curve is more regular than that of the distorted text curve, making it easier to determine the distribution direction of the distorted text lines.

[0066] Therefore, by using the coordinates of reference points in the reference point set, the target circular curve corresponding to the distorted text line is obtained, thereby improving the accuracy of subsequent orientation correction of the distorted text line. For example... Figure 5 The upper boundary curve A shown is the distorted text curve of the distorted text line. Based on the coordinates of reference points 1, 2, 3, and 4 on the upper boundary curve A, a circular curve is fitted to obtain the target circular curve B. The coordinates of these four points can be denoted as (x1, y1), (x2, y2), (x3, y3), and (x4, y4).

[0067] For example, obtaining the center of the target circular curve includes: obtaining the coordinates of reference points in the reference point set; and performing circular curve fitting on the distorted text curve based on the coordinates of the reference points and the least squares method to obtain the target circular curve and the center of the circle in which the target circular curve is located.

[0068] A circular curve can be fitted based on the coordinates of the reference points in the reference point set to obtain the target circular curve. For example, the least squares method can be used to estimate the minimum variance to obtain the corresponding equation of the target circular curve, as shown in formula (1):

[0069] (1)

[0070] Where i is an integer greater than 0 and less than or equal to N, N is the number of reference points in the reference point set, and x i The x-coordinate of reference point i is represented by y. i Let represent the ordinate of reference point i, f represent the variance, a represent the abscissa of the center of the target circular curve equation, b represent the ordinate of the center of the target circular curve equation, and R represent the radius of the circle in the target circular curve equation.

[0071] For x in the function f shown in the above formula (1) c y c Taking the partial derivative with R, to minimize the value of function f, we can set the result of the partial derivative to 0, which yields the following system of equations:

[0072] (2)

[0073] (3)

[0074] Based on the above formulas (2) and (3), the following system of equations can be obtained:

[0075] (4) C = N∑x i 2 -∑x i ∑x i

[0076] (5) D=NΣx i y i -∑x i ∑y i

[0077] (6)

[0078] (7) G = N∑y i 2 -∑y i ∑y i

[0079] (8)

[0080] Where N represents the number of reference points on the distorted text curve.

[0081] The coordinates of the four points on the distorted text curve are used as the coordinates of (x) in formulas (4) to (8) above. i y i The values ​​of C, D, E, G, and H can be calculated from this.

[0082] Based on the above formulas (1) to (8), the following system of equations can be obtained:

[0083] (9)

[0084] (10)

[0085] (11)

[0086] Based on the values ​​of C, D, E, G, H and the above formulas (9) to (10), the values ​​of a, b and R can be calculated, and the curve equation of the circle containing the distorted text curve and the coordinates of the center of the circle (a, b) can be obtained.

[0087] The equation of the target circular curve is:

[0088] (12), (x) i -a) 2 +(y i -b) 2 =R 2

[0089] Step S440: Calculate the rotation angle of the target character in the distorted text line according to the target circular curve and the preset reference direction, and rotate the target character according to the rotation angle to obtain a distorted text line distributed in the positive direction.

[0090] It should be noted that the preset reference direction can be any direction, such as vertical or horizontal. The rotation angle refers to the angle through which the target character in the distorted text line passes during rotation processing. The target character refers to the character that needs to be rotated, usually all characters contained in the distorted text line. Positive direction distribution means that the character arrangement conforms to the reading and writing standard, that is, the characters are arranged from top to bottom and from left to right.

[0091] For example, the distribution direction of the current distorted text line can be obtained based on the target circular curve. Further, based on the angle between the distribution direction of the current distorted text line and the preset reference direction, the rotation angle of the target character in the distorted text line is obtained, and then the target character is rotated according to the rotation angle to obtain a distorted text line with a positive distribution.

[0092] By rotating the distorted text lines in the image to be processed, distorted text lines with a positive orientation are obtained, which improves the accuracy of text recognition when performing text recognition on the distorted text lines in the subsequent process.

[0093] Please see Figure 6 In the above exemplary embodiments, the method for correcting the direction of distorted text lines includes steps S610 to S660, which are described in detail below:

[0094] Step S610: Obtain the image to be processed.

[0095] The specific implementation method of step S610 and Figure 4 The steps in step S410 are the same and will not be repeated here.

[0096] Step S620: Divide the pixels in the image to be processed that belong to text into regions to obtain text regions containing distorted text lines.

[0097] It is understood that the image to be processed includes pixels that belong to text and pixels that do not belong to text. By dividing the pixels in the image to be processed into regions that belong to text, text regions containing distorted text lines are obtained. A single image to be processed may include multiple text regions containing distorted text lines.

[0098] For example, the image to be processed can be input into a text region detection network. The text region detection network generates a classification result for each pixel in the image, indicating whether the pixel belongs to the text. After the text region detection network performs the classification operation on each pixel, it obtains a text region containing distorted text lines. Specifically, the classification result can be represented as a character or numeric code indicating whether a pixel belongs to the text. For example, the classification result could be "yes" and "no"; another example could be "0" and "1", where "0" represents a pixel that does not belong to the text, and "1" represents a pixel that does belong to the text, etc. The classification result can also be represented as a probability value for whether a pixel belongs to the text. For example, an output classification result of "0-0.3" and "1-0.7" means that the probability of the pixel not belonging to the text is 0.3, and the probability of the pixel belonging to the text is 0.7, etc. The specific form of the classification result is not limited here.

[0099] Step S630: Extract the curve features of the distorted text lines in the text region to obtain the distorted text curve.

[0100] It should be noted that the curve feature of the distorted text line is used to characterize the deformation features of the distorted text line, and the distorted text curve is obtained through the curve feature of the distorted text line.

[0101] For example, the distortion curve of the text region corresponding to the distortion text line in the image to be processed can be estimated by binarizing the image to be processed. This can include: binarizing the image to be processed, performing a closing operation on the resulting binary image to form the shape of the text line, finding the boundary line of the text line shape through gradient operation, and fitting the boundary line to obtain the distortion text curve of the distortion text line.

[0102] It is understandable that other methods can be used to obtain distorted text curves, such as machine learning methods. No specific method for obtaining distorted text curves is restricted here.

[0103] Step S640: Based on the distorted text lines and corresponding distorted text curves in the image to be processed, obtain reference points located on the distorted text curves and generate a set of reference points.

[0104] The specific implementation method of step S640 and Figure 4 The steps in step S420 are the same and will not be repeated here.

[0105] Step S650: Obtain the target circular curve corresponding to the distorted text line based on the coordinates of the reference points in the reference point set.

[0106] The specific implementation method of step S650 and Figure 4 The steps in step S430 are the same and will not be repeated here.

[0107] Step S660: Calculate the rotation angle of the target character in the distorted text line according to the target circular curve and the preset reference direction, and rotate the target character according to the rotation angle to obtain a distorted text line distributed in the positive direction.

[0108] The specific implementation method of step S660 and Figure 4 The steps in step S440 are the same and will not be repeated here.

[0109] Optionally, in this application embodiment, step S630 can also be implemented using the following steps, as detailed below: obtaining a set of boundary points for the boundary markers of the text region; fitting a Bézier curve as a distorted text curve based on the set of boundary points.

[0110] It should be noted that the boundary of the text region refers to the intersection of the text region and the non-text region, and the coordinates of the boundary points in the boundary point set are used to indicate the position of the text region of the distorted text line.

[0111] For example, based on the set of boundary points of the distorted text line, a set of control points in the distorted text line is generated using either least squares estimation or stochastic gradient descent. The concept of the boundary point set corresponds to the concept of the control point set. The control points in the control point set are used to input into the Bézier curve equation to generate a Bézier curve, which indicates the location of the text region in the image to be processed.

[0112] Among them, the Bézier curve is a mathematical curve used in two-dimensional graphics applications. It is a smooth curve drawn based on the coordinates of i arbitrary points (i.e., the coordinates of i control points). The value of i is the order of the Bézier curve plus 1. i can be any integer greater than or equal to 3. The larger the value of i, the higher the precision of the Bézier curve, and thus the more accurate the range of the text area defined by the Bézier curve. However, the larger the value of i, the more computer resources are used. Generally, i can be 3, 4, 5, 6 or other values. To further understand this scheme, the generation formula of the Bézier curve is disclosed as follows (13):

[0113] Formula (13),

[0114] Where c(t) represents the dependent variable of the Bézier curve, b i B represents the control point parameter. i ,(t) represents the parameters of the Bézier equation, n represents the order of the Bézier equation, and t represents the independent variable of the Bézier curve.

[0115] For a further understanding of this solution, please refer to [link / reference]. Figure 7 , Figure 7 This is a schematic diagram of generating a set of control points in the method for correcting the direction of distorted text lines provided in the embodiments of this application. Figure 7 Taking an image to be processed containing a distorted text line, with a Bézier curve of order 3 as an example, Figure 7 The left image shows the set of boundary points for the distorted text region in the image to be processed, including the coordinates of 10 boundary points from b1 to b10. The right image shows the set of control points to be fitted into a Bézier curve, including 8 control points from c1 to c8. c1, c2, c3, and c4 are the control points for the first Bézier curve, which indicates the lower boundary of the text region; c5, c6, c7, and c8 are the control points for the second Bézier curve, which indicates the upper boundary of the text region. It should be understood that... Figure 7 The examples in this document are for illustrative purposes only and are not intended to limit the scope of this solution.

[0116] For example, text detection can be performed on the image to be processed to obtain image regions (i.e., text regions) containing distorted text lines, where the text regions are circular. For instance, an electronic device can perform text detection on the image to be processed to obtain... Figure 8 The text region shown is a circularly distributed, distorted text line. The text region is a circular image and contains two curved boundaries, namely an inner curved boundary and an outer curved boundary. Figure 8The "212345789123567" in the text area represents the characters contained in the twisted text line, which are distributed in a circular pattern.

[0117] In one implementation, the image to be processed can be input into a pre-trained ABCNet (Adaptive Bezier-Curve Network) model to obtain the position of the distorted text line in the image to be processed, as output by the ABCNet model. The server can then extract the image region at that position from the image to obtain the text region. The position of the distorted text line output by the ABCNet model includes the coordinates of four points evenly distributed within the inner curved boundary of the distorted text line in the image to be processed, and the coordinates of four points evenly distributed within the outer curved boundary in the image to be processed.

[0118] The ABCNet model is trained based on sample images and the locations of sample text within those images. The sample text in the sample images is distributed in a circular pattern. The locations of the sample text can include the coordinates of four points evenly distributed within the inner curved boundary of the sample text in the sample image, and the coordinates of four points evenly distributed within the outer curved boundary of the sample text in the sample image.

[0119] For example, see Figure 9a , Figure 9a This is a schematic diagram illustrating the working principle of text detection using the ABCNet model provided in this embodiment of the disclosure. The input image is the image to be processed input to ABCNet, which contains circularly distributed distorted text lines.

[0120] By using the Bezier Curve Detection layer of the ABCNet model, the backbone of the region containing distorted text lines in the image to be processed can be determined. By fitting the determined backbone to the Bezier curve, the corresponding Bezier curve can be obtained.

[0121] The Bezier Align layer of the ABCNet model is used to extract features from the image to be processed. Based on the feature extraction results and the determined Bezier curve, the positions of the distorted text lines in the image to be processed are obtained, that is, the coordinates of four points evenly distributed in the inner curved boundary of the distorted text lines and the coordinates of four points evenly distributed in the outer curved boundary of the distorted text lines in the image to be processed.

[0122] The ABCNet model can adaptively fit arbitrarily shaped twisted text lines using parameterized Bézier curves and extract convolutional features of these lines through the BezierAlign layer. Compared to standard bounding box detection methods, the ABCNet model can improve the accuracy of text detection, and the computational cost of Bézier curve detection is negligible, giving ABCNet advantages in both efficiency and accuracy for text detection.

[0123] See Figure 9b , Figure 9b The image on the left is a region in the image to be processed that contains distorted text lines, detected based on horizontal sampling. Figure 9b The image in the middle is obtained based on Quadrilateral sampling detection, and the image to be processed contains regions containing distorted text lines. Figure 9b The image on the right is obtained based on the Bezier Align layer in ABCNet, and the image to be processed contains regions with distorted text lines.

[0124] Horizontal sampling and Quadrilateral sampling are both standard bounding box detection methods.

[0125] As can be seen, text detection based on the Bezier Align layer in ABCNet can largely preserve the original shape of distorted text lines, that is, it can accurately fit the shape of the region containing distorted text lines without introducing a large amount of invalid background information.

[0126] Subsequently, the text regions are extracted based on the positions of the distorted text lines in the image to be processed. A Lightweight Recognition Head algorithm is then used to recognize the text within these regions, yielding the text content of the distorted text lines. For example, Figure 9a The text content of the distorted text lines in the image to be processed includes: "HANDORF", "NEWSAGENCY", and "GIFTS".

[0127] When extracting a text region, the coordinates of four points evenly distributed within the outer curved boundary of the text region, and the coordinates of four points evenly distributed within the inner curved boundary, can be obtained. The server can select a curved boundary of the text region (i.e., a distorted text curve), calculate the average of the coordinates of the four points on the distorted text curve, and obtain the coordinates of the midpoint on the distorted text curve. The specified curve boundary can be either the outer curved boundary or the inner curved boundary of the text region.

[0128] The server can also use the circle containing the distorted text curve as the target circle curve to obtain the coordinates of the center of the target circle curve.

[0129] Optionally, in the above exemplary embodiments, such as Figure 10 As shown, the method for correcting the direction of distorted text lines includes steps S1010 to S1050, which are described in detail below:

[0130] Step S1010: Obtain the image to be processed.

[0131] The specific implementation method of step S1010 and Figure 4 The steps in step S410 are the same and will not be repeated here.

[0132] Step S1020: Based on the distorted text lines and corresponding distorted text curves in the image to be processed, obtain reference points located on the distorted text curves and generate a set of reference points.

[0133] The specific implementation method of step S1020 and Figure 4 The steps in step S420 are the same and will not be repeated here.

[0134] Step S1030: Obtain the target circular curve corresponding to the distorted text line based on the coordinates of the reference points in the reference point set.

[0135] The specific implementation method of step S1030 and Figure 4 The steps in step S430 are the same and will not be repeated here.

[0136] Step S1040: Confirm the direction of the distorted text line based on the target circular curve.

[0137] The distribution direction of the distorted text line in the image to be processed can be obtained by using the target circular curve.

[0138] For example, the distribution direction of the current distorted text lines can be the direction of the tangent at the midpoint of the target circular curve. The distribution direction of the distorted text lines can be obtained by acquiring the midpoint of the target circular curve and then using that tangent.

[0139] For example, the current distribution direction of the distorted text lines can be the direction of the median of the target circular curve. This can be achieved by obtaining the center and midpoint of the target circular curve; the straight line passing through the center and midpoint is the median of the target circular curve, and thus the distribution direction of the distorted text lines can be obtained based on the median.

[0140] Understandably, the specific form of text line distortion can be chosen based on the actual application scenario, and no restrictions are placed on the specific form of text line distortion here.

[0141] Optionally, in this embodiment of the application, step S1040 may also be implemented using the following steps, as detailed below: calculating the mean value of the coordinates of each reference point in the reference point set to obtain the mean point; and obtaining the center of the target circular curve; obtaining the center line of the target circular curve based on the mean point and the center line; and obtaining the direction of the distorted text line based on the center line.

[0142] Please see Figure 11a as well as Figure 11b , Figure 11a as well as Figure 11b To obtain a schematic diagram of the direction of the distorted text line, as shown in the figure, the target circular curve is curve B, and the reference point set includes reference point 1, reference point 2, reference point 3, and reference point 4. Then, based on the coordinates of each reference point, the reference point set is obtained as follows:

[0143] {(x1,y1), (x2,y2), (x3,y3), (x4,y4)}

[0144] By calculating the coordinates of each reference point, the coordinates of the mean point y are obtained as follows:

[0145] ((x1+x2+x3+x4) / 4, (y1+y2+y3+y4) / 4)

[0146] The coordinates of the center of the target circular curve are (a, b). Connecting the center c and the mean point y, we obtain the direction of the distorted text line.

[0147] For example, one can determine which side of the character in the distorted text line is closer to curve B, thus determining the direction of the distorted text line. For instance, if the side of the character in the distorted text line closer to curve B is the top side of the character, then... Figure 11a As shown, the direction of the distorted text line is the side closest to the distorted text line. If the side of a character in the distorted text line closest to curve B is the bottom side of the character itself, then as shown... Figure 11b As shown, the direction of the distorted text line is away from the side of the distorted text line.

[0148] Understandably, the direction of the distorted text line can be obtained and defined according to the actual application scenario, and there are no restrictions on the methods for obtaining and defining the direction of the distorted text line here.

[0149] Step 1050: Obtain the rotation angle of the target character in the distorted text line according to the direction of the distorted text line and the preset reference direction, and rotate the target character in the distorted text line according to the rotation angle to obtain a distorted text line distributed in the positive direction.

[0150] The reference direction can be defined according to the actual application scenario. However, it should be noted that different reference directions will result in different rotation angles of the target characters in the distorted text line.

[0151] For example, such as Figure 11a As shown, the reference direction is the vertical direction. The angle θ between the reference direction and the direction of the distorted text line is obtained, and this angle θ is used as the rotation angle.

[0152] For example, such as Figure 11b As shown, the reference direction is horizontal, and the angle between this reference direction and the direction of the distorted text line is obtained. The included angle As the rotation angle.

[0153] For example, a rectangular coordinate system is drawn with the center of the circle as the origin. Connecting the center of the circle with the mean point yields the direction of the distorted text line. The acute angle between the reference direction and the direction of the distorted text line can be directly calculated, and the rotation angle is obtained based on the quadrant in which the mean point is located. For instance, if the reference direction is horizontal and the acute angle between the reference direction and the direction of the distorted text line is β, the rotation angle is 90-β when the mean point is in the first quadrant; -(90-β) when the mean point is in the second quadrant; -(90-β) when the mean point is in the third quadrant; and 90+β when the mean point is in the fourth quadrant.

[0154] Understandably, when the rotation angle is 0, the surface reference direction and the vertical direction coincide, so there is no need to rotate the distorted text line.

[0155] In some implementations, the preset reference direction is the horizontal axis of the target coordinate system, the origin of the target coordinate system is the center of the circle corresponding to the target circular curve, the horizontal axis of the target coordinate system is horizontal, and the vertical axis of the target coordinate system is vertical. The step of obtaining the rotation angle of the target character in the distorted text line based on the direction of the distorted text line and the preset reference direction includes: calculating the acute angle between the direction of the distorted text line and the horizontal axis of the target coordinate system based on the coordinates of the direction of the distorted text line in the target coordinate system, and obtaining the coordinate quadrant in which the direction of the distorted text line is located in the target coordinate system; in a second correspondence between preset coordinate quadrants and rotation angles, determining the rotation angle corresponding to the coordinate quadrant in which the direction of the distorted text line is located in the target coordinate system as the rotation angle of the target character in the distorted text line; wherein the second correspondence includes: the rotation angles corresponding to the first and second quadrants are the difference between the preset angle and the acute angle; the rotation angles corresponding to the third and fourth quadrants are the sum of the preset angle and the acute angle.

[0156] A coordinate system can be established with the center of the target circular curve as the origin, the horizontal direction as the abscissa, and the vertical direction as the ordinate, thus obtaining the target coordinate system.

[0157] The quadrant in which the midpoint of the distorted text curve lies in the target coordinate system is also the quadrant in which the direction of the distorted text line lies in the target coordinate system. Therefore, based on the quadrant in which the direction of the distorted text line lies in the target coordinate system, the rotation direction and angle corresponding to the target character within the distorted text line can be determined.

[0158] For example, the direction of the distorted text line can be obtained from the line segment (i.e., the target line segment) between the midpoint of the distorted text curve and the center of the target circular curve.

[0159] The direction of the target line segment represents the tilt direction of the distorted text lines within the text area relative to the horizontal direction. The target line segment is perpendicular to the reading direction of the distorted text lines, which is the direction of the tangent line at the midpoint of the distorted text curve in the text area. To adjust the distorted text lines to a positive orientation, the adjusted target line segment needs to be vertical, thus making the reading direction of the distorted text lines horizontal. Having the adjusted target line segment vertical also ensures that the target characters within the distorted text lines in the text area are in a positive orientation.

[0160] If the target line segment lies in one quadrant of the target coordinate system, then a larger portion of the image area within the text region also lies in that quadrant. For example, see [link to relevant documentation]. Figure 12a , Figure 12a Point M is the midpoint of the distorted text curve in the text region, point O is the origin of the target coordinate system, and MO represents the target line segment. Figure 12a The target line segment is located in the first quadrant, and a large portion of the image area within the text region is also located in the first quadrant. If a small angle needs to be rotated to make the target line segment vertical, that is, to make the text region an image in the positive direction, then the text region needs to be rotated counterclockwise.

[0161] If the target line segment is located in the second quadrant of the target coordinate system, it means that the text area is located in the second quadrant of the target coordinate system. If you want to rotate the target line segment by a small angle so that it is vertical, that is, so that the text area is in the positive direction, then you need to rotate the text area clockwise.

[0162] If the target line segment is located in the third quadrant of the target coordinate system, it means that the text area is located in the third quadrant of the target coordinate system. If you want to rotate the target line segment by a small angle so that it is vertical, that is, so that the text area is in the positive direction, then you need to rotate the text area clockwise.

[0163] If the target line segment is located in the fourth quadrant of the target coordinate system, it means that the text area is located in the fourth quadrant of the target coordinate system. If you want to rotate the target line segment by a small angle so that it is vertical, that is, so that the text area is in the positive direction, then you need to rotate the text area counterclockwise.

[0164] Furthermore, if the target line segment lies on the positive half-axis of the horizontal coordinate system, it can be determined that the target line segment is located in the first or fourth quadrant. If the target line segment lies on the negative half-axis of the horizontal coordinate system, it can be determined that the target line segment is located in the second or third quadrant. Similarly, if the target line segment lies on the positive half-axis of the vertical coordinate system, it can be determined that the target line segment is located in the first or second quadrant. If the target line segment lies on the negative half-axis of the vertical coordinate system, it can be determined that the target line segment is located in the third or fourth quadrant.

[0165] If the target line segment is located on the horizontal axis of the target coordinate system, the acute angle between the target line segment and the horizontal axis of the target coordinate system can be determined to be 0 degrees. If the target line segment is located on the vertical axis of the target coordinate system, the acute angle between the target line segment and the horizontal axis of the target coordinate system can be determined to be 90 degrees.

[0166] In the second correspondence between the preset coordinate quadrants and rotation angles, the rotation angle corresponding to the coordinate quadrant in the target coordinate system where the target line segment is located is determined, and used as the rotation angle corresponding to the text area.

[0167] The second correspondence includes: the rotation angles corresponding to the first and second quadrants are the difference between the preset angle and the acute angle; the rotation angles corresponding to the third and fourth quadrants are the sum of the preset angle and the acute angle.

[0168] The average coordinates of each reference point in the distorted text curve of the text region can be calculated to obtain the coordinates of the midpoint of the distorted text curve, denoted as (x0, y0). The electronic device can calculate the acute angle between the target line segment and the horizontal axis of the target coordinate system according to the following formula.

[0169] (14)

[0170] α represents the acute angle between the target line segment and the x-axis of the target coordinate system, arctan represents the arctangent function, (x0, y0) represents the coordinates of the midpoint of the text region, |y0| represents the absolute value of y0, and |x0| represents the absolute value of x0.

[0171] Furthermore, the electronic device can determine the rotation angle corresponding to the distorted text line based on the coordinate quadrant in which the target line segment is located in the target coordinate system and the calculated acute angle.

[0172] The preset angle is 90 degrees. Therefore, the second correspondence includes: if the target line segment is located in the first and second quadrants, the corresponding rotation angle is 90-α; if the target line segment is located in the third and fourth quadrants, the corresponding rotation angle is 90+α.

[0173] After determining the rotation direction and rotation angle of the target character in the distorted text line, the origin of the target coordinate system can be used as the rotation center. The target character in the distorted text line can be rotated according to the determined rotation angle and rotation direction to obtain a distorted text line with a positive direction distribution. The target line segment after rotation is in the vertical direction.

[0174] For example, taking the origin O of the target coordinate system as the center of rotation, and rotating by an angle of 90-α, for... Figure 12a Rotating the distorted text line counterclockwise as shown yields... Figure 12b The distorted text line shown. Figure 12b In the distorted text line shown, the target line segment MO is vertical. Since the target line segment is perpendicular to the reading direction of the distorted text line, the reading direction of the distorted text line can be made horizontal.

[0175] Optionally, in the above exemplary embodiments, such as Figure 13 As shown, the method for correcting the direction of distorted text lines includes steps S1310 to S1360, which are described in detail below:

[0176] Step S1310: Obtain the image to be processed.

[0177] The specific implementation method of step S1310 and Figure 4 The steps in step S410 are the same and will not be repeated here.

[0178] Step S1320: Based on the distorted text lines and corresponding distorted text curves in the image to be processed, obtain reference points located on the distorted text curves and generate a set of reference points.

[0179] The specific implementation method of step S1320 and Figure 4 The steps in step S420 are the same and will not be repeated here.

[0180] Step S1330: Obtain the target circular curve corresponding to the distorted text line based on the coordinates of the reference points in the reference point set.

[0181] The specific implementation method of step S1330 and Figure 4 The steps in step S430 are the same and will not be repeated here.

[0182] Step S1340: Confirm the direction of the distorted text line based on the target circular curve.

[0183] The specific implementation method of step S1340 and Figure 10 The steps in step S1040 are the same and will not be repeated here.

[0184] Step S1350: Obtain the rotation direction of the target character in the distorted text line according to the direction of the distorted text line, and select a point located on the direction of the distorted text line to obtain the rotation center point.

[0185] After obtaining the rotation angle, it is also necessary to obtain the rotation center point corresponding to the rotation processing of the distorted text line. The rotation center point is the point whose position will not change during the rotation processing.

[0186] In some embodiments, obtaining the rotation direction of the target character in the distorted text line based on its direction includes: obtaining the coordinate quadrant in the target coordinate system where the direction of the distorted text line is located; wherein the origin of the target coordinate system is the center of the circle corresponding to the target circular curve; the horizontal axis of the target coordinate system is horizontal; the vertical axis of the target coordinate system is vertical; in a preset first correspondence between coordinate quadrants and rotation directions, determining the rotation direction corresponding to the coordinate quadrant in the target coordinate system where the direction of the distorted text line is located, as the rotation direction of the target character in the distorted text line; wherein the first correspondence includes: the rotation direction corresponding to the first quadrant and the fourth quadrant is counterclockwise rotation; the rotation direction corresponding to the second quadrant and the third quadrant is clockwise rotation. Step S1360: Rotate the target character according to the rotation center point, rotation angle, and rotation direction to obtain a distorted text line distributed in a positive direction.

[0187] The target characters of the distorted text lines are rotated according to the rotation center point and rotation angle to correct the randomly distributed distorted text lines in the image to be oriented in a positive direction.

[0188] For example, please refer to Figure 14 , Figure 14 This is a schematic diagram illustrating the rotation process applied to the target character. For example... Figure 14 As shown, the reference direction is the vertical direction. The rotation angle between the reference direction and the direction of the distorted text line is obtained as θ. The center of the circle in the direction of the distorted text line is taken as the rotation center point. The target characters corresponding to the distorted text line are rotated according to the rotation center point and the rotation angle to obtain the distorted text line distributed in the positive direction.

[0189] Since there are more text samples with a positive orientation when training a text recognition model, the trained text recognition model is more accurate in recognizing text with a positive orientation. In this embodiment, the orientation of randomly distributed distorted text lines is corrected before text recognition to improve the text recognition effect of the distorted text lines.

[0190] The image to be processed after rotation is now in the forward orientation. The distorted text lines in this rotated image remain circularly distributed. Therefore, distortion correction is needed to adjust the distorted text lines back to a forward-oriented distribution. For example, for... Figure 12b The distorted text lines shown can be corrected. Figure 15 The text image shown, Figure 15 The text in the shown image is positively distributed, meaning... Figure 15 The characters in the text shown are arranged from top to bottom and from left to right, which conforms to human reading and writing standards.

[0191] Please see Figure 16 In the above exemplary embodiments, the method for correcting the direction of distorted text lines includes steps S1610 to S1660, which are described in detail below:

[0192] Step S1610: Obtain the image to be processed.

[0193] The specific implementation method of step S1610 and Figure 4 The steps in step S410 are the same and will not be repeated here.

[0194] Step S1620: Based on the distorted text lines and corresponding distorted text curves in the image to be processed, obtain reference points located on the distorted text curves and generate a set of reference points.

[0195] The specific implementation method of step S1620 and Figure 4 The steps in step S420 are the same and will not be repeated here.

[0196] Step S1630: Obtain the target circular curve corresponding to the distorted text line based on the coordinates of the reference points in the reference point set.

[0197] The specific implementation method of step S1630 and Figure 4 The steps in step S430 are the same and will not be repeated here.

[0198] Step S1640: Calculate the rotation angle of the target character in the distorted text line according to the target circular curve and the preset reference direction, and rotate the target character according to the rotation angle to obtain a distorted text line distributed in the positive direction.

[0199] The specific implementation method of step S1640 and Figure 4 The steps in step S440 are the same and will not be repeated here.

[0200] Step S1650: Straighten the text lines that are positively distributed according to the distorted text curve to obtain the pre-recognized text lines.

[0201] In this embodiment, straightening the distorted text lines distributed in a positive direction according to the distorted text curve to obtain the pre-recognized text lines may include: discretizing the distorted text curve into line segments according to a pre-set pixel scale, and mapping the line segments to horizontal text lines using a rheostat transform. Other methods can also be used to complete the straightening process, which are not limited herein.

[0202] Please see Figure 17 , Figure 17 This is an example diagram illustrating the process of straightening distorted text lines. (See diagram below.) Figure 17 As shown, the straightening process can include: first, discretizing the distorted text curve into line segments according to a pre-set pixel scale (e.g., a scale where step = 1 pixel), and then mapping the corresponding line segments onto the horizontal text lines using a photosensitive transformation. Here, for pixels that are mapped multiple times during this process, the average value of multiple pixels can be taken.

[0203] In some implementations, straightening the positively distributed distorted text lines according to the distorted text curve to obtain pre-recognized text lines includes: calculating a mapping matrix between the image to be processed and the preset image region based on the coordinates of each vertex in the positively distributed distorted text lines in the image to be processed, and the coordinates of vertices of the preset image region in the preset image region; for each pixel in the image to be processed, calculating the corresponding pixel coordinates of the pixel in the preset image region based on the mapping matrix and the pixel coordinates of the pixel in the image to be processed; and using the pixel value of the pixel as the pixel value of the corresponding pixel coordinates to obtain the pre-recognized text line.

[0204] The preset image region is the image area occupied by the pre-recognized text line to be generated. Each pixel in the image to be processed is mapped to the preset image region to obtain the pre-recognized text line. Since each vertex of the preset image region corresponds to each vertex of the image to be processed, a mapping matrix between the image to be processed and the preset image region can be obtained based on the coordinates of each vertex in the preset image region and the coordinates of each vertex in the image to be processed. The mapping matrix represents the transformation relationship between the image coordinate system of the image to be processed and the image coordinate system of the preset image region.

[0205] Then, for each pixel in the image to be processed, the corresponding pixel coordinates in the preset image region can be calculated based on the mapping matrix and the pixel's coordinates in the image to be processed. That is, based on the transformation relationship between the image coordinate system of the image to be processed and the image coordinate system of the preset image region, the coordinates of the pixel are transformed to obtain the corresponding pixel coordinates in the preset image region. Furthermore, the pixel value of this pixel can be used as the pixel value of the corresponding pixel coordinates to obtain the pre-recognized text line. Step S1660: Perform text recognition processing on the pre-recognized text line to obtain the text recognition result.

[0206] It should be noted that the text recognition result refers to the recognition result of the text content corresponding to the target character in the distorted text line.

[0207] For example, the pre-recognized text lines are input into the trained text recognition model to obtain the text recognition result output by the model. The text recognition model is trained based on sample images, which contain text line images and corresponding text content labels. During the training process, the text line images are used as input, and the corresponding text content labels are used as the target output. The loss value between the actual output and the target output of the text recognition model is calculated. When this loss value is less than a preset loss value or the number of training iterations reaches a preset number, the trained text recognition model is obtained.

[0208] In one implementation, the pre-identified text line region can be segmented into multiple sub-regions based on the pixels in the pre-identified text line. Each sub-region is the image area occupied by one character in the distorted text line. Feature extraction can be performed on each sub-region to obtain its feature information, and the similarity between the feature information of the sub-region and each feature information in a preset feature library can be calculated. Then, the character corresponding to the feature information with the highest similarity can be determined as the character in that sub-region, thus obtaining the text content of the pre-identified text line.

[0209] In another implementation, the pre-recognized text line can be input into a pre-trained text recognition model. The feature extraction network of the text recognition model can then process the pre-recognized text line to obtain its feature sequence. Each element in the feature sequence corresponds one-to-one with several sub-regions in the pre-recognized text line; each element represents the image feature of the corresponding sub-region. The feature extraction network can be a CNN (Convolutional Neural Networks) model.

[0210] Then, the feature sequence of the pre-recognized text line can be processed by the classification network of the text recognition model to obtain the label sequence corresponding to the feature sequence. The elements in the label sequence correspond one-to-one with several sub-regions in the pre-recognized text line; each element represents the character feature of the character in the corresponding sub-region. The classification network can be an RNN (Recurrent Neural Network) model.

[0211] Furthermore, the label sequences of the pre-identified text lines can be processed using the transcription network of the text recognition model to obtain the text content of the text to be detected. The transcription network can be a CTC (connectionist temporal classification) model.

[0212] The following describes in detail a specific application scenario of this application embodiment, taking the identification of product markings on steel coils as an example:

[0213] Please see Figure 18 The system acquires an image containing the product logo of a steel coil in the current scene using a camera, obtaining the image to be processed. It detects the presence of characters in the image; if so, it identifies the distorted text curve corresponding to the distorted text line in the image based on Bézier curves, and obtains reference points on the distorted text curve, generating a reference point set. Based on the coordinates of the reference points in the reference point set, it fits the target circular curve corresponding to the distorted text line. Then, based on the target circular curve, it obtains the direction of the distorted text line and a preset reference direction, and obtains the rotation angle. Further, it rotates the target characters in the distorted text line according to the rotation angle, obtaining distorted text lines distributed in a positive direction. Then, it corrects the distortion of the positively distributed distorted text lines, obtaining straightened pre-recognized text lines, which are then input into a text recognition model to obtain the text recognition result output by the model.

[0214] For example, a detection network can be used to determine whether an image to be processed contains text. If the image does not contain text, no further processing is required.

[0215] If the image to be processed contains text, the target circular curve can be obtained by fitting the equation of a circular curve based on the detected outer curve. Specifically, a set of reference points for the character region is obtained from the detection network. The first half of the reference points is taken and denoted as P_data[]. The determined points are used as the points for fitting the circular equation. Least square fitting is performed based on the reference points in P_data[] to obtain the coordinates (a, b) of the center (denoted as O) and the radius R of the circle. That is, when the server obtains the distorted text line, four reference points are uniformly distributed on the distorted text curve of the determined distorted text line. Based on the coordinates of these four reference points (i.e., P_data[]) and the least square algorithm, the center (a, b) and radius R of the circle containing the distorted text curve are obtained.

[0216] Then, the server can calculate the angle between the line connecting the center of the circle and the mean point of the outer curve and the horizontal direction. Specifically, it can calculate the average coordinates of each reference point in P_data[] to obtain the coordinates of the midpoint (denoted as M) of the distorted text curve of the distorted text line as (x0, x0), and connect point M and point O to obtain the target line segment MO, and calculate the acute angle (denoted as α) between the target line segment and the horizontal direction.

[0217] Furthermore, the server can rotate the characters to the positive direction based on the included angle. Specifically, the server determines the rotation angle in the vertical-horizontal coordinate system (i.e., the target coordinate system) with the center of the circle as the origin. If point M is in the first quadrant, the rotation angle Rotate_d is 90 - α, and the rotation direction is counterclockwise; if point M is in the second quadrant, the rotation angle Rotate_d is 90 - α, and the rotation direction is clockwise; if point M is in the third quadrant, the rotation angle Rotate_d is 90 + α, and the rotation direction is clockwise; if point M is in the fourth quadrant, the rotation angle Rotate_d is 90 + α, and the rotation direction is counterclockwise. The electronic device rotates the circle by an angle Rotate_d with the center (a, b) as the rotation center to obtain the final distorted text line in the positive direction.

[0218] The server can perform distortion correction, that is, it corrects the distortion of the distorted text lines after orientation correction to obtain pre-recognized text lines, thereby adjusting the circularly distributed distorted text lines in the image to be oriented normally. Then, the pre-recognized text lines are input into a character recognition network to obtain the recognition result. In other words, it uses a text recognition algorithm to perform text recognition on the pre-recognized text lines to obtain the text content within the distorted text lines.

[0219] See Figure 19 , Figure 19This is a flowchart of a text recognition method provided in related technologies. The input image is the image to be processed, which contains distorted text lines. Then, the image to be processed is input to a character position detection network, i.e., a server, to perform text detection on the image to be processed, obtaining the image region occupied by the distorted text lines in the image to be processed, which is taken as the text image.

[0220] Then, the server performs distortion correction on the text image, adjusting the circularly distributed distorted text lines in the text image to a normal distribution, resulting in pre-recognized text lines. These pre-recognized text lines are then input into a character recognition network to obtain the output. In other words, the pre-recognized text lines are processed using a text recognition algorithm to obtain the text content of the distorted text lines.

[0221] In related technologies, because the tilt angle of the distorted text lines in the text image cannot be determined, even after distortion correction of the distorted text lines, the distorted text in the resulting pre-recognized text lines may still not be positively distributed. For example, directly... Figure 20a The text image shown is distorted and corrected to obtain... Figure 20b The pre-identified text line shown, Figure 20b The distorted text in the pre-identified text line shown is still not normally distributed text, resulting in poor text recognition performance from text recognition algorithms. In other words, among related technologies, the accuracy of text recognition for circular text is relatively low.

[0222] See Figure 21 , Figure 21 This is a flowchart illustrating another method for correcting the orientation of distorted text lines provided in this embodiment. The input image is the image to be processed, which contains distorted text lines. Then, the image to be processed is input to a character position detection network (server) to perform text detection on the image, obtaining the image region occupied by the distorted text lines, which is then used as the text image.

[0223] Then, the server fits the equation of the circle to the external curve, that is, the server performs a circular curve fitting on the distorted text curve of the text image to obtain the curve equation of the target circular curve, and determines the target line segment between the midpoint of the distorted text curve of the text image and the center of the target circular curve. The server can also calculate the tilt angle, that is, the server calculates the acute angle between the target line segment and the horizontal axis of the target coordinate system based on the coordinates of the midpoint of the distorted text curve of the text image in the target coordinate system.

[0224] Furthermore, the server can rotate the detection area to the positive direction. That is, the server determines the rotation direction and rotation angle of the text image based on the coordinate quadrant where the target line segment is located in the target coordinate system, and rotates the text image according to the rotation direction and rotation angle to obtain the distorted text line in the positive direction.

[0225] The server can also perform distortion correction, that is, correct the distortion of the distorted text lines after orientation correction to obtain pre-recognized text lines, thereby adjusting the text in the orientation-corrected distorted text lines to be positively distributed text. Then, the pre-recognized text lines are input into a character recognition network to obtain the output result. In other words, text recognition algorithms are used to recognize the text in the pre-recognized text lines to obtain the text content of the distorted text lines.

[0226] The method for correcting the orientation of distorted text lines provided in this application involves: acquiring an image to be processed; obtaining reference points located on the distorted text curve corresponding to the distorted text line in the image to be processed, and generating a set of reference points; obtaining the target circular curve corresponding to the distorted text line based on the coordinates of the reference points in the set of reference points, so as to represent the distribution characteristics of the distorted text curve with a more standardized circular curve, which facilitates subsequent calculations and improves the accuracy of orientation correction; then, calculating the rotation angle of the target character in the distorted text line based on the target circular curve and the preset reference direction, and rotating the target character according to the rotation angle to obtain a distorted text line with a positive orientation, which is convenient for users to read and improves the accuracy of text recognition when performing text recognition on the distorted text line in the future.

[0227] Figure 22 This is a block diagram illustrating a distorted text line orientation correction device according to an embodiment of this application, as shown below. Figure 22 As shown, the device includes:

[0228] Image acquisition module 2210 is configured to acquire an image to be processed;

[0229] The reference point set acquisition module 2220 is configured to acquire reference points located on the distorted text curve based on the distorted text curve corresponding to the distorted text line in the image to be processed, and generate a reference point set.

[0230] The circular curve acquisition module 2230 is configured to obtain the target circular curve corresponding to the distorted text line based on the coordinates of the reference points in the reference point set;

[0231] The orientation correction module 2240 is configured to calculate the rotation angle of the target character in the distorted text line based on the target circular curve and the preset reference direction, and rotate the target character according to the rotation angle to obtain a distorted text line with a positive orientation.

[0232] In one embodiment of this application, before obtaining reference points located on the distorted text curve based on the distorted text curve corresponding to the distorted text line in the image to be processed, and generating a set of reference points, the apparatus may further include:

[0233] The text region acquisition unit is configured to divide the pixels belonging to text in the image to be processed into regions to obtain text regions containing distorted text lines.

[0234] The distorted text curve acquisition unit is configured to extract the curve features of distorted text lines in a text region to obtain the distorted text curve.

[0235] In one embodiment of this application, the distorted text curve acquisition unit may include:

[0236] The boundary point set acquisition unit is configured to acquire a set of boundary points for the boundary markers of the text region;

[0237] The fitting unit is configured to fit a Bézier curve as a distorted text curve based on the set of boundary points.

[0238] In one embodiment of this application, the orientation correction module 2240 may include:

[0239] The first direction confirmation unit is configured to confirm the direction of the distorted text line based on the target circular curve.

[0240] Angle calculation unit is configured to obtain the rotation angle of the target character in the distorted text line based on the direction of the distorted text line and a preset reference direction.

[0241] In one embodiment of this application, the first direction confirmation unit may include:

[0242] The mean point and circle center acquisition unit is configured to calculate the mean of the coordinates of each reference point in the reference point set to obtain the mean point; and to acquire the circle center corresponding to the target circular curve.

[0243] The centerline acquisition unit is configured to obtain the centerline corresponding to the target circular curve based on the mean point and the center of the circle.

[0244] The direction acquisition sub-unit is configured to obtain the direction of the distorted text line based on the center line.

[0245] In one embodiment of this application, the mean point and center point acquisition unit may include:

[0246] The coordinate confirmation unit is configured to obtain the coordinates of reference points in the reference point set.

[0247] The fitting unit is configured to perform circular curve fitting on the distorted text curve based on the coordinates of the reference point and the least squares method, so as to obtain the target circular curve and the center of the circle in which the target circular curve is located.

[0248] In one embodiment of this application, the preset reference direction is the abscissa axis of the target coordinate system, the origin of the target coordinate system is the center of the circle corresponding to the target circular curve, the abscissa axis of the target coordinate system is the horizontal direction, and the ordinate axis of the target coordinate system is the vertical direction; the angle calculation unit includes:

[0249] The angle and quadrant determination unit is configured to calculate the acute angle between the direction of the distorted text line and the horizontal axis of the target coordinate system based on the coordinates of the distorted text line direction in the target coordinate system, and to obtain the coordinate quadrant in which the distorted text line direction is located in the target coordinate system.

[0250] The rotation angle determination unit is configured to determine, within a preset second correspondence between coordinate quadrants and rotation angles, the rotation angle corresponding to the quadrant in the target coordinate system where the direction of the distorted text line is located, as the rotation angle of the target character in the distorted text line; wherein, the second correspondence includes: the rotation angles corresponding to the first and second quadrants are the difference between a preset angle and an acute angle; the rotation angles corresponding to the third and fourth quadrants are the sum of a preset angle and an acute angle. In one embodiment of this application, the direction correction module 2240 may include:

[0251] The second direction confirmation unit is configured to confirm the direction of the distorted text line based on the target circular curve.

[0252] The center point and direction determination unit is configured to obtain the rotation direction of the target character in the distorted text line according to the direction of the distorted text line, and to select a point located on the direction of the distorted text line to obtain the rotation center point;

[0253] The correction unit is configured to rotate the target character according to the rotation center point, rotation angle and rotation direction to obtain distorted text lines that are distributed in the positive direction.

[0254] In one embodiment of this application, obtaining the rotation direction of the target character in the distorted text line based on its direction includes: obtaining the coordinate quadrant in the target coordinate system where the direction of the distorted text line is located; wherein the origin of the target coordinate system is the center of the circle corresponding to the target circular curve; the horizontal axis of the target coordinate system is horizontal; the vertical axis of the target coordinate system is vertical; in a preset first correspondence between coordinate quadrants and rotation directions, determining the rotation direction corresponding to the coordinate quadrant in the target coordinate system where the direction of the distorted text line is located, as the rotation direction of the target character in the distorted text line; wherein the first correspondence includes: the rotation direction corresponding to the first quadrant and the fourth quadrant is counterclockwise rotation; the rotation direction corresponding to the second quadrant and the third quadrant is clockwise rotation. In one embodiment of this application, after obtaining the distorted text lines distributed in a positive direction, the device may further include:

[0255] The distortion correction unit is configured to straighten the text lines that are positively distributed according to the distortion text curve to obtain the pre-recognized text lines.

[0256] The text recognition unit is configured to perform text recognition processing on the pre-recognized text line to obtain the text recognition result.

[0257] In one embodiment of this application, the text alignment of a positively distributed distorted text line is straightened according to the distorted text curve to obtain a pre-recognized text line. This includes: calculating a mapping matrix between the image to be processed and a preset image region based on the coordinates of each vertex in the positively distributed distorted text line in the image to be processed and the coordinates of the vertices of a preset image region in the preset image region; for each pixel in the image to be processed, calculating the corresponding pixel coordinates of the pixel in the preset image region based on the mapping matrix and the coordinates of the pixel in the image to be processed; and using the pixel value of the pixel as the pixel value of the corresponding pixel coordinates to obtain the pre-recognized text line. Figure 23 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown.

[0258] It should be noted that, Figure 23 The computer system 2300 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0259] like Figure 23 As shown, the electronic device 2300 is manifested in the form of a general-purpose computing device. The components of the electronic device 2300 may include, but are not limited to: at least one processing unit 2310, at least one storage unit 2320, a bus 2330 connecting different system components (including storage unit 2320 and processing unit 2310), and a display unit 2340.

[0260] The storage unit stores program code, which can be executed by the processing unit 2310, causing the processing unit 2310 to perform the steps described in the "Exemplary Methods" section above according to various exemplary embodiments of this disclosure.

[0261] Storage unit 2320 may include readable media in the form of volatile storage units, such as random access memory (RAM) 2321 and / or cache memory 2322, and may further include read-only memory (ROM) 2323.

[0262] Storage unit 2320 may also include a program / utility 2324 having a set (at least one) program module 2325, such program module 2325 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0263] Bus 2330 can represent one or more of several types of bus structures, including memory cell bus or memory cell controller, peripheral bus, graphics acceleration port, processing unit, or local bus using any of the multiple bus structures.

[0264] Electronic device 2300 can also communicate with one or more external devices 2370 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 2300, and / or with any device that enables electronic device 2300 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 2350. Furthermore, electronic device 2300 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 2360. As shown, network adapter 2360 communicates with other modules of electronic device 2300 via bus 2330. It should be understood that, although not shown in the figures, other hardware and / or application modules can be used in conjunction with electronic device 2300, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0265] In particular, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer applications. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. When the computer program is executed by the processing unit 2310, it performs various functions defined in the system of this application.

[0266] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. The transmitted data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0267] The units described in the embodiments of this application can be implemented by application programs or by hardware, and the described units can also be located in a processor. The names of these units do not necessarily constitute a limitation on the unit itself.

[0268] Another aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned method for correcting the orientation of distorted text lines. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently without being assembled into the electronic device.

[0269] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the distorted text line orientation correction method provided in the various embodiments described above.

[0270] The above description is merely a preferred exemplary embodiment of this application and is not intended to limit the implementation of this application. Those skilled in the art can easily make corresponding modifications or alterations based on the main concept and spirit of this application. Therefore, the scope of protection of this application should be determined by the scope of protection claimed in the claims.

Claims

1. A method for correcting the direction of distorted text lines, characterized in that, The method includes: Obtain the image to be processed; Based on the distorted text curve corresponding to the distorted text line in the image to be processed, obtain the reference points located on the distorted text curve and generate a set of reference points; Based on the coordinates of the reference points in the set of reference points, the target circular curve corresponding to the distorted text line is obtained; The direction of the distorted text line is determined based on the target circular curve; The rotation angle of the target character in the distorted text line is obtained based on the direction of the distorted text line and the preset reference direction. The target character is rotated according to the rotation angle to obtain a distorted text line distributed in a positive direction. The target character refers to the character contained in the distorted text line.

2. The method according to claim 1, characterized in that, Before obtaining reference points located on the distorted text curves corresponding to the distorted text lines in the image to be processed, and generating a set of reference points, the method further includes: The pixels belonging to text in the image to be processed are divided into regions to obtain the text region containing the distorted text line; Extract the curve features of the distorted text lines in the text region to obtain the distorted text curve.

3. The method according to claim 2, characterized in that, The step of extracting the curve features of the distorted text lines in the text region to obtain the distorted text curve includes: Obtain the set of boundary points for the boundary markers of the text region; A Bézier curve is fitted based on the set of boundary points to serve as the distorted text curve.

4. The method according to claim 1, characterized in that, Determining the direction of the distorted text line based on the target circular curve includes: Calculate the mean value of the coordinates of each reference point in the reference point set to obtain the mean value point; and obtain the center of the circle corresponding to the target circular curve; Based on the mean point and the center of the circle, the center line corresponding to the target circular curve is obtained; The direction of the distorted text line is obtained based on the center line.

5. The method according to claim 4, characterized in that, Obtaining the center of the circle corresponding to the target circular curve includes: Obtain the coordinates of the reference points in the set of reference points; Based on the coordinates of the reference point and the least squares method, a circular curve is fitted to the distorted text curve to obtain the target circular curve and the center of the circle containing the target circular curve.

6. The method according to claim 1, characterized in that, The preset reference direction is the horizontal axis of the target coordinate system, the origin of the target coordinate system is the center of the circle corresponding to the target circular curve, the horizontal axis of the target coordinate system is horizontal, and the vertical axis of the target coordinate system is vertical; obtaining the rotation angle of the target character in the distorted text line based on the direction of the distorted text line and the preset reference direction includes: Based on the coordinates of the direction of the distorted text line in the target coordinate system, calculate the acute angle between the direction of the distorted text line and the horizontal axis of the target coordinate system, and obtain the coordinate quadrant in which the direction of the distorted text line is located in the target coordinate system; In the second correspondence between the preset coordinate quadrants and rotation angles, the rotation angle corresponding to the coordinate quadrant in which the direction of the distorted text line is located in the target coordinate system is determined as the rotation angle of the target character in the distorted text line; wherein, the second correspondence includes: the rotation angles corresponding to the first quadrant and the second quadrant are: the difference between the preset angle and the acute angle; the rotation angles corresponding to the third quadrant and the fourth quadrant are: the sum of the preset angle and the acute angle.

7. The method according to claim 1, characterized in that, The process of rotating the target characters according to the stated rotation angle to obtain distorted text lines distributed in a positive direction includes: The direction of the distorted text line is determined based on the target circular curve; The rotation direction of the target character in the distorted text line is obtained based on the direction of the distorted text line, and a point located on the direction of the distorted text line is selected to obtain the rotation center point; The target character is rotated according to the rotation center point, the rotation angle, and the rotation direction to obtain a distorted text line distributed in a positive direction.

8. The method according to claim 7, characterized in that, The step of obtaining the rotation direction of the target character in the distorted text line based on the direction of the distorted text line includes: Obtain the coordinate quadrant in the target coordinate system where the direction of the distorted text line is located; wherein, the origin of the target coordinate system is the center of the circle corresponding to the target circular curve; the horizontal axis of the target coordinate system is the horizontal direction; and the vertical axis of the target coordinate system is the vertical direction. In the first correspondence between the preset coordinate quadrants and rotation directions, the rotation direction corresponding to the coordinate quadrant in the target coordinate system where the direction of the distorted text line is located is determined as the rotation direction of the target character in the distorted text line; wherein, the first correspondence includes: the rotation direction corresponding to the first quadrant and the fourth quadrant is counterclockwise rotation; the rotation direction corresponding to the second quadrant and the third quadrant is clockwise rotation.

9. The method according to any one of claims 1 to 8, wherein after obtaining the distorted text lines distributed in a positive direction, the method further comprises: Based on the distorted text curve, the positively distributed distorted text lines are straightened to obtain pre-identified text lines; The pre-identified text line is subjected to text recognition processing to obtain the text recognition result.

10. The method according to claim 9, wherein the step of straightening the positively distributed distorted text lines according to the distorted text curve to obtain the pre-identified text lines includes: Based on the coordinates of each vertex in the positively oriented twisted text lines in the image to be processed, and the coordinates of the vertices of the preset image region in the preset image region, a mapping matrix between the image to be processed and the preset image region is calculated. For each pixel in the image to be processed, the corresponding pixel coordinates in the preset image region are calculated based on the mapping matrix and the coordinates of the pixel in the image to be processed. The pixel value of the given pixel is used as the pixel value of the corresponding pixel coordinates to obtain the pre-recognized text line.

11. A device for correcting the direction of distorted text lines, characterized in that, The device includes: The image acquisition module is configured to acquire the image to be processed. The reference point set acquisition module is configured to acquire reference points located on the distorted text curves corresponding to the distorted text lines in the image to be processed, and generate a reference point set. The circular curve acquisition module is configured to obtain the target circular curve corresponding to the distorted text line based on the coordinates of the reference points in the reference point set. The orientation correction module is configured to determine the orientation of the distorted text line based on the target circular curve; obtain the rotation angle of the target character in the distorted text line based on the orientation of the distorted text line and a preset reference orientation; rotate the target character according to the rotation angle to obtain a distorted text line distributed in a positive direction; the target character refers to the character contained in the distorted text line.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method for correcting the direction of distorted text lines as described in any one of claims 1 to 10.

13. An electronic device, characterized in that, include: processor; as well as A memory for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the orientation correction method for distorted text lines as described in any one of claims 1 to 10.

Images