Inclined text direction classification method and device, terminal equipment and readable storage medium

Abstract

This application provides a method, apparatus, terminal device, and readable storage medium for classifying tilted text orientation. The method includes: inputting an image to be detected into a trained tilted text detection model; predicting and outputting text bounding box information of the region containing text in the image; the text bounding box information including the coordinates of two vertices of a rotating rectangle used to enclose the corresponding text string; selecting a preset proportion of text bounding boxes; and calculating the text orientation classification result of each selected text bounding box according to the coordinates of the two vertices of the rotating rectangle using a bounding box orientation classification algorithm; and determining the final text orientation classification of the image based on each of the text orientation classification results. This method avoids implementing text orientation determination as an independent functional module, saving on model parameter overhead, training overhead, prediction overhead, and subsequent computational overhead related to orientation classification tasks.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of text image processing, and in particular to a tilted text direction classification method and device, a terminal device and a readable storage medium. BACKGROUND

[0002] With the rapid development of smart phones, artificial intelligence, deep learning and other fields, image character recognition OCR technology has been widely used in recent years. People use mobile phones to take various images containing text, such as certificates, bills, documents, test questions, natural scenes, etc., and use OCR technology to recognize text from images to quickly obtain the required information. People use mobile phones to take images with certain randomness and non-standardization, which greatly increases the possibility of text tilt in images, and the image may also be rotated, further affecting the direction of the image text. Whether based on traditional image algorithms or deep learning-based text recognition technology, the text needs to be obtained in the forward direction for recognition, so image text direction judgment or classification is an important part of text recognition.

[0003] The existing image text direction classification method usually needs an independent function module to support, for example, a method based on deep learning, which uses an independent text direction classification model for separate processing. As an independent function module, an additional step is required for application deployment, resulting in greater computational and resource overhead. SUMMARY

[0004] The embodiments of the present application provide a tilted text direction classification method, device, terminal device and readable storage medium, which avoids implementing text direction determination as an independent function module, saving model parameter overhead, training overhead, prediction overhead and subsequent calculation overhead related to direction classification tasks.

[0005] In a first aspect, the embodiments of the present application provide a tilted text direction classification method, comprising:

[0006] inputting a to-be-detected image into a trained tilted text detection model to predict and output text bounding box information of a text region in the image, the text bounding box information comprising two vertex coordinates of a rotated rectangle surrounding a corresponding text string;

[0007] selecting a preset number of text bounding boxes, and calculating a text direction classification result of each selected text bounding box according to the two vertex coordinates of the rotated rectangle and a bounding box direction classification algorithm;

[0008] determining a final text direction classification of the image according to the text direction classification results.

[0009] In one embodiment, the two vertices of the rotating rectangle are denoted as the first vertex and the second vertex, wherein the first vertex and the second vertex are defined as the top-left and top-right vertices of the rotating rectangle when the enclosed text is in a normalized state, respectively; the step of calculating the text direction classification result of each selected text enclosing box according to the coordinates of the two vertices of the rotating rectangle using the enclosing box direction classification algorithm includes:

[0010] The tilt angle of the rotating rectangle is calculated using the coordinates of the first and second vertices of the rotating rectangle.

[0011] Based on multiple predefined preset directions and their corresponding specified angle intervals, the angle interval in which the tilt angle is located is determined, and the preset direction corresponding to the angle interval in which the tilt angle is located is used as the direction classification result of the rotating rectangle.

[0012] In one embodiment, calculating the tilt angle of the rotating rectangle using the coordinates of the first and second vertices of the rotating rectangle includes:

[0013] A slope value is calculated based on the coordinates of the first and second vertices of the rotated rectangle using the slope formula.

[0014] Based on the comparison of the x-coordinates of the first and second vertices of the rotating rectangle, and the comparison of the y-coordinates of the first and second vertices, the corresponding angle calculation formula is selected.

[0015] The tilt angle of the rotating rectangle is calculated based on the slope value and the selected angle calculation formula.

[0016] In one embodiment, selecting the corresponding angle calculation formula based on the comparison results of the x-coordinates of the first and second vertices of the rotated rectangle and the comparison results of the y-coordinates of the first and second vertices includes:

[0017] When the x-coordinate of the first vertex is less than the x-coordinate of the second vertex, the first angle calculation formula is selected. The first angle calculation formula is:

[0018] θ = arctan(K) * 180 / π;

[0019] Where θ is the tilt angle of the rotated rectangle; K is the slope value;

[0020] If the x-coordinate of the first vertex is greater than the x-coordinate of the second vertex, and the y-coordinate of the first vertex is less than or equal to the y-coordinate of the second vertex, then the second angle calculation formula is selected. The second angle calculation formula is as follows:

[0021] θ=180°+arctan(K)*180 / π;

[0022] If the x-coordinate of the first vertex is greater than the x-coordinate of the second vertex, and the y-coordinate of the first vertex is greater than the y-coordinate of the second vertex, then the third angle calculation formula is selected. The third angle calculation formula is as follows:

[0023] θ=-180°+arctan(K)*180 / π.

[0024] In one embodiment, the plurality of preset directions include a first direction to a fourth direction, wherein the specified angle intervals corresponding to the first direction to the fourth direction are respectively (-45°, 45°], (45°, 135°], (-180°, -135°]∪(135°, 180°], and (-135°, -45°).

[0025] In one implementation, the pre-training process of the tilted text detection model includes:

[0026] Obtain multiple original images containing slanted text, and label all text regions in each original image with bounding boxes according to the state where the text in the image is upright, in order to obtain multiple labeled training images.

[0027] The labeled training images are classified according to multiple preset directions and rotated for data augmentation to obtain several augmented images with updated labels. The several augmented images and the multiple training images constitute a training image set.

[0028] The constructed neural network model is trained for skewed text detection using the training image set, and training is stopped when the neural network model meets the preset convergence condition, thus obtaining the trained skewed text detection model.

[0029] In one implementation, determining the final text direction classification of the image based on each of the text direction classification results includes:

[0030] The text direction classification with the highest percentage among all the text direction classification results is used as the final text direction classification for the image.

[0031] Secondly, embodiments of this application provide a tilted text orientation classification device, including:

[0032] The prediction output module is used to input the image to be detected into the trained text detection model and predict and output the text bounding box information of the region where each text is located in the image. The text bounding box information includes the coordinates of the two vertices of the rotating rectangle used to enclose the corresponding text.

[0033] The orientation classification calculation module is used to select a preset proportion of text bounding boxes, and calculate the text orientation classification result of each selected text bounding box according to the coordinates of the two vertices of the rotating rectangle and the bounding box orientation classification algorithm.

[0034] The orientation classification determination module is used to determine the final text orientation classification of the image based on the various text orientation classification results.

[0035] Thirdly, embodiments of this application provide a terminal device, which includes a processor and a memory. The memory stores a computer program, and the processor executes the computer program to implement the tilted text direction classification method.

[0036] Fourthly, embodiments of this application provide a readable storage medium storing a computer program that, when executed on a processor, implements the aforementioned tilted text orientation classification method.

[0037] The embodiments of this application have the following beneficial effects:

[0038] The tilted text orientation classification method of this application involves inputting the image to be detected into a trained tilted text detection model, predicting and outputting text bounding box information for the region containing the text in the image. The text bounding box information includes the coordinates of the two vertices of a rotating rectangle used to enclose the corresponding text string. A preset proportion of text bounding boxes are selected, and the text orientation classification result for each selected text bounding box is calculated according to the coordinates of the two vertices of the rotating rectangle using a bounding box orientation classification algorithm. The final text orientation classification of the image is determined based on each of the text orientation classification results. This method, based on the prediction results of the tilted text detection model combined with a post-processing algorithm, can directly achieve image text orientation classification, avoiding its implementation as an independent functional module and saving on model parameter overhead, training overhead, prediction overhead, and subsequent computational overhead related to orientation classification tasks. Attached Figure Description

[0039] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 A flowchart illustrating a tilted text orientation classification method according to an embodiment of this application is shown.

[0041] Figure 2This diagram illustrates the training process of the text detection model for the tilted text orientation classification method according to an embodiment of this application.

[0042] Figure 3a and Figure 3b The images shown are schematic diagrams illustrating text that has not been straightened and text that has been straightened in the tilted text orientation classification method of this application, respectively.

[0043] Figure 4 This illustration shows a text annotation diagram of the tilted text direction classification method according to an embodiment of this application;

[0044] Figure 5 A schematic flowchart illustrating the text direction classification calculation process of the tilted text direction classification method according to an embodiment of this application is shown.

[0045] Figure 6 A schematic flowchart illustrating the tilt angle calculation of the tilt text direction classification method according to an embodiment of this application is shown;

[0046] Figures 7a-7d Schematic diagrams illustrating the four text direction classifications in embodiments of this application are shown respectively;

[0047] Figure 8 A schematic diagram of the structure of the tilted text orientation classification device according to an embodiment of this application is shown. Detailed Implementation

[0048] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0049] The components of the embodiments of this application described and illustrated in the accompanying drawings can be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of this application provided in the drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0050] In the following, the terms “comprising,” “having,” and their cognates, which may be used in various embodiments of this application, are intended only to indicate a particular feature, number, step, operation, element, component, or combination thereof, and should not be construed as excluding, firstly, the presence of one or more other features, numbers, steps, operations, elements, components, or combinations thereof, or adding the possibility of one or more features, numbers, steps, operations, elements, components, or combinations thereof.

[0051] Furthermore, the terms "first," "second," and "third" are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.

[0052] Unless otherwise specified, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of this application pertain. Terms (such as those defined in commonly used dictionaries) shall be interpreted as having the same meaning as in their contextual meaning in the relevant technical field and shall not be construed as having an idealized or overly formal meaning, unless clearly defined in the various embodiments of this application.

[0053] Example 1

[0054] Figure 1 The diagram shown is a schematic flowchart of a tilted text orientation classification method according to an embodiment of this application. Exemplarily, the tilted text orientation classification method includes:

[0055] Step S110: Input the image to be detected into the trained text detection model and predict and output the text bounding box information of the corresponding text. The text bounding box information includes the coordinates of the two vertices of the rotating rectangle used to enclose the corresponding text.

[0056] The aforementioned text detection model is primarily used to detect and locate text strings in the input image, outputting text bounding box information corresponding to each text string. In this embodiment, the text bounding box information mainly includes the coordinates of the corresponding vertices of the rotating rectangle used to enclose the corresponding text string in the image, and may also include the height or width of the rotating rectangle, etc. Furthermore, the coordinates of these vertices are stored in a certain order, such as clockwise or counterclockwise, for convenient subsequent access.

[0057] In some text images that require text recognition and other processing, it is often necessary to first determine the text orientation of the text image. Therefore, this embodiment will directly use the text bounding box information predicted and output by the text detection model to directly calculate the text orientation classification of the text image, without having to input the obtained text bounding box into an additional text orientation classification model. This can greatly reduce the computational overhead and is computationally efficient.

[0058] The text detection model described above can be obtained through pre-training. It is worth noting that the training images for the text detection model in this embodiment should satisfy the independent and identically distributed condition with the image to be detected. For example, when the text detection model is used for invoice recognition, its training images can be invoice images collected in different scenarios; or, when the text detection model is used for property certificate information recognition, its training images should also be collected property certificate images, etc. This ensures that the corresponding vertices of the text bounding boxes in unlabeled images can be accurately predicted and located.

[0059] In one implementation, such as Figure 2 As shown, the training process of this text detection model includes:

[0060] Step S210: Obtain multiple original images containing slanted text, and label all text regions in each original image with bounding boxes according to the state where the text in the image is upright, in order to obtain multiple labeled training images.

[0061] This embodiment will primarily use horizontally formatted text as an example. Figure 3a The image shown is an original photograph with a tilted text string that has not been straightened. It's clear that this angle is inconvenient for users to read or for computers to perform text recognition. Therefore, the entire image needs to be manually straightened so that the text is upright, resulting in the image shown. Figure 3b The image shown shows the text after it has been turned upright.

[0062] As an example, for each acquired original image, in order to train the text detection model, text bounding boxes will be added to all text in the original images. Specifically, taking each text string as a unit, and with the text upright, the two vertices and height of the rotated rectangle of each text string in the image will be labeled using manual annotation or semi-automatic annotation by a pre-trained model.

[0063] In one implementation, the annotation information of a text string can be stored as (Xtop1, Ytop1, Xtop2, Ytop2, h), such as... Figure 4As shown, (Xtop1, Ytop1) represent the x and y coordinates of the top-left vertex of the rotating rectangle used to enclose the text string when the text in the image is rotated. Similarly, (Xtop2, Ytop2) represent the x and y coordinates of the top-right vertex of the rotating rectangle. Of course, the height h can also be obtained by labeling all four vertices and further calculating; this is not a limitation here. It should be understood that when determining the text direction classification, in addition to using only the top-left and top-right vertices mentioned above, the bottom-left and bottom-right vertices can also be used, with similar calculation methods. This embodiment mainly uses the top-left and top-right vertices as examples for explanation.

[0064] Step S220: Rotate and augment the labeled training images according to multiple preset directions to obtain several augmented images with updated labels. The several augmented images and the multiple training images constitute a training image set.

[0065] Since the number of original images collected is often limited, this embodiment will also utilize labeled training images for data augmentation to ensure the robustness and detection performance of the trained model. Exemplarily, augmented images can be obtained by rotating the labeled training images according to multiple pre-defined directional classifications. For example, this can be divided into two stages: the first stage involves rotating the labeled images at small angles, such as randomly rotating them within a range of ±5 degrees to ±10 degrees, to obtain cases with different small-angle tilts in a single direction; the second stage involves rotating the numerous images with small-angle tilts in a single direction at large angles, such as 90 degrees, 180 degrees, and 270 degrees, according to the predetermined directional classifications, thereby augmenting them into augmented images with different tilt angles belonging to different directional classifications. During the rotation process, the corresponding vertex coordinates of the bounding boxes of the text in the image are also calculated and updated according to the corresponding rotation angle. Finally, the augmented images and the labeled training images are used together as a training image set for training the text detection model.

[0066] Step S230: Use the training image set to train the constructed neural network model for tilted text detection, and stop training when the neural network model meets the preset conditions to obtain the trained text detection model.

[0067] In one implementation, the aforementioned neural network model can be constructed using networks such as Faster R-CNN, SSD, and the YOLO series, without specific limitations. In this embodiment, bounding box regression is mainly used for skewed text detection; for example, it can be a one-stage regression or a two-stage regression.

[0068] The aforementioned preset conditions can be set according to actual needs. For example, the value of the loss function corresponding to the neural network model may be less than a preset threshold, i.e., the convergence condition has been met; or the number of iterations may meet a certain number, etc. There are no restrictions here.

[0069] It is understandable that a well-trained text detection model can predict bounding box information for text regions in unlabeled images. Each text string is enclosed by a rotating rectangle. The predicted bounding box information will include the first and second vertices of the rotating rectangle used to enclose the corresponding text string, namely (Xtop1, Ytop1) and (Xtop2, Ytop2), etc.

[0070] For step S110 above, exemplaryly, a trained text detection model is used to detect text bounding boxes on the input target image, outputting bounding box information for each text string in the image. Then, this bounding box information is used to calculate the text direction classification of the corresponding text string.

[0071] Step S120: Select a preset proportion of text bounding boxes, and calculate the text direction classification result of each selected text bounding box according to the coordinates of the two vertices of the rotating rectangle and the bounding box direction classification algorithm.

[0072] The preset ratio can be set according to actual needs. For example, it can be 80% or 90% of the total number of text boxes. If the number of boxes is small, all boxes can be used for calculation.

[0073] As an example, a certain number of text bounding boxes can be randomly selected, and then the text orientation classification of each selected text bounding box can be calculated according to a predefined bounding box orientation classification algorithm. It is understandable that, in some cases, selecting a certain proportion of bounding boxes for calculation can further reduce computational overhead and improve processing efficiency.

[0074] In one implementation, such as Figure 5 As shown, the text direction classification results for each selected text bounding box calculated according to the bounding box direction classification algorithm based on the coordinates of the two vertices of the rotated rectangle include:

[0075] Sub-step S310: Calculate the tilt angle of the rotating rectangle using the coordinates of the first and second vertices of the rotating rectangle corresponding to the tilted text; Sub-step S320: Determine the angle interval where the tilt angle is located based on multiple predefined preset directions and their corresponding specified angle intervals, and use the preset direction corresponding to the angle interval where the tilt angle is located as the direction classification result of the current rotating rectangle.

[0076] Exemplary, such asFigure 6 As shown in Figure 6 , for the above sub-step S310, it includes:

[0077] Sub-step S311, calculating a slope value according to the coordinates of the first vertex and the second vertex of the rotation rectangle by the slope formula.

[0078] Sub-step S312, selecting the corresponding angle calculation formula according to the comparison result of the abscissas of the first vertex and the second vertex of the rotation rectangle, and the comparison result of the ordinates of the first vertex and the second vertex.

[0079] Sub-step S313, calculating the tilt angle of the rotation rectangle according to the slope value and the selected angle calculation formula.

[0080] Combined with Figure 4 , according to the line connecting the first vertex (Xtop1, Ytop1) and the second vertex (Xtop2, Ytop2), the calculation formula for the slope value K is:

[0081]

[0082] Thus, if taking (Xtop1, Ytop1) as the origin, the included angle θ between the line connecting the above two vertices and the x-axis (passing through the point (xtop1, ytop1)) is the included angle between the rotation rectangle enclosing the text string and the X-axis. It can be understood that since the rotation rectangle in this embodiment is formed along the direction when the text string is rotated to be upright, therefore, the included angle between the rotation rectangle and the X-axis also reflects the tilt degree of the text string.

[0083] In one implementation manner, when selecting the corresponding angle calculation formula, the following rules can be adopted:

[0084] (1) When the abscissa of the first vertex is less than the abscissa of the second vertex, that is, satisfying (Xtop1 < Xtop2 & Ytop2 <= Ytop1) or (Xtop1 < Xtop2 & Ytop2 >= Ytop1), then select the first angle calculation formula. Exemplarily, the first angle calculation formula is:

[0085] θ = arctan(K) * 180 / π;

[0086] Where, θ is the tilt angle of the rotation rectangle; K is the calculated slope value.

[0087] (2) When the abscissa of the first vertex is greater than the abscissa of the second vertex, and the ordinate of the first vertex is less than or equal to the ordinate of the second vertex, that is, satisfying (xtop2 < xtop1 & ytop1 ≤ ytop2), then select the second angle calculation formula. Exemplarily, the second angle calculation formula is:

[0088] θ=180°+arctan(K)*180 / π.

[0089] (3) When the x-coordinate of the first vertex is greater than the x-coordinate of the second vertex, and the y-coordinate of the first vertex is greater than the y-coordinate of the second vertex, that is, when (xtop2) is satisfied.<xtop1&ytop1> If ytop2), then the third angle calculation formula is selected. For example, this third angle calculation formula is:

[0090] θ=-180°+arctan(K)*180 / π.

[0091] In addition to the above cases, if the x-coordinate of the first vertex is equal to the x-coordinate of the second vertex and the y-coordinate of the first vertex is greater than the y-coordinate of the second vertex, then the tilt angle of the rotated rectangle is 90°. Conversely, if the y-coordinate of the first vertex is less than the y-coordinate of the second vertex, then the tilt angle of the rotated rectangle is -90°.

[0092] Therefore, after determining the tilt angle of the rotating rectangle, the text direction classification is further determined.

[0093] In this embodiment, the aforementioned multiple preset directions may include four directions, namely the first direction to the fourth direction, and these four directions can cover all angles in the coordinate system established in the image. For example, in one implementation, the angle intervals corresponding to each of these four directions are defined as shown in Table 1 below.

[0094] Table 1

[0095]

[0096] The first direction, also known as the forward direction, corresponds to an angle range of (-45°, 45°); the second direction, also known as the rightward direction, corresponds to an angle range of (45°, 135°); the third direction, also known as the reverse direction, corresponds to an angle range of (-180°, -135°]∪(135°, 180°); and the fourth direction, also known as the leftward direction, corresponds to an angle range of (-135°, -45°). It can be understood that when any of the above directions is determined, different processing can be performed according to subsequent operations.

[0097] For example, taking subsequent text recognition as an example, if the direction classification is determined to be positive, such as Figure 7a As shown, in this case, no rectangular rotation is needed, and the text can be directly input into the text recognition model for text recognition. If it is determined to be right-handed, as... Figure 7b As shown, the current text image needs to be rotated 90 degrees counterclockwise so that the text in the image is now facing forward before being input into the model for text recognition. Similarly, if it is determined to be upside down, such as... Figure 7cAs shown, the text image needs to be rotated 180 degrees clockwise or counterclockwise first; the specific rotation can be set according to the actual situation. If it is determined to be right-handed, such as... Figure 7d As shown, the text image needs to be rotated 90 degrees clockwise before text recognition can be performed.

[0098] Step S130: Determine the final text direction classification of the image based on the text direction classification results.

[0099] For example, a voting mechanism can be used to select the text direction category with the highest percentage among all text direction categories as the final text direction category for the image containing tilted text.

[0100] For example, if there are 15 text direction classification results in total, and 12 of them are all determined to be the second direction mentioned above, while the remaining 3 are other directions, then the second direction with the largest proportion will be determined as the final direction classification result of the image.

[0101] Furthermore, after obtaining the final text orientation classification, corresponding subsequent processing can be performed according to actual needs. For example, when text recognition is required, the image can be rotated 90 degrees clockwise to ensure that the text orientation is correct before performing text recognition. Of course, it can also be used in other scenarios, which are not limited here.

[0102] The tilted text orientation classification method in this embodiment utilizes a deep learning model to detect the bounding box of tilted text, and the predicted bounding box is represented as a rotated rectangle. Then, the coordinate information of the two vertices of this rotated rectangle is used to determine the text orientation, thus obtaining the final text orientation classification result for the image. This method, based on the prediction results of the tilted text detection model combined with a post-processing algorithm, can directly classify the text orientation of an image, avoiding its implementation as an independent functional module (such as model granularity or task granularity), saving on model parameter overhead, training overhead, prediction overhead, and subsequent computational overhead related to orientation classification tasks.

[0103] Example 2

[0104] Please refer to Figure 8 Based on the method of Embodiment 1 above, this embodiment proposes a tilted text direction classification device 100, which, exemplary, includes:

[0105] The prediction output module 110 is used to input the image to be detected into a trained text detection model and predict and output the text bounding box information of the region where each text is located in the image. The text bounding box information includes the coordinates of the two vertices of a rotating rectangle used to enclose the corresponding text.

[0106] The orientation classification calculation module 120 is used to select a preset proportion of text bounding boxes, and calculate the text orientation classification result of each selected text bounding box according to the coordinates of the two vertices of the rotating rectangle and the bounding box orientation classification algorithm.

[0107] The orientation classification determination module 130 is used to determine the final text orientation classification of the image based on the text orientation classification results.

[0108] It is understood that the apparatus of this embodiment corresponds to the method of embodiment 1 above, and the options in embodiment 1 above are also applicable to this embodiment, so they will not be described again here.

[0109] This application also provides a terminal device, such as a computer, which, by way of example, includes a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to enable the terminal device to perform the functions of the various modules in the above-described tilted text orientation classification method or the above-described tilted text orientation classification device.

[0110] This application also provides a readable storage medium for storing the computer program used in the aforementioned terminal device.

[0111] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that, in alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0112] In addition, the functional modules or units in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0113] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a smartphone, personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0114] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for classifying slanted text orientation, characterized in that, The method comprises: inputting a picture to be detected into a trained inclined text detection model, and predicting output text bounding box information of a text region in the picture, the text bounding box information comprising two vertex coordinates of a rotated rectangle used for enclosing a corresponding text string; the two vertexes of the rotated rectangle are denoted as a first vertex and a second vertex, wherein the first vertex and the second vertex are defined as the left upper vertex and the right upper vertex of the rotated rectangle when the enclosed text is in a normal state; selecting a preset number of text bounding boxes, and calculating a text direction classification result of each selected text bounding box according to the coordinates of the two vertexes of the rotated rectangle and a bounding box direction classification algorithm; determining a final text direction classification of the picture according to the text direction classification results of the text bounding boxes; wherein the calculating of the text direction classification result of each selected text bounding box according to the coordinates of the two vertexes of the rotated rectangle and the bounding box direction classification algorithm comprises: calculating an inclination angle of the rotated rectangle using the coordinates of the first vertex and the second vertex of the rotated rectangle; determining an angle interval in which the inclination angle is located according to a plurality of preset directions and corresponding specified angle intervals, and taking a preset direction corresponding to the angle interval in which the inclination angle is located as the direction classification result of the rotated rectangle; wherein the calculating of the inclination angle of the rotated rectangle using the coordinates of the first vertex and the second vertex of the rotated rectangle comprises: calculating a slope value according to the coordinates of the first vertex and the second vertex of the rotated rectangle according to a slope formula; selecting a corresponding angle calculation formula according to the comparison results of the horizontal coordinates of the first vertex and the second vertex and the comparison results of the vertical coordinates of the first vertex and the second vertex; calculating the inclination angle of the rotated rectangle according to the slope value and the selected angle calculation formula; wherein the plurality of preset directions comprise a first direction to a fourth direction, wherein the corresponding specified angle intervals of the first direction to the fourth direction are (-45°, 45°], (45°, 135°], (-180°, -135°]∪(135°, 180°], and (-135°, -45°] in sequence.

2. The skewed text direction classification method of claim 1, wherein, the selecting of the corresponding angle calculation formula according to the comparison results of the horizontal coordinates of the first vertex and the second vertex and the comparison results of the vertical coordinates of the first vertex and the second vertex comprises: when the horizontal coordinate of the first vertex is less than the horizontal coordinate of the second vertex, a first angle calculation formula is selected, and the first angle calculation formula is: θ=arctan(K)* 180 / π; wherein θ is the inclination angle of the rotated rectangle; K is the slope value; when the horizontal coordinate of the first vertex is greater than the horizontal coordinate of the second vertex, and the vertical coordinate of the first vertex is less than or equal to the vertical coordinate of the second vertex, a second angle calculation formula is selected, and the second angle calculation formula is: θ=180°+arctan(K)* 180 / π; When the horizontal coordinate of the first vertex is greater than the horizontal coordinate of the second vertex, and the vertical coordinate of the first vertex is greater than the vertical coordinate of the second vertex, a third angle calculation formula is selected, and the third angle calculation formula is: θ = -180° + arctan(K) * 180 / π.

3. The skewed text direction classification method of claim 1, wherein, The pre-training process of the inclined text detection model comprises: a plurality of original pictures containing inclined text are obtained, and all text regions in each original picture are annotated in a text string unit according to a state in which the text in the picture is turned right to obtain a plurality of annotated training pictures; the plurality of annotated training pictures are rotated and data-augmented according to a plurality of preset directions to obtain a plurality of augmented pictures with updated annotations, and the plurality of augmented pictures and the plurality of training pictures constitute a training image set; the neural network model is trained for inclined text detection by using the training image set, and the training is stopped when the neural network model meets a preset convergence condition to obtain a trained inclined text detection model.

4. The skewed text direction classification method of claim 1, wherein, The final text direction classification of the picture is determined according to each of the text direction classification results, comprising: one of the text direction classification results with the highest proportion is taken as the final text direction classification of the picture through a voting mechanism.

5. A skewed text direction classification apparatus, characterized by, comprise: a prediction output module configured to input a to-be-detected picture into a trained text detection model and output text bounding box information of a region where each text in the picture is located, the text bounding box information comprising two vertex coordinates of a rotated rectangle used to enclose the corresponding text; the two vertices of the rotated rectangle are defined as a first vertex and a second vertex, wherein the first vertex and the second vertex are respectively defined as a left upper vertex and a right upper vertex of the rotated rectangle in a state in which the enclosed text is turned right; a direction classification calculation module configured to select a preset number of text bounding boxes and calculate text direction classification results of each of the selected text bounding boxes according to the coordinates of the two vertices of the rotated rectangle according to a bounding box direction classification algorithm; a direction classification determination module configured to determine a final text direction classification of the picture according to each of the text direction classification results; wherein the calculation of the text direction classification results of each of the selected text bounding boxes according to the coordinates of the two vertices of the rotated rectangle according to the bounding box direction classification algorithm comprises: calculating an inclination angle of the rotated rectangle by using the coordinates of the first vertex and the second vertex of the rotated rectangle; determining an angle interval in which the inclination angle is located according to a plurality of preset directions and respective specified angle intervals corresponding to the preset directions, and taking a preset direction corresponding to the angle interval in which the inclination angle is located as a direction classification result of the rotated rectangle; wherein the calculation of the inclination angle of the rotated rectangle by using the coordinates of the first vertex and the second vertex of the rotated rectangle comprises: calculating a slope value according to the coordinates of the first vertex and the second vertex of the rotated rectangle according to a slope formula; According to a comparison result of the horizontal coordinates of the first vertex and the second vertex of the rotated rectangle and a comparison result of the vertical coordinates of the first vertex and the second vertex, a corresponding angle calculation formula is selected; According to the slope value and the selected angle calculation formula, an inclination angle of the rotated rectangle is calculated; The plurality of preset directions include a first direction to a fourth direction, and the specified angle intervals corresponding to the first direction to the fourth direction are (-45°, 45°], (45°, 135°], (-180°, -135°]∪(135°, 180°], and (-135°, -45°] in sequence.

6. A terminal device, characterized by comprising: The terminal device comprises a processor and a memory, the memory stores a computer program, and the processor is configured to execute the computer program to implement the inclined text direction classification method in any one of claims 1-4.

7. A readable storage medium, characterized by, The computer program is stored in the memory and is configured to, when executed on the processor, implement the inclined text direction classification method in any one of claims 1-4.

Images