A method, system and device for recognizing skewed text lines

Abstract

The application relates to the technical field of text recognition methods, in particular to a slanted text line recognition method, system and device, which can solve the problem that slanted fonts in a text line cannot be accurately recognized when the text line is recognized. The slanted text line recognition method comprises the following steps: acquiring an image to be recognized; extracting multi-scale features of the image and decoding the multi-scale features to obtain decoded multi-scale features; obtaining shared features based on the decoded multi-scale features; obtaining a nine-channel feature map based on the shared features, and reading four vertex coordinates of a text box in the feature map; performing perspective transformation on the shared features based on the four vertex coordinates of the text box to obtain text region features of the feature map; obtaining a final recognized character string based on the text region features, and outputting the character string.

Description

Technical Field

[0001] This application relates to the field of text recognition methods, and more specifically, to a method, system, and device for recognizing slanted text lines. Background Technology

[0002] With the development of science and technology, a large amount of information is now recorded, edited, organized, stored, and transmitted using computers. As the amount of information increases, there is a need for effective methods to quickly and accurately input information into computers. Manually processing large amounts of text data is inefficient; using computers to automatically recognize text can solve this problem. Text recognition technology is a comprehensive technology involving a wide range of disciplines, including pattern recognition, digital signal processing, image processing, computer vision, and fuzzy mathematics. Currently, text line recognition is becoming increasingly important in the field of text recognition technology.

[0003] In the implementation of text line recognition, feature extraction is often performed using a single convolutional neural network, combined with a deep learning model for text line recognition. Deep learning is widely used in industrial vision, and its performance in complex scenes is significantly better than traditional image processing algorithms. Deep learning is data-driven, mapping images to a high-dimensional feature space, and then performing different processing according to different tasks.

[0004] However, since the fonts that form text in an image may be italic with a rotation angle or horizontal italic, a simple text line recognition process cannot effectively identify the text information in the image. The features extracted by a single convolutional neural network are limited and often cannot extract the direction information of the text, resulting in the inability to accurately identify the slanted fonts in the text line. Summary of the Invention

[0005] To address the problem of inaccurate identification of slanted text lines, this application provides a method, system, and device for slanted text line recognition.

[0006] The embodiments of this application are implemented as follows:

[0007] A first aspect of this application provides a method for recognizing slanted text lines, the method comprising:

[0008] Obtain the image to be identified, the image containing text lines;

[0009] Extract multi-scale features from the image and decode the multi-scale features to obtain decoded multi-scale features; and obtain shared features based on the decoded multi-scale features.

[0010] Based on the shared features, a nine-channel feature map is obtained, and the coordinates of the four vertices of the text box in the feature map are read; wherein, the first channel of the feature map is a fractional map, and the region with high confidence in the fractional map corresponds to the text line region in the image; the remaining eight channels of the feature map represent the coordinate values ​​of the four vertices of the text box clockwise respectively;

[0011] Based on the coordinates of the four vertices of the text box, a perspective transformation is performed on the shared feature channel by channel to obtain the text region feature in the feature map;

[0012] Based on the text region features, the final recognition string is obtained and output.

[0013] In some embodiments, the step of extracting multi-scale features of the image and decoding the multi-scale features to obtain the decoded multi-scale features further includes:

[0014] Extract the first scale feature, the second scale feature, the third scale feature and the fourth scale feature from the image; wherein the first scale, the second scale, the third scale and the fourth scale represent the proportion of the image size to the size of the image to be identified in the scale feature, respectively;

[0015] The first scale feature in the multi-scale features is used as a shared feature.

[0016] In some embodiments, the step of extracting multi-scale features of the image and decoding the multi-scale features to obtain the decoded multi-scale features further includes:

[0017] Extract the first scale feature, the second scale feature, the third scale feature and the fourth scale feature from the image; wherein the first scale, the second scale, the third scale and the fourth scale represent the proportion of the image size to the size of the image to be identified in the scale feature, respectively;

[0018] A first-scale feature map is generated based on the fusion of the decoded multi-scale features in the channel dimension. The first-scale feature map is a shared feature map.

[0019] In some embodiments, the step of obtaining a nine-channel feature map at a first scale based on shared features and reading the coordinates of the four vertices of the text box in the text line further includes:

[0020] The shared features are processed by convolutional layers to generate a first-scale feature map with nine channels. In the remaining eight channels of the feature map, regions with high confidence are represented by coordinate values. Regions with multiple pixel values ​​whose confidence is greater than a first threshold are represented as high-confidence regions, while the pixel values ​​of the remaining regions are 0.

[0021] In some embodiments, the step of performing perspective transformation on the shared feature channel by channel based on the coordinates of the four vertices of the text box further includes:

[0022] Starting from the top left corner of the text box, the vertices are sorted clockwise, and the coordinates of the four vertices are obtained sequentially.

[0023] A perspective transformation is performed on the coordinates of the four vertices. The coordinates of the four vertices after the perspective transformation are (0, 0), (w, 0), (w, h), and (0, h), respectively, where h is the first height value and w is the width value after scaling when the aspect ratio of the text box remains unchanged before and after the perspective transformation.

[0024] In some embodiments, the step of obtaining the final recognized string based on text region features further includes:

[0025] Based on text region features, feature sequences are generated through convolutional neural networks, recurrent neural networks, and fully connected layers to output a posterior probability matrix;

[0026] The final recognized string is obtained based on the posterior probability matrix.

[0027] A second aspect of this application provides a slanted text line recognition system, comprising:

[0028] An image acquisition unit is used to acquire an image to be recognized, the image containing text lines;

[0029] A shared feature acquisition unit is used to extract multi-scale features of the image and decode the multi-scale features to obtain the decoded multi-scale features; and based on the decoded multi-scale features, to acquire shared features.

[0030] A vertex coordinate unit is used to obtain a nine-channel feature map based on the shared features, and to read the coordinates of the four vertices of the text box in the feature map; wherein, the first channel of the feature map is a fractional map, and the region with high confidence in the fractional map corresponds to the text line region in the image; the remaining eight channels of the feature map represent the coordinate values ​​of the four vertices of the text box clockwise respectively;

[0031] A perspective transformation unit is used to perform perspective transformation on the shared features channel by channel based on the coordinates of the four vertices of the text box to obtain the text region features in the feature map;

[0032] The output posterior probability matrix unit is used to generate feature sequences based on text region features through convolutional neural networks, recurrent neural networks, and fully connected layers to output the posterior probability matrix.

[0033] String output unit: used to obtain the final recognized string based on the posterior probability matrix and output the string.

[0034] In some embodiments, in the step of extracting multi-scale features of the image and decoding the multi-scale features to obtain the decoded multi-scale features, the shared feature acquisition unit is further configured to:

[0035] Extract the first scale feature, the second scale feature, the third scale feature and the fourth scale feature from the image; wherein the first scale, the second scale, the third scale and the fourth scale represent the proportion of the image size to the size of the image to be identified in the scale feature, respectively;

[0036] The first scale feature in the multi-scale features is used as a shared feature.

[0037] In some embodiments, in the step of obtaining a nine-channel feature map at a first scale based on shared features and reading the coordinates of the four vertices of the text box in the text line, the vertex coordinate acquisition unit is further configured to:

[0038] The shared features are processed by convolutional layers to generate a first-scale feature map with nine channels. In the remaining eight channels of the feature map, regions with high confidence are represented by coordinate values. Regions with multiple pixel values ​​whose confidence is greater than a first threshold are represented as high-confidence regions, while the pixel values ​​of the remaining regions are 0.

[0039] A third aspect of this application provides a tilted text line recognition device, including: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other through the communication bus;

[0040] The memory is used to store at least one executable instruction that causes the processor to perform operations as described in any of the first aspects above for the slanted text line recognition method.

[0041] The beneficial effects of this application are as follows: by acquiring shared features and obtaining a nine-channel feature map based on the shared features, the coordinate values ​​of the four clockwise vertices of the text box can be obtained quickly, which can adapt to text lines of various shapes and help to accurately identify slanted text lines; furthermore, through perspective transformation, text region features can be obtained efficiently, which in turn helps to accurately identify text lines; furthermore, the process of obtaining the final recognized string based on the text region features is efficient and convenient, and the string can be accurately output. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a flowchart illustrating a method for recognizing slanted text lines according to one or more embodiments of this application;

[0044] Figure 2 This is a schematic diagram of the text content before and after operation of a tilted text line recognition system according to one or more embodiments of this application;

[0045] Figure 3 This is a schematic diagram of the structure of a slanted text line recognition system according to one or more embodiments of this application;

[0046] Figure 4 This is a schematic diagram of the structure of a tilted text line recognition device according to one or more embodiments of this application. Detailed Implementation

[0047] To make the objectives, implementation methods and advantages of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the described exemplary embodiments are only some embodiments of this application, and not all embodiments.

[0048] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0049] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0050] The terms “include” and “have”, and any variations thereof, are intended to cover but not exclusively include, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0051] Figure 1 This is a flowchart illustrating a method for recognizing slanted text lines.

[0052] like Figure 1 As shown, the method for recognizing slanted text lines includes the following steps:

[0053] In step 100, the image to be recognized is obtained, and the image contains a text line.

[0054] In step 200, multi-scale features of the image are extracted and decoded to obtain the decoded multi-scale features; and shared features are obtained based on the decoded multi-scale features.

[0055] Specifically, the first scale feature, second scale feature, third scale feature and fourth scale feature in the image are extracted through the mainstream backbone network. The first scale, second scale, third scale and fourth scale represent the proportion of the image size to the image size to be identified in the scale features.

[0056] In some embodiments, the first scale is 1 / 4, the second scale is 1 / 8, the third scale is 1 / 16, and the fourth scale is 1 / 32. Features are then decoded layer by layer to obtain the decoded [1 / 4, 1 / 8, 1 / 16, 1 / 32] multi-scale features. In some embodiments, the 1 / 4 scale features are directly used as shared features, which helps simplify the entire processing while ensuring the model has good expressive power. In other embodiments, the decoded [1 / 4, 1 / 8, 1 / 16, 1 / 32] multi-scale features are further fused along the channel dimension to generate a [1 / 4] scale feature map, obtaining shared features. In this case, 1 / 8, 1 / 16, and 1 / 32 are each upsampled to 1 / 4 and then concatenated with the 1 / 4 scale, resulting in a more complete shared feature after fusion.

[0057] In summary, extracting high-dimensional features from images by fusing mainstream backbone features with multi-scale features can improve the accuracy of the final text line recognition results.

[0058] In step 300, a nine-channel feature map is obtained based on shared features, and the coordinates of the four vertices of the text box in the feature map are read. The first channel of the feature map is a score map, and the region with high confidence in the score map corresponds to the text line region in the image. The remaining eight channels in the feature map represent the coordinate values ​​of the four vertices of the text box clockwise.

[0059] It should be noted that since the text box of a text line can be rectangular, trapezoidal, or parallelogram, based on shared features, a nine-channel [1 / 4] scale feature map is generated from the image through a convolutional layer to obtain the coordinates of the four vertices of the text box. In this [1 / 4] scale feature map, the first channel is a score map, where regions with high confidence represent the text line region, and a confidence value greater than 0.5 is generally considered high. The remaining eight channels represent the horizontal and vertical coordinates of the four vertices of the text box clockwise. These nine channels only represent coordinate values ​​in regions with high confidence in the score map; the pixel values ​​in other regions are 0. The process of obtaining the coordinates of the four vertices of the text box by dense prediction based on the shared features through detection box regression is simple and efficient.

[0060] In step 400, a perspective transformation is performed on the shared features channel by channel based on the coordinates of the four vertices of the text box to obtain the text region features. The perspective transformation specifically includes the following steps:

[0061] Starting from the top left corner of the text box, sort the vertices clockwise and obtain the coordinates of the four vertices in sequence;

[0062] A perspective transformation is performed on the coordinates of the four vertices. The coordinates of the four vertices after the perspective transformation are (0, 0), (w, 0), (w, h), and (0, h), respectively. Here, h is the first height value, and w is the width of the text box after scaling while keeping the aspect ratio unchanged before and after the perspective transformation.

[0063] It should be noted that the sampling is performed by performing perspective transformation and cropping features based on the coordinates of the four vertices of the text box. When the cropped features are used as text region features, the height of the text region features can be fixed at 8, that is, the first height value is 8. The width of the text box is scaled proportionally, which helps to perform sequence modeling on the basis of cropped shared features and accurately obtain the final text line recognition result.

[0064] It is understandable that during the deep learning training phase, the text needs to be labeled. At this time, the label serves as a reference ground truth to constrain the direction of deep learning optimization. Therefore, in order to facilitate accurate recognition in the future, the minimum area quadrilateral of the slanted text line needs to be labeled. At this time, the output of the deep learning model can also be the minimum area quadrilateral of the slanted text line, thereby ensuring that the perspective transformation here can correct its corresponding shared features to the features of the regular text.

[0065] In step 500, based on the text region features, the final recognized string is obtained and output, specifically including the following steps:

[0066] In step 510, based on the text region features, a feature sequence is generated through a convolutional neural network, a recurrent neural network, and a fully connected layer to output the posterior probability matrix;

[0067] It should be noted that the convolutional layer performs downsampling only three times in the height dimension, generating feature vectors with width and height scales of [1 / 4, 1 / 32], where the feature vector height is 1. Then, a posterior probability matrix is ​​output through Long Short-Term Memory (LSTM) and fully connected classification. The posterior probability matrix records the probability of different characters appearing at different times, serving as auxiliary information generated by the neural network recognition model during text recognition. The number of rows in the posterior probability matrix represents the total number of recognizable characters (with an additional placeholder), while the number of columns represents the width of the cropped text image after downsampling.

[0068] In step 520, based on the posterior probability matrix, a greedy search and deduplication operation are performed on the posterior probability matrix to obtain the final recognized string.

[0069] Understandably, since each column of the posterior probability matrix is ​​a one-hot vector, a greedy search is used to select the character corresponding to the maximum value in each column. Then, a deduplication operation removes duplicate characters and hyphens from the recognition process to obtain the accurate string. For example, during recognition, "state" might be recognized as "-st-aatte". The deduplication operation removes duplicate characters and hyphens to obtain the final recognized string.

[0070] It should be noted that, as Figure 2 As shown, when a slanted text line is identified within a text behavior box in an image, after the above-mentioned slanted text line recognition step, the output is the string above the text behavior box in the image. In addition to the string, the coordinates of the four vertices of the text box clockwise are also output for later use.

[0071] In summary, by sharing the feature layer, the text detection and text recognition tasks are bridged in the feature domain, accelerating the execution efficiency of deep learning character recognition. Secondly, the detection box regression directly obtains the coordinates of the four vertices of the text box, and then the perspective transformation corrects the tilted text line based on the vertex coordinates. Compared with other methods that predict the top left vertex, width, height, and rotation angle, this method is more adaptable to the tilted text line recognition method and has a wider range of applications.

[0072] Secondly, this application also discloses a slanted text line recognition system, such as... Figure 3 As shown, the slanted text line recognition system includes:

[0073] The image acquisition unit is used to acquire the image to be recognized, which contains text lines.

[0074] A shared feature unit is obtained to extract and decode multi-scale features of an image, thereby obtaining the decoded multi-scale features; and based on the decoded multi-scale features, shared features are obtained.

[0075] The vertex coordinate unit is used to obtain a nine-channel feature map based on shared features, and to read the coordinates of the four vertices of the text box in the feature map. The first channel of the feature map is a score map, and the region with high confidence in the score map corresponds to the text line region in the image. The remaining eight channels of the feature map represent the coordinate values ​​of the four vertices of the text box clockwise.

[0076] The perspective transformation unit is used to perform perspective transformation on the shared features channel by channel based on the coordinates of the four vertices of the text box to obtain the text region features in the feature map.

[0077] The output posterior probability matrix unit is used to generate feature sequences based on text region features through convolutional neural networks, recurrent neural networks, and fully connected layers to output the posterior probability matrix.

[0078] String output unit: Used to obtain the final recognized string based on the posterior probability matrix and output the string.

[0079] In some embodiments, in the step of extracting multi-scale features of an image and decoding the multi-scale features to obtain the decoded multi-scale features, the shared feature unit is further used for:

[0080] First-scale features, second-scale features, third-scale features, and fourth-scale features are extracted from the image; where the first scale, second scale, third scale, and fourth scale represent the proportion of the image size to the image size to be identified in the scale features, respectively; the first-scale feature in the multi-scale features is used as a shared feature. In some embodiments, a first-scale feature map is generated based on the fusion of the decoded multi-scale features in the channel dimension, and the first-scale feature map is a shared feature.

[0081] In some embodiments, in the steps of obtaining a nine-channel feature map at a first scale based on shared features and reading the coordinates of the four vertices of the text boxes in the text line, the vertex coordinate acquisition unit is further used for:

[0082] The shared features are processed by convolutional layers to generate a nine-channel first-scale feature map. In the remaining eight channels of the feature map, regions with high confidence are represented by coordinate values. Among them, regions with multiple pixel values ​​with confidence greater than the first threshold are represented as high-confidence regions, while the pixel values ​​of the remaining regions are 0.

[0083] In some embodiments, in the step of performing perspective transformation on the shared features channel by channel based on the coordinates of the four vertices of the text box, the perspective transformation unit is further used for:

[0084] Starting from the top left corner of the text box, sort the vertices clockwise and obtain the coordinates of the four vertices in sequence;

[0085] A perspective transformation is performed on the coordinates of the four vertices. The coordinates of the four vertices after the perspective transformation are (0, 0), (w, 0), (w, h), and (0, h), respectively. Here, h is the first height value, and w is the width of the text box after scaling while keeping the aspect ratio unchanged before and after the perspective transformation.

[0086] Thirdly, this application also discloses a slanted text line recognition device, such as... Figure 4 As shown, it includes a processor, memory, communication interface and communication bus, and the processor, memory and communication interface communicate with each other through the communication bus;

[0087] The memory is used to store at least one executable instruction that causes the processor to perform the following operation of the slanted text line recognition method.

[0088] Obtain the image to be recognized, which contains lines of text;

[0089] Extract and decode multi-scale features from the image to obtain the decoded multi-scale features; and obtain shared features based on the decoded multi-scale features.

[0090] Based on shared features, a nine-channel feature map is obtained, and the coordinates of the four vertices of the text box in the feature map are read. The first channel of the feature map is a score map, and the region with high confidence in the score map corresponds to the text line region in the image. The remaining eight channels of the feature map represent the coordinate values ​​of the four vertices of the text box clockwise.

[0091] Based on the coordinates of the four vertices of the text box, a perspective transformation is performed on the shared features channel by channel to obtain the text region features in the feature map;

[0092] Based on text region features, the final recognized string is obtained and output.

[0093] In some embodiments, in the steps of extracting and decoding multi-scale features of an image to obtain decoded multi-scale features, a first scale feature, a second scale feature, a third scale feature, and a fourth scale feature are extracted from the image; wherein the first scale, the second scale, the third scale, and the fourth scale represent the proportion of the image size to the size of the image to be identified in the scale features; and the first scale feature in the multi-scale features is used as a shared feature.

[0094] In some embodiments, in the steps of extracting and decoding multi-scale features of an image to obtain decoded multi-scale features, a first scale feature, a second scale feature, a third scale feature, and a fourth scale feature are extracted from the image; wherein, the first scale, the second scale, the third scale, and the fourth scale represent the proportion of the image size to the size of the image to be identified in the scale features; a first scale feature map is generated based on the fusion of the decoded multi-scale features in the channel dimension, and the first scale feature map is a shared feature.

[0095] In some embodiments, in the steps of obtaining a nine-channel feature map of the first scale based on shared features and reading the coordinates of the four vertices of the text box in the text line, the shared features are processed by a convolutional layer to generate a nine-channel first-scale feature map. In the remaining eight channels of the feature map, regions with high confidence represent coordinate values. Regions including multiple pixel values ​​with confidence greater than a first threshold are represented as regions with high confidence, while the pixel values ​​of the remaining regions are 0.

[0096] In some embodiments, in the step of performing perspective transformation on the shared features channel by channel based on the coordinates of the four vertices of the text box, the vertices are sorted clockwise starting from the top left corner of the text box, and the coordinates of the four vertices are obtained in sequence; perspective transformation is performed on the coordinates of the four vertices, and the coordinates of the four vertices after perspective transformation are (0, 0), (w, 0), (w, h), and (0, h), respectively, where h is the first height value, and w is the value of the width scaled when the aspect ratio of the text box remains unchanged before and after the perspective transformation.

[0097] In some embodiments, in the step of obtaining the final recognition string based on text region features, a feature sequence is generated by using a convolutional neural network, a recurrent neural network, and a fully connected layer based on the text region features to output a posterior probability matrix; and the final recognition string is obtained based on the posterior probability matrix.

[0098] In summary, the various modules in the entire slanted text line recognition device work closely together to achieve efficient and accurate recognition of slanted text lines.

[0099] Fourthly, this application also discloses a computer-readable storage medium storing at least one executable instruction that, when executed on a computer, causes the slanted text line recognition device to perform the following slanted text line recognition method:

[0100] Obtain the image to be recognized, which contains lines of text;

[0101] Extract and decode multi-scale features from the image to obtain the decoded multi-scale features; and obtain shared features based on the decoded multi-scale features.

[0102] Based on shared features, a nine-channel feature map is obtained, and the coordinates of the four vertices of the text box in the feature map are read. The first channel of the feature map is a score map, and the region with high confidence in the score map corresponds to the text line region in the image. The remaining eight channels of the feature map represent the coordinate values ​​of the four vertices of the text box clockwise.

[0103] Based on the coordinates of the four vertices of the text box, a perspective transformation is performed on the shared features channel by channel to obtain the text region features in the feature map;

[0104] Based on text region features, the final recognized string is obtained and output.

[0105] In some embodiments, in the steps of extracting and decoding multi-scale features of an image to obtain decoded multi-scale features, a first scale feature, a second scale feature, a third scale feature, and a fourth scale feature are extracted from the image; wherein the first scale, the second scale, the third scale, and the fourth scale represent the proportion of the image size to the size of the image to be identified in the scale features; and the first scale feature in the multi-scale features is used as a shared feature.

[0106] In some embodiments, in the steps of extracting and decoding multi-scale features of an image to obtain decoded multi-scale features, a first scale feature, a second scale feature, a third scale feature, and a fourth scale feature are extracted from the image; wherein, the first scale, the second scale, the third scale, and the fourth scale represent the proportion of the image size to the size of the image to be identified in the scale features; a first scale feature map is generated based on the fusion of the decoded multi-scale features in the channel dimension, and the first scale feature map is a shared feature.

[0107] In some embodiments, in the steps of obtaining a nine-channel feature map of the first scale based on shared features and reading the coordinates of the four vertices of the text box in the text line, the shared features are processed by a convolutional layer to generate a nine-channel first-scale feature map. In the remaining eight channels of the feature map, regions with high confidence represent coordinate values. Regions including multiple pixel values ​​with confidence greater than a first threshold are represented as regions with high confidence, while the pixel values ​​of the remaining regions are 0.

[0108] In some embodiments, in the step of performing perspective transformation on the shared features channel by channel based on the coordinates of the four vertices of the text box...

[0109] Starting from the top left corner of the text box, sort the vertices clockwise and obtain the coordinates of the four vertices in sequence. Perform a perspective transformation on the four vertex coordinates. The coordinates of the four vertices after the perspective transformation are (0, 0), (w, 0), (w, h), and (0, h), where h is the first height value and w is the width value after scaling when the aspect ratio of the text box remains unchanged before and after the perspective transformation.

[0110] In some embodiments, in the step of obtaining the final recognition string based on text region features, a feature sequence is generated by using a convolutional neural network, a recurrent neural network, and a fully connected layer based on the text region features to output a posterior probability matrix; and the final recognition string is obtained based on the posterior probability matrix.

[0111] The beneficial effect of this embodiment is that it extracts high-dimensional features of the image by fusing mainstream backbone and multi-scale features, and then performs dense prediction on the basis of shared features by detection box regression to obtain the coordinates of the four vertices of the text box. It then performs perspective transformation to crop the features according to the coordinates of the four vertices of the text box, and then performs sequence modeling on the cropped features to obtain the final text line recognition result. This makes it possible to accurately recognize slanted text lines, whether they are horizontal italics or italics with rotation angles. Moreover, the entire slanted text line recognition method has a wide range of applications.

[0112] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0113] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0114] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for recognizing slanted text lines, characterized in that, The method includes: Obtain the image to be identified, the image containing text lines; Extract multi-scale features from the image and decode the multi-scale features to obtain decoded multi-scale features; and obtain shared features based on the decoded multi-scale features. Based on the shared features, a nine-channel feature map is obtained, and the coordinates of the four vertices of the text box in the feature map are read; wherein, the first channel of the feature map is a fractional map, and the region with high confidence in the fractional map corresponds to the text line region in the image; the remaining eight channels of the feature map represent the coordinate values ​​of the four vertices of the text box clockwise respectively; Based on the coordinates of the four vertices of the text box, a perspective transformation is performed on the shared features channel by channel to obtain the text region features; Based on text region features, the final recognition string is obtained and output. The step of obtaining a nine-channel feature map based on the shared features and reading the coordinates of the four vertices of the text box in the feature map further includes: The shared features are processed by convolutional layers to generate a nine-channel first-scale feature map. In the remaining eight channels of the feature map, regions with high confidence represent coordinate values. Regions including multiple pixel values ​​with confidence greater than a first threshold are represented as high-confidence regions, while the pixel values ​​of the remaining regions are 0. The step of performing perspective transformation on the shared feature channel by channel based on the coordinates of the four vertices of the text box further includes: Starting from the top left corner of the text box, the vertices are sorted clockwise, and the coordinates of the four vertices are obtained sequentially. A perspective transformation is performed on the coordinates of the four vertices. The coordinates of the four vertices after the perspective transformation are (0, 0), (w, 0), (w, h), and (0, h), respectively, where h is the first height value and w is the width value after scaling when the aspect ratio of the text box remains unchanged before and after the perspective transformation.

2. The method for recognizing slanted text lines as described in claim 1, characterized in that, The step of extracting multi-scale features from the image and decoding the multi-scale features to obtain the decoded multi-scale features further includes: Extract the first scale feature, the second scale feature, the third scale feature and the fourth scale feature from the image; wherein the first scale, the second scale, the third scale and the fourth scale represent the proportion of the image size to the size of the image to be identified in the scale feature, respectively; The first scale feature in the multi-scale features is used as a shared feature.

3. The method for recognizing slanted text lines as described in claim 1, characterized in that, The step of extracting multi-scale features from the image and decoding the multi-scale features to obtain the decoded multi-scale features further includes: Extract the first scale feature, the second scale feature, the third scale feature and the fourth scale feature from the image; wherein the first scale, the second scale, the third scale and the fourth scale represent the proportion of the image size to the size of the image to be identified in the scale feature, respectively; A first-scale feature map is generated by fusing the decoded multi-scale features in the channel dimension. The first-scale feature map is a shared feature map.

4. The method for recognizing slanted text lines as described in claim 1, characterized in that, The step of obtaining the final recognized string based on text region features further includes: Based on text region features, feature sequences are generated through convolutional neural networks, recurrent neural networks, and fully connected layers to output a posterior probability matrix; The final recognized string is obtained based on the posterior probability matrix.

5. A system for recognizing slanted text lines, characterized in that, include: An image acquisition unit is used to acquire an image to be recognized, the image containing text lines; A shared feature unit is used to extract multi-scale features of the image and decode the multi-scale features to obtain the decoded multi-scale features. And based on the decoded multi-scale features, shared features are obtained; A vertex coordinate unit is used to obtain a nine-channel feature map based on the shared features, and to read the coordinates of the four vertices of the text box in the feature map; wherein, the first channel of the feature map is a fractional map, and the region with high confidence in the fractional map corresponds to the text line region in the image; the remaining eight channels of the feature map represent the coordinate values ​​of the four vertices of the text box clockwise respectively; A perspective transformation unit is used to perform perspective transformation on the shared features channel by channel based on the coordinates of the four vertices of the text box to obtain text region features; The output posterior probability matrix unit is used to generate feature sequences through convolutional neural networks, recurrent neural networks, and fully connected layers to output the posterior probability matrix. String output unit: used to obtain the final recognized string based on the posterior probability matrix and output the string; In the step of obtaining a nine-channel feature map based on the shared features and reading the coordinates of the four vertices of the text box in the feature map, the vertex coordinate acquisition unit is further used for: The shared features are processed by convolutional layers to generate a first-scale feature map with nine channels. In the remaining eight channels of the feature map, regions with high confidence are represented by coordinate values. Regions with multiple pixel values ​​whose confidence is greater than a first threshold are represented as high-confidence regions, while the pixel values ​​of the remaining regions are 0.

6. The slanted text line recognition system as described in claim 5, characterized in that, In the steps of extracting and decoding the multi-scale features of the image to obtain the decoded multi-scale features, the shared feature acquisition unit is further configured to: Extract the first scale feature, the second scale feature, the third scale feature and the fourth scale feature from the image; wherein the first scale, the second scale, the third scale and the fourth scale represent the proportion of the image size to the size of the image to be identified in the scale feature, respectively; The first scale feature in the multi-scale features is used as a shared feature.

7. A device for recognizing slanted text lines, characterized in that, It includes a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other through the communication bus; The memory is used to store at least one executable instruction that causes the processor to perform the operation of the slanted text line recognition method as described in any one of claims 1-4.

Images