Text line recognition method, device, storage medium and electronic device

Abstract

The present disclosure relates to a text line recognition method, device, storage medium, and electronic device, and relates to the field of image recognition technology. The method comprises: obtaining a text line image to be recognized; converting the text line image into a binary image; obtaining a binary image rectangular frame in the binary image, and a pending text line frame corresponding to the binary image rectangular frame in the text line image; removing redundant information in the pending text line frame based on the binary image rectangular frame to obtain a target text line frame; the target text line frame contains only one line of text; and determining text line information corresponding to printed text in the text line image based on the target text line frame. In this way, it can be ensured that each target text line frame contains only one line of text, so that when text recognition is performed based on the target text line frame, the text line information of the corresponding printed text can be accurately recognized.

Description

Technical Field

[0001] The present disclosure relates to the field of image recognition technology, and in particular, to a text line recognition method, device, storage medium, and electronic device. Background Art

[0002] In educational scenarios, it's often necessary to identify paper exam papers to obtain electronic versions. In some cases, paper exam papers may also contain handwritten text, necessitating the recognition of printed text. However, due to the complexity of the photography environment, the text in the captured image often appears curved, meaning the paper surface is uneven. This can affect the accuracy of printed text recognition. Summary of the Invention

[0003] In order to solve the above technical problems, the present disclosure provides a text line recognition method, device, storage medium and electronic device.

[0004] According to a first aspect of an embodiment of the present disclosure, a method for recognizing text lines is provided, the method comprising: obtaining a text line image to be recognized; converting the text line image into a binary image; obtaining a binary image rectangular frame in the binary image, and a pending text line frame corresponding to the binary image rectangular frame in the text line image; based on the binary image rectangular frame, removing redundant information in the pending text line frame to obtain a target text line frame; the target text line frame contains only one line of text; and determining text line information corresponding to the printed text in the text line image based on the target text line frame.

[0005] Optionally, the method further includes: determining the number of connected domains contained in the binary image rectangular frame; removing redundant information in the pending text line frame based on the binary image rectangular frame to obtain the target text line frame includes: when the number of connected domains is greater than or equal to a preset number threshold, removing redundant information in the pending text line frame based on the binary image rectangular frame to obtain the target text line frame.

[0006] Optionally, the method further includes: when the number of the connected domains is less than the preset number threshold, using the pending text line box as the target text line box.

[0007] Optionally, removing redundant information in the pending text line box based on the binary image rectangular box to obtain the target text line box includes: expanding the binary image rectangular box; covering the pending text line box according to the background part in the expanded binary image rectangular box to obtain the target text line box.

[0008] Optionally, obtaining the binary image rectangular frame in the binary image and the pending text line frame corresponding to the binary image rectangular frame in the text line image includes: obtaining the minimum circumscribed rectangle of each text line in the binary image to obtain the binary image rectangular frame; and obtaining the pending text line frame from the text line image according to the position of the binary image rectangular frame in the binary image.

[0009] Optionally, converting the text line image into a binary image includes: inputting the text line image into a pre-generated text line detection model to obtain the binary image output by the text line detection model.

[0010] Optionally, the text line detection model includes: a feature sampling module, a channel attention module, a feature fusion module and a processing module; the feature sampling module is used to downsample the text line image through multiple downsampling channels to obtain a multi-scale feature map; the channel attention module is used to weight feature maps of different scales; the feature fusion module is used to perform feature fusion on multiple weighted feature maps to obtain a target feature map; the processing module is used to convert the target feature map into the binary image.

[0011] Optionally, the channel attention module includes: a pooling submodule, a fully connected layer and a weighted submodule; the pooling submodule is used to perform global average pooling on the feature map; the fully connected layer is used to predict the weight of each downsampling channel based on the feature map after global average pooling; the weighted submodule is used to multiply the feature map corresponding to the downsampling channel by the weight corresponding to the downsampling channel for each downsampling channel to obtain a weighted feature map.

[0012] Optionally, the downsampling channel includes: a 4-fold downsampling channel, an 8-fold downsampling channel, and a 16-fold downsampling channel.

[0013] Optionally, the text line detection model is trained in the following manner: obtaining a sample image and a text line label corresponding to the sample image; and training a preset training model based on the sample image and the text line label to obtain the text line detection model.

[0014] Optionally, the sample image is obtained in the following manner: obtaining a target character string from a preset corpus; rendering the target character string to obtain a sample text line image containing the target character string; obtaining a sample background image; the sample background image includes at least one or more of a handwritten text line image, a formula line image, a watermark image, and an illustration; and obtaining the sample image based on the sample text line image and the sample background image.

[0015] According to a second aspect of an embodiment of the present disclosure, a device for identifying a text line is provided, the device comprising:

[0016] A first acquisition module is used to acquire a text line image to be recognized;

[0017] A conversion module, configured to convert the text line image into a binary image;

[0018] A second acquisition module is configured to acquire a binary image rectangular frame in the binary image and a pending text line frame corresponding to the binary image rectangular frame in the text line image;

[0019] a removal module, configured to remove redundant information in the pending text line frame based on the binary image rectangular frame to obtain a target text line frame; the target text line frame only contains one line of text;

[0020] A determination module is used to determine text line information corresponding to the printed text in the text line image according to the target text line frame.

[0021] Optionally, the determining module is further configured to determine the number of connected domains contained in the rectangular frame of the binary image;

[0022] The removal module is configured to remove redundant information in the pending text line frame based on the binary image rectangular frame to obtain the target text line frame when the number of connected domains is greater than or equal to a preset number threshold.

[0023] Optionally, the device further comprises:

[0024] A generating module is configured to use the pending text line frame as the target text line frame when the number of the connected domains is less than the preset number threshold.

[0025] Optionally, the removal module is configured to expand the binary image rectangular frame; and cover the undetermined text line frame according to the background portion of the expanded binary image rectangular frame to obtain the target text line frame.

[0026] Optionally, the second acquisition module is used to obtain the minimum enclosing rectangle of each text line in the binary image to obtain the binary image rectangular frame; and obtain the pending text line frame from the text line image according to the position of the binary image rectangular frame in the binary image.

[0027] Optionally, the conversion module is used to input the text line image into a pre-generated text line detection model to obtain the binary image output by the text line detection model.

[0028] Optionally, the text line detection model includes: a feature sampling module, a channel attention module, a feature fusion module and a processing module;

[0029] The feature sampling module is used to downsample the text line image through multiple downsampling channels to obtain a multi-scale feature map;

[0030] The channel attention module is used to weight feature maps of different scales;

[0031] The feature fusion module is used to perform feature fusion on multiple weighted feature maps to obtain a target feature map;

[0032] The processing module is used to convert the target feature map into the binary image.

[0033] Optionally, the channel attention module includes: a pooling submodule, a fully connected layer and a weighted submodule;

[0034] The pooling submodule is used to perform global average pooling processing on the feature map;

[0035] The fully connected layer is used to predict the weight of each down-sampled channel based on the feature map after global average pooling;

[0036] The weighting submodule is used to multiply the feature map corresponding to each downsampling channel by the weight corresponding to the downsampling channel to obtain a weighted feature map.

[0037] Optionally, the downsampling channel includes: a 4-fold downsampling channel, an 8-fold downsampling channel, and a 16-fold downsampling channel.

[0038] Optionally, the text line detection model is trained in the following manner:

[0039] Obtaining a sample image and a text line label corresponding to the sample image;

[0040] A preset training model is trained according to the sample images and the text line labels to obtain the text line detection model.

[0041] Optionally, the sample image is obtained by:

[0042] Get the target string from the preset corpus;

[0043] Rendering the target character string to obtain a sample text line image containing the target character string;

[0044] Acquire a sample background image; the sample background image includes at least one or more of a handwritten text line image, a formula line image, a watermark image, and an illustration;

[0045] The sample image is obtained according to the sample text line image and the sample background image.

[0046] According to a third aspect of an embodiment of the present disclosure, a non-temporary computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the steps of the text line recognition method provided by the first aspect of the present disclosure are implemented.

[0047] According to a fourth aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a memory on which a computer program is stored; and a processor for executing the computer program in the memory to implement the steps of the text line recognition method provided in the first aspect of the present disclosure.

[0048] According to the above technical solution, first, a text line image to be recognized is obtained. Secondly, the text line image is converted into a binary image. Then, a binary rectangular frame in the binary image and a pending text line frame corresponding to the binary rectangular frame in the text line image are obtained. Based on the binary rectangular frame, redundant information in the pending text line frame is removed to obtain a target text line frame; the target text line frame contains only one line of text. Finally, based on the target text line frame, the text line information corresponding to the printed text in the text line image is determined. According to the above method, redundant information in the pending text line frame can be removed using the binary rectangular frame in the binary image to obtain a target text line frame containing only one line of text. Subsequently, the text line information corresponding to the printed text in the text line image can be determined based on the target text line frame. This ensures that each target text line frame contains only one line of text. Therefore, when performing text recognition based on the target text line frame, the text line information of the corresponding printed text can be accurately identified, solving the problem of multiple lines of text in a target text line frame affecting text recognition accuracy.

[0049] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following detailed description, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:

[0051] Figure 1 is a flowchart of a method for recognizing a text line according to an exemplary embodiment;

[0052] Figure 2 is a flow chart of another text line recognition method according to an exemplary embodiment;

[0053] Figure 3 is a block diagram of a device for identifying a text line according to an exemplary embodiment;

[0054] Figure 4 is a block diagram of another text line recognition device according to an exemplary embodiment;

[0055] Figure 5 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0056] The following describes the specific embodiments of the present disclosure in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present disclosure and are not intended to limit the present disclosure.

[0057] The terms "first," "second," and the like in the specification and claims of this application and the accompanying drawings are used to distinguish similar objects and are not necessarily to be construed as implying a particular order or precedence. In addition, in the description with reference to the accompanying drawings, the same reference numerals in different drawings represent the same elements.

[0058] In the description of the present disclosure, unless otherwise specified, "multiple" means two or more than two, and other quantifiers are similar thereto; "at least one item(s)", "one item(s) or multiple items(s)" or similar expressions refer to any combination of these items(s), including any combination of single items(s) or plural items(s). For example, at least one item(s) a can represent any number of a's; for another example, one item(s) or multiple items(s) among a, b and c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple; "and / or" is a type of description of the association relationship of associated objects, indicating that three relationships can exist, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " indicates that the associated objects before and after are in an "or" relationship.

[0059] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0060] Figure 1 is a flowchart of a method for recognizing a text line according to an exemplary embodiment. Figure 1 As shown, the method may include the following steps:

[0061] In step S101 , a text line image to be recognized is obtained.

[0062] The text line image may be an image obtained by the user through photography or scanning, and may include both printed text and handwritten text. In some scenarios, the text line image may include an image of a test paper, and the user may obtain an electronic version of the test paper through the text line image. In some cases, it may be necessary to recognize a student's completed test paper. In this case, it is necessary to accurately recognize the printed text in the test paper and filter out the handwritten part. Therefore, in this embodiment, the recognition is mainly focused on the printed text in the text line image.

[0063] In step S102 , the text line image is converted into a binary image.

[0064] In this step, the text line image can be input into a pre-generated text line detection model to obtain the binary image (English: Binary Image; it means that each pixel on the image has only two possible values ​​or grayscale levels) output by the text line detection model.

[0065] The text line detection model may include, but is not limited to, network structures such as DBNet, ContourNet, PANNet, PSENet, and Craft. Furthermore, considering that in the context of test paper image recognition, test papers often contain densely packed printed text lines, which are prone to adhesion issues, in this embodiment, DBNet is preferably used as the network structure of the text line detection model.

[0066] The text line detection model can identify the binary image containing printed text corresponding to the text line image. In this binary image, the printed text can be clearly distinguished from the background, thus facilitating subsequent recognition of the printed text.

[0067] In step S103 , a binary image rectangular frame in the binary image and a pending text line frame corresponding to the binary image rectangular frame in the text line image are obtained.

[0068] Due to the diversity of shooting scenes, there may be uneven paper surfaces, which directly affects the distortion of text lines in the captured text line images. Printed text lines often have the following characteristics: the font size is usually small, and the font size is generally the same within the line. It is suitable to determine the text position of the text line by taking the minimum bounding rectangle operation. Therefore, in the process of further text recognition on the binary image, it is often necessary to take the minimum bounding rectangle for each text line (the minimum bounding rectangle refers to the maximum range of several two-dimensional shapes (such as points, lines, polygons) represented by two-dimensional coordinates, that is, the rectangle with the maximum horizontal coordinate, minimum horizontal coordinate, maximum vertical coordinate, and minimum vertical coordinate of each vertex of a given two-dimensional shape) and then recognize the printed text in the text line image based on multiple minimum bounding rectangles.

[0069] However, if the text lines are distorted, the context text may be included in the minimum bounding rectangle due to the distortion. This will cause excessive redundant information in the minimum bounding rectangle during text recognition, which can easily lead to recognition errors.

[0070] To improve recognition accuracy, in this embodiment, the binary image rectangle and the pending text line frame corresponding to the binary image rectangle in the text line image are first obtained. Redundant information in the pending text line frame is then removed based on the binary image rectangle, ensuring that each pending text line frame contains only one line of text.

[0071] For example, in this embodiment, the minimum bounding rectangle of each text line in the binary image can be obtained to obtain the binary image rectangular frame. Subsequently, the undetermined text line frame can be obtained from the text line image based on the position of the binary image rectangular frame in the binary image. In other words, the rectangular frame in the text line image located at the same position as the binary image rectangular frame is used as the undetermined text line frame.

[0072] In step S104, based on the binary image rectangular frame, redundant information in the pending text line frame is removed to obtain a target text line frame.

[0073] The target text line box contains only one line of text.

[0074] For example, the binary image rectangular frame can be expanded first. Specifically, the foreground part of the binary image rectangular frame can be expanded, that is, the part containing the printed text can be expanded. Afterwards, the pending text line frame is masked based on the background part in the expanded binary image rectangular frame to obtain the target text line frame. Wherein, when masking, the average RGB value of the area in the original text line image, or other specified colors, can be selected, and the present disclosure does not make specific restrictions on this. After masking, the target text line frame will only contain one line of text, and the remaining redundant information of the context will be covered by the background part in the binary image rectangular frame. In this way, by masking, the redundant information in the pending text line frame can be removed, and the processing method is simple, time-saving, and can effectively improve the accuracy of text recognition and effectively improve the robustness of curved text recognition.

[0075] In step S105 , text line information corresponding to the printed text in the text line image is determined according to the target text line frame.

[0076] After obtaining the target text line frame, the printed text in the text line image can be recognized using a related text recognition model to obtain text line information corresponding to the printed text in the text line image. The text recognition method can be a method well known to those skilled in the art and is not described in detail herein.

[0077] Using this method, redundant information within the undetermined text line frame can be removed using the binary rectangular frame in the binary image to obtain a target text line frame containing only one line of text. The target text line frame can then be used to determine the text line information corresponding to the printed text in the text line image. This ensures that each target text line frame contains only one line of text. Therefore, when performing text recognition based on the target text line frame, the corresponding printed text line information can be accurately identified, resolving the issue of multiple lines of text within a single target text line frame affecting text recognition accuracy.

[0078] In some scenarios, if the above processing is performed on each undetermined text line box in the text line image, this step is redundant for straight text and will cause unnecessary time consumption. Therefore, in order to improve the processing efficiency, Figure 2 As shown, the method may further include the following steps:

[0079] In step S106 , the number of connected components contained in the rectangular frame of the binary image is determined.

[0080] The connected domain is used to represent the area composed of pixels with the same pixel value and adjacent positions in the binary image rectangle. The number of connected domains can reflect the number of text lines contained in the binary image rectangle.

[0081] Generally speaking, if the text lines are parallel, then the binary image rectangle should contain a black background and a white strip (assuming black represents the background and white represents the text line). If the text line is curved enough to allow other lines to be clipped in, then the binary image rectangle will not only have one white line, but also white areas of other lines that have been clipped in. Therefore, the number of connected domains in the binary image rectangle can be used to determine whether the binary image rectangle contains curved text with redundant information.

[0082] Correspondingly, in the above step S104, based on the binary image rectangular frame, redundant information in the pending text line frame is removed to obtain the target text line frame, including: when the number of connected domains is greater than or equal to a preset number threshold, based on the binary image rectangular frame, redundant information in the pending text line frame is removed to obtain the target text line frame.

[0083] For example, if the number of connected domains is greater than or equal to a preset threshold, it can be considered that the binary image rectangle contains redundant information. Based on the binary image rectangle, the redundant information in the undetermined text line box can then be removed to obtain the target text line box. The preset threshold can be pre-set based on actual needs and is not specifically limited in this disclosure.

[0084] In addition, when the number of the connected domains is less than the preset number threshold, the undetermined text line frame is used as the target text line frame.

[0085] In this step, if the number of connected domains is less than the preset threshold, it can be considered that the binary image rectangle frame does not contain redundant information, and it can be omitted and the pending text line frame can be directly used as the target text line frame.

[0086] The model structure of the above-mentioned text line detection model is described in detail below. In this embodiment, the text line detection model may include: a feature sampling module, a channel attention module, a feature fusion module and a processing module.

[0087] Because test paper images often have long text lines with a large aspect ratio, the shrinkage ratio needs to be adjusted to a proportion suitable for long text strips when the model is indented. The shrinkage ratio is A(1-r^2) / L, where A is the area, L is the perimeter, and r is an adjustable parameter ranging from 0 to 1. The closer r is to 0, the greater the shrinkage ratio. The value of r can be adjusted as needed. For example, r can be set to 0.7.

[0088] The feature sampling module is used to downsample the text line image through multiple downsampling channels to obtain a multi-scale feature map.

[0089] In the case where the text line detection model includes a DBNet structure, DBNet tends to detect relatively large targets, but in the test paper scenario, the text is generally small. DBNet usually performs 5 downsamplings and 32-fold downsampling. For the test paper image, if it is reduced by 32 times, the text line features are already incomplete, with more losses, and redundant operations will be increased. Therefore, for the test paper image, the number of downsampling channels can be adaptively changed. For example, the downsampling channels can include: 4-fold downsampling channels, 8-fold downsampling channels, and 16-fold downsampling channels. Of course, the number of downsampling channels and the reduction multiple can also be adjusted according to different usage scenarios, and the present disclosure is not limited to the above scenarios.

[0090] The channel attention module is used to weight feature maps of different scales.

[0091] In order to enable the model to pay more attention to each downsampling channel, thereby highlighting important features and suppressing unimportant features, in this embodiment, after the downsampling process, a channel attention module can be added to add a weight value to each downsampling channel to make the model recognition more accurate.

[0092] For example, the channel attention module (for example, a SE module) may include: a pooling submodule, a fully connected layer, and a weighted submodule.

[0093] The pooling submodule is used to perform global average pooling processing on the feature map.

[0094] For example, a feature map of size C*W*H (where W represents the width of the feature map, H represents the height of the feature map, and C represents the number of channels) can be converted into a feature map of size C*1*1 through a pooling submodule.

[0095] This fully connected layer is used to predict the weight of each downsampled channel based on the feature map processed by global average pooling.

[0096] Among them, the fully connected layer can also be used to reduce the feature dimension of the feature map (for example, it can be reduced to 1 / 16 of the input), and then return to the original dimension after ReLu activation.

[0097] The weighted submodule is used to multiply the feature map corresponding to each downsampling channel by the weight corresponding to the downsampling channel to obtain a weighted feature map.

[0098] Specifically, the normalized weight can be added to the features of each channel, that is, the feature map corresponding to the downsampled channel is multiplied by the weight corresponding to the downsampled channel to obtain a weighted feature map.

[0099] In this way, by adding a channel attention module without disrupting the original main structure of the network, the model can pay attention to the importance of different downsampling channels and thus highlight important features.

[0100] The feature fusion module is used to perform feature fusion on multiple weighted feature maps to obtain a target feature map.

[0101] The processing module is used to convert the target feature map into the binary image.

[0102] In this way, the binary image corresponding to the text line image can be obtained through the above text line detection module.

[0103] In some embodiments, the text line detection model is trained by:

[0104] S1, obtain a sample image and a text line label corresponding to the sample image.

[0105] The text line label is the text information corresponding to the printed text in the sample image (eg, text content, text position, etc.).

[0106] S2: Train a preset training model according to the sample image and the text line label to obtain the text line detection model.

[0107] In real-world scenarios, sample images and corresponding text line labels used for training are often manually annotated, which is time-consuming, labor-intensive, and costly, and the number of sample images is also very limited. To address these issues, expand the number of sample images, and improve the efficiency of sample labeling, the sample images are obtained using the following method:

[0108] First, the target string can be obtained from the preset corpus.

[0109] Specifically, a character string of specific text content (e.g., an e-book in txt format) can be randomly selected from a preset corpus, and the character string of specific text content in the preset corpus can be converted into a plurality of character strings of different fonts based on a preset printed font library, thereby obtaining a target character string. The length of the character string can be preset, that is, a character string of a preset length can be obtained from the preset corpus. In addition, the font size of each font in the character string can also be preset, which is not specifically limited in this disclosure.

[0110] Secondly, the target character string may be rendered to obtain a sample text line image containing the target character string.

[0111] In some implementations, in order to adapt to the test paper scenario, uppercase / lowercase question numbers, ABCD and other option numbers can also be randomly added before each text line.

[0112] Then, a sample background image can be obtained.

[0113] Considering that actual test paper scenarios often include more background images, in order to be closer to the application scenario, sample background images can also be obtained, which include at least one or more of handwritten text line images, formula line images, watermark images, and illustrations.

[0114] The handwritten text line image can be, for example, an image containing handwritten text that simulates the answer area. Formulas are also an essential part of science exams, so a formula line image can also be prepared. Furthermore, multiple watermark images and illustrations can be prepared to simulate elements found in real exams.

[0115] Finally, the sample image can be obtained according to the sample text line image and the sample background image.

[0116] In this step, the sample text line image and the sample background image may be synthesized to obtain the label of the sample image, that is, the position coordinates of each text line.

[0117] For example, the width and height of the sample background image can be determined first, and based on these, the margins of the sample image can be determined. The width of the sample text line image, i.e., the width of each line of printed text, can be set, and the sample text line images can be pasted line by line. At this point, the coordinate position of each text line, i.e., the label of the sample image, can be obtained based on the starting coordinates when pasting the sample text line image and the width and height of the text line.

[0118] Furthermore, when synthesizing sample images, simply pasting text line images can result in poor data generalization. Therefore, we randomly insert blank lines, handwritten text lines, or ABCD content. Furthermore, when inserting these content, the coordinates are not recorded. This allows us to obtain labeled sample images that resemble the photographed / scanned test papers.

[0119] Furthermore, in order to be closer to the photographing scene, the light and dark of the sample image may be transformed and / or noise may be added to obtain multiple sample images.

[0120] In this way, the generated sample images have flexible plasticity and strong generalization, which can greatly expand the number of sample images and improve the labeling efficiency of sample image labels.

[0121] Using this method, redundant information within the undetermined text line frame can be removed using the binary rectangular frame in the binary image to obtain a target text line frame containing only one line of text. The target text line frame can then be used to determine the text line information corresponding to the printed text in the text line image. This ensures that each target text line frame contains only one line of text. Therefore, when performing text recognition based on the target text line frame, the corresponding printed text line information can be accurately identified, resolving the issue of multiple lines of text within a single target text line frame affecting text recognition accuracy.

[0122] Figure 3 is a block diagram of a text line recognition device according to an exemplary embodiment. Figure 3 As shown, the device 200 includes:

[0123] A first acquisition module 201 is used to acquire a text line image to be recognized;

[0124] A conversion module 202, configured to convert the text line image into a binary image;

[0125] The second acquisition module 203 is used to acquire a binary image rectangular frame in the binary image and a pending text line frame corresponding to the binary image rectangular frame in the text line image;

[0126] A removal module 204 is configured to remove redundant information from the undetermined text line frame based on the binary image rectangular frame to obtain a target text line frame; the target text line frame only contains one line of text;

[0127] The determination module 205 is configured to determine text line information corresponding to the printed text in the text line image according to the target text line frame.

[0128] Optionally, the determining module 205 is further configured to determine the number of connected domains contained in the rectangular frame of the binary image;

[0129] The removal module 204 is configured to remove redundant information from the pending text line frame based on the binary image rectangle frame to obtain the target text line frame when the number of connected domains is greater than or equal to a preset number threshold.

[0130] Alternatively, as Figure 4 As shown, the device 200 further includes:

[0131] The generating module 206 is configured to use the pending text line frame as the target text line frame when the number of the connected domains is less than the preset number threshold.

[0132] Optionally, the removal module 204 is configured to expand the binary image rectangular frame; and cover the pending text line frame according to the background portion of the expanded binary image rectangular frame to obtain the target text line frame.

[0133] Optionally, the second acquisition module 203 is used to obtain the minimum circumscribed rectangle of each text line in the binary image to obtain the binary image rectangular frame; and obtain the pending text line frame from the text line image according to the position of the binary image rectangular frame in the binary image.

[0134] Optionally, the conversion module 202 is configured to input the text line image into a pre-generated text line detection model to obtain the binary image output by the text line detection model.

[0135] Optionally, the text line detection model includes: a feature sampling module, a channel attention module, a feature fusion module and a processing module;

[0136] The feature sampling module is used to downsample the text line image through multiple downsampling channels to obtain a multi-scale feature map;

[0137] The channel attention module is used to weight feature maps of different scales;

[0138] The feature fusion module is used to fuse multiple weighted feature maps to obtain a target feature map;

[0139] The processing module is used to convert the target feature map into the binary image.

[0140] Optionally, the channel attention module includes: a pooling submodule, a fully connected layer and a weighted submodule;

[0141] The pooling submodule is used to perform global average pooling processing on the feature map;

[0142] This fully connected layer is used to predict the weight of each downsampled channel based on the feature map processed by global average pooling;

[0143] The weighted submodule is used to multiply the feature map corresponding to each downsampling channel by the weight corresponding to the downsampling channel to obtain a weighted feature map.

[0144] Optionally, the downsampling channel includes: a 4-fold downsampling channel, an 8-fold downsampling channel, and a 16-fold downsampling channel.

[0145] Optionally, the text line detection model is trained in the following way:

[0146] Get the sample image and the text line label corresponding to the sample image;

[0147] The preset training model is trained according to the sample image and the text line label to obtain the text line detection model.

[0148] Optionally, the sample image is obtained by:

[0149] Get the target string from the preset corpus;

[0150] Rendering the target character string to obtain a sample text line image containing the target character string;

[0151] Acquire a sample background image; the sample background image includes at least one or more of a handwritten text line image, a formula line image, a watermark image, and an illustration;

[0152] The sample image is obtained according to the sample text line image and the sample background image.

[0153] Using the above-described device, redundant information within the undetermined text line frame can be removed using the binary rectangular frame in the binary image to obtain a target text line frame containing only one line of text. The target text line frame can then be used to determine the text line information corresponding to the printed text in the text line image. This ensures that each target text line frame contains only one line of text, allowing accurate identification of the corresponding printed text line information during text recognition based on the target text line frame. This solves the problem of multiple lines of text in a single target text line frame affecting text recognition accuracy.

[0154] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0155] Figure 5 3 is a block diagram of an electronic device 300 according to an exemplary embodiment. For example, the electronic device 300 can be provided as a server. Figure 5 The electronic device 300 includes a processor 322, which may be one or more, and a memory 332 for storing a computer program executable by the processor 322. The computer program stored in the memory 332 may include one or more modules, each corresponding to a set of instructions. In addition, the processor 322 may be configured to execute the computer program to perform the above-mentioned text line recognition method.

[0156] In addition, the electronic device 300 may further include a power supply component 326 and a communication component 350. The power supply component 326 may be configured to perform power management of the electronic device 300, and the communication component 350 may be configured to implement communication, such as wired or wireless communication, of the electronic device 300. In addition, the electronic device 300 may further include an input / output (I / O) interface 358. The electronic device 300 may operate based on an operating system stored in the memory 332.

[0157] In another exemplary embodiment, a computer-readable storage medium including program instructions is further provided. When executed by a processor, the program instructions implement the steps of the above-described text line recognition method. For example, the non-transitory computer-readable storage medium may be the aforementioned memory 332 including the program instructions. The program instructions may be executed by the processor 322 of the electronic device 300 to perform the above-described text line recognition method.

[0158] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program executable by a programmable device, and has a code portion for performing the above-mentioned text line recognition method when executed by the programmable device.

[0159] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings. However, the present disclosure is not limited to the specific details of the above embodiments. Within the technical concept of the present disclosure, various simple modifications can be made to the technical solutions of the present disclosure, and these simple modifications all fall within the scope of protection of the present disclosure.

[0160] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any appropriate manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0161] In addition, the various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A method for identifying a text line, characterized in that: The method comprises: Get the text line image to be recognized; Converting the text line image into a binary image; Acquire a binary image rectangular frame in the binary image and a pending text line frame corresponding to the binary image rectangular frame in the text line image; Based on the binary image rectangular frame, redundant information in the pending text line frame is removed to obtain a target text line frame; the target text line frame only contains one line of text; Determining text line information corresponding to the printed text in the text line image according to the target text line frame; The method further comprises: Determine the number of connected domains contained in the binary image rectangular frame; The removing of redundant information in the pending text line frame based on the binary image rectangular frame to obtain the target text line frame includes: When the number of connected domains is greater than or equal to a preset threshold, based on the binary image rectangle, redundant information in the pending text line frame is removed to obtain the target text line frame; The method further includes: when the number of the connected domains is less than the preset number threshold, using the undetermined text line box as the target text line box.

2. The method according to claim 1, characterized in that The removing of redundant information in the pending text line frame based on the binary image rectangular frame to obtain the target text line frame includes: Expanding the binary image rectangular frame; The target text line frame is obtained by covering the pending text line frame according to the background portion in the expanded binary image rectangular frame.

3. The method according to claim 1, characterized in that The obtaining of the binary image rectangular frame in the binary image and the undetermined text line frame corresponding to the binary image rectangular frame in the text line image comprises: Obtaining the minimum bounding rectangle of each text line in the binary image to obtain the binary image rectangular frame; The undetermined text line frame is obtained from the text line image according to the position of the binary image rectangular frame in the binary image.

4. The method according to claim 1, wherein Converting the text line image into a binary image comprises: The text line image is input into a pre-generated text line detection model to obtain the binary image output by the text line detection model.

5. The method according to claim 4, characterized in that The text line detection model includes: a feature sampling module, a channel attention module, a feature fusion module and a processing module; The feature sampling module is used to downsample the text line image through multiple downsampling channels to obtain a multi-scale feature map; The channel attention module is used to weight feature maps of different scales; The feature fusion module is used to perform feature fusion on multiple weighted feature maps to obtain a target feature map; The processing module is used to convert the target feature map into the binary image.

6. The method according to claim 5, characterized in that The channel attention module includes: a pooling submodule, a fully connected layer and a weighted submodule; The pooling submodule is used to perform global average pooling processing on the feature map; The fully connected layer is used to predict the weight of each downsampled channel based on the feature map after global average pooling; The weighting submodule is used to multiply the feature map corresponding to each downsampling channel by the weight corresponding to the downsampling channel to obtain a weighted feature map.

7. The method according to claim 5, characterized in that The downsampling channels include: a 4-fold downsampling channel, an 8-fold downsampling channel, and a 16-fold downsampling channel.

8. The method according to claim 4, characterized in that The text line detection model is trained in the following way: Obtaining a sample image and a text line label corresponding to the sample image; A preset training model is trained according to the sample images and the text line labels to obtain the text line detection model.

9. The method according to claim 8, characterized in that The sample image is obtained by: Get the target string from the preset corpus; Rendering the target character string to obtain a sample text line image containing the target character string; Acquire a sample background image; the sample background image includes at least one or more of a handwritten text line image, a formula line image, a watermark image, and an illustration; The sample image is obtained according to the sample text line image and the sample background image.

10. A text line recognition device, characterized in that: The device comprises: A first acquisition module is used to acquire a text line image to be recognized; A conversion module, configured to convert the text line image into a binary image; A second acquisition module is configured to acquire a binary image rectangular frame in the binary image and a pending text line frame corresponding to the binary image rectangular frame in the text line image; a removal module, configured to remove redundant information in the pending text line frame based on the binary image rectangular frame to obtain a target text line frame; the target text line frame only contains one line of text; a determination module, configured to determine text line information corresponding to the printed text in the text line image according to the target text line frame; The determining module is further configured to determine the number of connected domains contained in the binary image rectangular frame; The removal module is configured to remove redundant information from the pending text line frame based on the binary image rectangular frame to obtain the target text line frame when the number of connected domains is greater than or equal to a preset number threshold; The device further comprises: A generating module is configured to use the pending text line frame as the target text line frame when the number of the connected domains is less than the preset number threshold.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

12. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 9.

Images