Text information acquisition method, device, storage medium and electronic device

Through the optimized text line detection model and post-processing technology, the problems of unruly recognition and disconnection caused by tilting and bending of handwritten text in the image are solved, achieving more accurate and efficient text information acquisition.

CN114565925BActive Publication Date: 2024-12-06BEIJING DINGSHIXINGJIAOYU CONSULTATION CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210179841.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-02-25
Publication Date
2024-12-06
Estimated Expiration
2042-02-25

AI Technical Summary

Technical Problem

In the prior art, the handwritten text in the image has the problem of inclination and bending, resulting in poor recognition results, and long sentences with different text spacing can easily lead to disconnection of sentences.

Method used

The optimized text line detection model obtains the binary map of the target image, obtains the text enclosure box and performs post-processing, and merges the intersecting and overlapping surrounding boxes to avoid unsupported recognition results and disconnection of statements. Specific methods include judging and filtering the intersection of adjacent text enclosure boxes, calculating the interchange area, filtering and merging the text enclosure boxes, using the DBNet model and replacing its partial 3*3 convolution kernel to 1*3 convolution kernel to improve the pairing. Sensitivity of horizontal orientation characteristics.

Benefits of technology

It improves the accuracy and efficiency of text information acquisition, avoids statement breakage caused by different text spacing, and enhances the robustness of tilted curved text.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114565925B_ABST
    Figure CN114565925B_ABST
Patent Text Reader

Abstract

The present disclosure relates to a method, device, storage medium and electronic device for acquiring text information, which solves the technical problem in the related art that the recognition result is not robust due to the tilt and curvature of the text in the image. The method comprises: acquiring a target image; inputting the target image into an optimized text line detection model to obtain a binary image corresponding to the text content in the target image output by the text line detection model; acquiring a text bounding box corresponding to each text content in the target image according to the binary image; post-processing each text bounding box to obtain a target text bounding box corresponding to the target image; performing text recognition on the text content in the target text bounding box to obtain text information corresponding to the target image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a method, device, storage medium and electronic device for acquiring text information. Background Art

[0002] In the educational scenario, handwritten text detection in test papers is an important step in obtaining students' answers. The targets of handwritten text detection can include English handwritten content such as words, phrases, and sentences, and Chinese handwritten content such as characters, words, and sentences. Handwritten text detection can obtain images containing a high content of the test paper by scanning the test paper content or taking a photo of the test paper content, and then detect the text in the image to obtain the text information in the image.

[0003] In the related art, when detecting text in an image through an object detection method or a text detection method, the recognition result is not robust because the text in the image is tilted and curved, and because the text in the image contains long sentences with uneven text spacing, sentences may be disconnected. Summary of the invention

[0004] The purpose of the present disclosure is to provide a method, device, storage medium and electronic device for acquiring text information, which solve the technical problem in the related art that the recognition result is not robust due to the tilt and curvature of the text in the image.

[0005] In order to achieve the above objectives, in a first aspect, the present disclosure provides a method for acquiring text information, the method comprising:

[0006] Get the target image;

[0007] Inputting the target image into the optimized text line detection model to obtain a binary image corresponding to the text content in the target image output by the text line detection model;

[0008] Acquire a text bounding box corresponding to each text content in the target image according to the binary image;

[0009] Post-processing each of the text enclosing frames to obtain a target text enclosing frame corresponding to the target image;

[0010] Perform text recognition on the text content in the target text enclosing box to obtain text information corresponding to the target image.

[0011] Optionally, the post-processing of each of the text enclosing frames to obtain a target text enclosing frame corresponding to the target image includes:

[0012] Traversing each of the text enclosing boxes to determine whether two adjacent text enclosing boxes have an intersection;

[0013] When two adjacent text enclosing boxes have an intersection, the two adjacent text enclosing boxes are screened;

[0014] The two adjacent text enclosing frames after screening are merged to obtain a target text enclosing frame.

[0015] Optionally, the two adjacent text enclosing boxes include a first text enclosing box and a second text enclosing box, and the determining whether the two adjacent text enclosing boxes have an intersection includes:

[0016] Obtaining the upper, lower, left, and right boundary values ​​of the first text enclosing box and the second text enclosing box on the coordinate axis Oxy;

[0017] Determine, according to the boundary value, whether the first text enclosing box and the second text enclosing box intersect in the X-axis direction or the Y-axis direction of the coordinate axis Oxy;

[0018] When the first text enclosing box intersects with the second text enclosing box in the X-axis direction or the Y-axis direction, it is determined that the first text enclosing box and the second text enclosing box have an intersection.

[0019] Optionally, the method further comprises: when the first text enclosing box and the second text enclosing box do not intersect in the X-axis direction and the Y-axis direction, calculating the intersection-and-combination ratio of the first text enclosing box and the second text enclosing box;

[0020] When the intersection-over-union ratio is greater than zero, it is determined that the first text enclosing box and the second text enclosing box have an intersection.

[0021] Optionally, the screening of two adjacent text enclosing boxes includes:

[0022] Obtaining a minimum value x1 and a maximum value x2 of the first text enclosing box on the X-axis, and a minimum value x3 and a maximum value x4 of the second text enclosing box on the X-axis;

[0023] In the case of x1<x3, x2<x4 and x3<x2, determining that the first text enclosing box is the preceding text enclosing box and the second text enclosing box is the succeeding text enclosing box;

[0024] A target preceding text enclosing box and a target following text enclosing box are screened out from the preceding text enclosing box and the following text enclosing box, wherein the target preceding text enclosing box and the target following text enclosing box satisfy the following conditions: an intersection of coordinate intervals of the target preceding text enclosing box and the target following text enclosing box in the X-axis direction is less than a first preset ratio of a union of coordinate intervals, and a distance between center points of two adjacent sides of the target preceding text enclosing box and the target following text enclosing box in the Y-axis direction does not exceed a second preset ratio of the shortest sides of the two text enclosing boxes.

[0025] Optionally, the merging of the two adjacent text enclosing frames after filtering to obtain a target text enclosing frame includes:

[0026] Obtaining the coordinates of each vertex of the target-before text enclosing box and the target-after text enclosing box;

[0027] The minimum circumscribed rectangle of the target preceding text enclosing box and the target following text enclosing box is determined according to the vertex coordinates, and is used as the target text enclosing box.

[0028] Optionally, the text line detection model includes a DBNet model, and the optimized text line detection model is obtained by replacing some 3*3 convolution kernels in the skeleton network ResNet of the DBNet model with multiple 1*3 convolution kernels.

[0029] Optionally, acquiring a text bounding box corresponding to each text content in the target image according to the binary image includes:

[0030] Obtaining a contracted text area of ​​text content in the target image according to the binary image;

[0031] The shrunken text area is expanded to obtain a text enclosing box corresponding to each text content in the target image.

[0032] In a second aspect, the present disclosure provides a text information acquisition device, the device comprising:

[0033] An acquisition module, configured to acquire a target image;

[0034] An execution module is configured to input the target image into the optimized text line detection model to obtain a binary image output by the text line detection model corresponding to the text content in the target image;

[0035] An expansion module is configured to obtain a text bounding box corresponding to each text content in the target image according to the binary image;

[0036] A post-processing module is configured to perform post-processing on each of the text enclosing frames to obtain a target text enclosing frame corresponding to the target image;

[0037] The recognition module is configured to perform text recognition on the text content in the target text enclosing box to obtain text information corresponding to the target image.

[0038] In a third aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above method when the computer program is executed by a processor.

[0039] In a fourth aspect, the present disclosure provides an electronic device, including:

[0040] a memory having a computer program stored thereon;

[0041] A processor is used to execute the computer program in the memory to implement the steps of the above method.

[0042] Through the above technical scheme, the present invention obtains a binary image of the text content in the corresponding target image through an optimized text line detection model, obtains a text bounding box of each text content in the corresponding target image according to the binary image, so as to obtain the text content in the text bounding box, thereby avoiding the recognition result being not robust, and merges the intersecting and overlapping text bounding boxes through post-processing of the text bounding boxes to obtain the target text bounding box of the corresponding target image, thereby avoiding the sentence disconnection caused by the existence of long sentences with uneven text spacing in the text of the target image.

[0043] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:

[0045] Figure 1 is a flowchart of a method for acquiring text information according to an exemplary embodiment;

[0046] Figure 2 is a schematic diagram of a text enclosing frame in a method for acquiring text information according to an exemplary embodiment;

[0047] Figure 3 is a schematic diagram of the intersection of text enclosing boxes in a method for acquiring text information according to an exemplary embodiment;

[0048] Figure 4is another intersection schematic diagram of text enclosing boxes in a method for acquiring text information according to an exemplary embodiment;

[0049] Figure 5 is another intersection schematic diagram of text enclosing boxes in a method for acquiring text information according to an exemplary embodiment;

[0050] Figure 6 is another intersection schematic diagram of text enclosing boxes in a method for acquiring text information according to an exemplary embodiment;

[0051] Figure 7 is a block diagram of a text information acquisition device according to an exemplary embodiment;

[0052] Figure 8 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0053] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0054] It should be noted that in the present disclosure, the terms "S101", "S102" and the like in the specification, claims and drawings are used to distinguish steps, and do not necessarily mean that the method steps are performed in a specific order or sequential sequence.

[0055] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0056] As mentioned in the background technology, in an educational scenario, detecting handwritten text in a test paper is an important step in obtaining student answers. Taking an English test paper as an example, the handwritten English text in the English test paper is detected. The target of handwritten English detection includes all handwritten English content such as letters, words, phrases, sentences, etc. The image of the English test paper can be obtained by scanning and taking pictures. Taking pictures is more convenient and quick, so the image of the English test paper is generally obtained by taking pictures.

[0057] In the related art, the text in the image is detected by a target detection method or a text detection method to obtain the handwritten English content in the English test paper. However, since the text in the image is tilted and bent, such as the handwritten English in the image obtained due to the shooting angle, the recognition result is not robust. In addition, since the text in the image has long sentences with uneven text spacing, such as the composition content in the test paper, long sentences with uneven word spacing are prone to disconnection, resulting in sentence disconnection.

[0058] In view of this, the present disclosure provides a text information acquisition method, device, storage medium and electronic device, which solve the problem in the related art that the recognition result is not robust due to the tilt and curvature of the text in the image, and the sentence is disconnected due to the presence of long sentences with uneven text spacing in the image.

[0059] Figure 1 is a flowchart of a method for obtaining text information according to an exemplary embodiment. Figure 1 , the method comprises the following steps.

[0060] In step S101, a target image is acquired.

[0061] In step S102, the target image is input into the optimized text line detection model to obtain a binary image corresponding to the text content in the target image output by the text line detection model.

[0062] In step S103, a text enclosing box corresponding to each text content in the target image is obtained according to the binary image.

[0063] In step S104, each text enclosing frame is post-processed to obtain a target text enclosing frame corresponding to the target image.

[0064] In step S105, text recognition is performed on the text content in the target text enclosing box to obtain text information corresponding to the target text image.

[0065] The present invention obtains a binary image of text content in a corresponding target image through an optimized text line detection model, obtains a text bounding box of each text content in the corresponding target image according to the binary image, so as to obtain the text content in the text bounding box, thereby avoiding a non-robust recognition result, and obtains a target text bounding box of the corresponding target image by merging intersecting text bounding boxes with overlapping contents through post-processing of the text bounding boxes, thereby avoiding sentence disconnection due to the presence of long sentences with uneven text spacing in the text of the target image.

[0066] In order to enable those skilled in the art to better understand the text information acquisition method provided by the present disclosure, the above steps are described in detail with examples below.

[0067] The handwritten text in the images obtained by taking photos of English test papers can be roughly divided into two categories: multiple-choice filling and composition. Among them, the multiple-choice filling is mainly composed of words or phrases, while the composition is mainly composed of long sentences. When detecting the fill-in-the-blank category, the expected requirement is to obtain the text bounding box of each empty word or phrase, referring to Figure 2 Fill in the blanks; when the phrases are close and need to be separated into multiple text boxes, refer to Figure 2 Medium and short sentences; when detecting essays, the expected requirement is to obtain the text content of the same line in a text bounding box, refer to Figure 2 Therefore, in order to improve the accuracy of obtaining the text bounding box in the image and avoid the situation where a sentence has multiple text bounding boxes, which makes a sentence disconnected, the present disclosure needs to post-process the text bounding box to improve the accuracy of obtaining text information.

[0068] In one embodiment, in step S104, each text enclosing frame is post-processed to obtain a target text enclosing frame corresponding to the target image, including:

[0069] Traverse each text bounding box to determine whether there is an intersection between two adjacent text bounding boxes;

[0070] When adjacent text bounding boxes have an intersection, two adjacent text bounding boxes are screened;

[0071] The two adjacent text bounding boxes after screening are merged to obtain the target text bounding box.

[0072] The present invention screens adjacent text enclosing boxes when there are intersections between adjacent text enclosing boxes, merges the two adjacent text enclosing boxes after the screening, obtains a target text enclosing box, and implements post-processing of the text enclosing boxes, thereby avoiding the situation where a sentence has multiple text enclosing boxes that disconnects the sentence, and improving the accuracy of obtaining text information according to the target text enclosing box.

[0073] Reference Figure 2 In composition, when detecting composition, the expected requirement is to obtain the text content of the same line in one text bounding box. However, due to semantics and other reasons, when the text content of the same line includes multiple sentences, the text content of the same line will include multiple text bounding boxes. In the composition scenario, the distance between each character of handwriting varies greatly. Sentences with large distance between characters may be bounded by multiple text bounding boxes, dividing the text content of the same line into several sections. When the text content of the same line includes multiple text bounding boxes, the text bounding boxes may not intersect or may intersect. When the text bounding boxes intersect, the same text content will be included in the adjacent text bounding boxes. Therefore, before obtaining the text content in the text bounding boxes, it is necessary to determine whether the text bounding boxes intersect.

[0074] In one embodiment, the two adjacent text enclosing boxes include a first text enclosing box and a second text enclosing box, and determining whether the two adjacent text enclosing boxes have an intersection includes:

[0075] Get the upper, lower, left, and right boundary values ​​of the first text enclosing box and the second text enclosing box in the coordinate axis Oxy;

[0076] Determine, according to the boundary value, whether the first text enclosing box and the second text enclosing box intersect in the X-axis direction or the Y-axis direction of the coordinate axis Oxy;

[0077] When the first text enclosing box intersects with the second text enclosing box in the X-axis direction or the Y-axis direction of the coordinate axis Oxy, it is determined that the first text enclosing box and the second text enclosing box have an intersection.

[0078] For example, obtain the coordinate values ​​max_y, min_y, max_x, min_x of the upper, lower, left and right boundaries of the first text enclosing box and the second text enclosing box in the coordinate axis Oxy; then determine whether the two text enclosing boxes intersect in the X-axis direction of the coordinate axis Oxy based on the max_x and min_x of the first text enclosing box and the second text enclosing box. Similarly, determine whether the two text enclosing boxes intersect in the Y-axis direction of the coordinate axis Oxy based on the max_y and min_y of the first text enclosing box and the second text enclosing box; when the two text enclosing boxes intersect in the X-axis direction or the Y-axis direction of the coordinate axis Oxy, determine that the first text enclosing box and the second text enclosing box have an intersection.

[0079] The present disclosure does not determine whether two text bounding boxes have an intersection by directly calculating the intersection over union (IoU) of two adjacent text bounding boxes because the calculation speed of the IoU is slow. The present disclosure obtains the coordinate values ​​of the upper, lower, left and right boundaries of the two adjacent text bounding boxes in the coordinate axis Oxy, and determines whether the two text bounding boxes intersect in the X-axis direction or the Y-axis direction in Oxy according to the obtained coordinate values, thereby determining whether the two text bounding boxes have an intersection, which shortens the time for obtaining text information and improves the efficiency of obtaining text information. In addition, by determining whether two adjacent text bounding boxes have an intersection, the situation of obtaining the same text content multiple times is avoided.

[0080] When two adjacent text enclosing boxes do not intersect in both the X-axis direction and the Y-axis direction in Oxy, it is possible to further determine whether the two adjacent text enclosing boxes have an intersection by calculating the intersection-and-combination ratio of the two adjacent text enclosing boxes.

[0081] In one embodiment, the method further comprises:

[0082] When the first text enclosing box and the second text enclosing box do not intersect in the X-axis direction and the Y-axis direction, calculating the intersection-and-combination ratio area of ​​the first text enclosing box and the second text enclosing box;

[0083] When the intersection-over-area ratio is greater than zero, it is determined that the first text enclosing box and the second text enclosing box have an intersection.

[0084] In the case where two adjacent text enclosing boxes do not intersect in the X-axis direction and the Y-axis direction, the present invention further determines whether the two text enclosing boxes have an intersection by judging the intersection-and-combination ratio area of ​​the two text enclosing boxes, thereby improving the accuracy and reliability of the judgment result.

[0085] Furthermore, when two adjacent text enclosing boxes do not have an intersection, it is continued to be determined whether the next adjacent text enclosing box has an intersection, until the determination of all text enclosing boxes is completed.

[0086] For example, if the text enclosing box No. i and the text enclosing box No. j are two adjacent text enclosing boxes, and if the text enclosing box No. i and the text enclosing box No. j do not intersect, continue to determine whether the text enclosing box No. j and the text enclosing box No. j+1 have an intersection, until all text enclosing boxes are determined.

[0087] When two adjacent text bounding boxes have an intersection, there are three types of intersections between the two adjacent text bounding boxes. Figure 3 : 1. The left and right text enclosing boxes have an intersection, 2. The upper and lower text enclosing boxes have an intersection, 3. The text enclosing box with the modified word and the text enclosing box of the main text content have an intersection. Since there are many cases of intersection, in this disclosure, only the two text enclosing boxes in the case of the left and right text enclosing boxes intersecting need to be merged, and the cases of type 2 and type 3 need to be filtered out. Type 1, type 2, and type 3 respectively correspond to judgment conditions, and type 1 is filtered out from the two adjacent text enclosing boxes that have an intersection according to the judgment conditions.

[0088] In one embodiment, screening two adjacent text bounding boxes includes:

[0089] Obtain the minimum value x1 and the maximum value x2 of the first text enclosing box on the X-axis, and the minimum value x3 and the maximum value x4 of the second text enclosing box on the X-axis;

[0090] In the case of x1<x3, x2<x4 and x3<x2, the first text enclosing box is determined to be the preceding text enclosing box, and the second text enclosing box is determined to be the succeeding text enclosing box;

[0091] A target preceding text enclosing box and a target following text enclosing box are screened out from the preceding text enclosing box and the following text enclosing box, and the target preceding text enclosing box and the target following text enclosing box meet the following conditions: an intersection of coordinate intervals of the target preceding text enclosing box and the target following text enclosing box in the X-axis direction is less than a first preset ratio of a union of coordinate intervals, and a distance between center points of two adjacent sides of the target preceding text enclosing box and the target following text enclosing box in the Y-axis direction does not exceed a second preset ratio of the shortest sides of the two text enclosing boxes.

[0092] Among them, the first preset ratio can be preset according to the intersection size of two adjacent text enclosing boxes. In the present disclosure, the first preset ratio is 30%, and the second preset ratio is preset according to the intersection size of two adjacent text enclosing boxes. In the present disclosure, the second preset ratio is 1 / 3.

[0093] For example, we first need to determine which of the two adjacent text bounding boxes is in front and which is in the back. Figure 4 , the maximum value of the front text enclosing box on the X axis is x2 and the minimum value is x1, the maximum value of the back text enclosing box on the X axis is x4 and the minimum value is x3, the front text enclosing box and the back text enclosing box meet the judgment condition: x1<x3, x2<x4 and x3<x2. The front text enclosing box and the back text enclosing box in the two adjacent text enclosing boxes are determined by the judgment condition.

[0094] Then, the target text enclosing box in front and the target text enclosing box in back are selected from the text enclosing box in front and the text enclosing box in back, that is, the text enclosing box in front and the text enclosing box in back are removed if the two text enclosing boxes intersect each other (see Figure 5 ) and the text bounding box of the modified word intersects with the text bounding box of the main text content (see Figure 6 ). The judgment condition for the previous text enclosing box and the next text enclosing box to intersect vertically is: the intersection of the coordinate intervals of the previous text enclosing box and the next text enclosing box in the X-axis direction is not less than 30% of the union of the coordinate intervals. Therefore, the target previous text enclosing box and the target next text enclosing box must meet the requirement that the intersection of the coordinate intervals in the X-axis direction is less than 30% of the union of the coordinate intervals, so as to eliminate Figure 5 The two text bounding boxes shown in the figure intersect vertically. The judgment condition for the intersection of the text bounding box of the previous text and the text bounding box of the next text is that the distance between the center points of the a side and the b side is greater than or equal to 1 / 3 of the shortest side of the two text bounding boxes. Therefore, the target text bounding box in front and the target text bounding box in the next text must also meet the requirement that the distance between the center points of the a side and the b side does not exceed 1 / 3 of the shortest side of the two text bounding boxes, so as to eliminate Figure 6 The text bounding box of the modified word intersects with the text bounding box of the main text content.

[0095] The present invention screens two adjacent text enclosing boxes to determine a target-in-front text enclosing box and a target-in-back text enclosing box that need to be merged, so as to merge the target-in-front text enclosing box and the target-in-back text enclosing box into one text enclosing box, thereby avoiding the situation where the same text content is obtained multiple times when there is repeated text content in the two text enclosing boxes.

[0096] In one embodiment, two adjacent text enclosing frames after screening are merged to obtain a target text enclosing frame, including:

[0097] Obtain the coordinates of each vertex of the target's front text enclosing box and the target's back text enclosing box;

[0098] The minimum circumscribed rectangle of the target preceding text enclosing box and the target following text enclosing box is determined according to the coordinates of each vertex, and is used as the target text enclosing box.

[0099] For example, the target text bounding box in front is the i text bounding box, and the target text bounding box in the back is the j text bounding box. All vertices of the i text bounding box and the j text bounding box are determined, and the minimum enclosing rectangle is taken according to all vertices of the two text bounding boxes. The minimum enclosing rectangle is used as the new i text bounding box, and the j text bounding box is deleted to realize the merging of the i text bounding box and the j text bounding box.

[0100] The present disclosure can realize merging of intersecting text bounding boxes in a target image through merging processing, thereby avoiding the situation where the same text content is obtained multiple times when there is repeated text content in two text bounding boxes.

[0101] Existing text line detection modules include: DBNet, ContourNet, PANNet, PSENet, Craft, etc. According to the characteristics of dense text lines and easy adhesion in the test paper, it is necessary to select a network that is not prone to adhesion problems for text line detection of the content in the test paper, and the real-time performance of the model must be considered at the same time. Therefore, the running speed of the model is also very important. Considering the above, ContourNet, PSENet, and Craft do not meet the real-time requirements. DBNet and PANNet run faster and basically meet the real-time requirements. Compared with DBNet and PANNet, PANNet is faster, but in order to prevent direct adhesion of text lines, DBNet shrinks the text inward in the pre-processing of training, and then expands it outward after prediction. It has good anti-adhesion properties, stronger anti-adhesion properties than PANNet, and higher accuracy. And DBNet is robust to tilt and bending. Therefore, DBNet is used as the basic network for text line detection in this disclosure.

[0102] Because the text information acquisition method disclosed in the present invention involves identifying long sentences that are easily disconnected in the composition, DBNet needs to be optimized. First, the text features applicable to the analysis model may include texts in certain scenes (such as street scenes), and the text styles are not uniform, such as horizontal text styles, vertical text styles, severely tilted texts, curved text styles, and wavy text styles, etc., and the text style features of the text content in the present invention are mostly horizontally laid out or slightly tilted and curved texts, so the model applicable to the present invention should be biased towards extracting horizontal image features.

[0103] The convolution kernel size of the existing text line detection model is usually 3*3, which focuses on image features in both horizontal and vertical directions, with a larger information collection range and irrelevant information collected at the same time. However, for horizontal text lines, it is more necessary to pay attention to image features in the horizontal direction. Therefore, the convolution kernel size can be optimized, such as optimizing the convolution kernel size of 3*3 in the text line detection model to 1*3. Because the convolution kernel size is changed to 1*3, the number of parameters involved in the text line detection model will be greatly reduced. The number of parameters of a 3*3 convolution kernel is equal to the number of parameters of three 1*3 convolution kernels. By replacing one 3*3 convolution kernel with three 1*3 convolution kernels, the range of the receptive field in the horizontal direction is increased, making it more sensitive to the features in the horizontal direction. Based on the above principle, to optimize the DBNet model, the DBNet model can increase the extraction of image features in the horizontal direction, but will not completely cancel the image features in the vertical direction.

[0104] In one embodiment, the text line detection model includes a DBNet model, and the optimized text line detection model is obtained by replacing some 3*3 convolution kernels in the skeleton network ResNet of the DBNet model with multiple 1*3 convolution kernels.

[0105] For example, in the skeleton network ResNet of the DBNet model, the convolution kernels in the 3rd, 4th, and 5th stages all use deformable convolutions. Deformable convolutions have good compatibility with targets of various shapes, but are not as targeted as horizontal convolution kernels. Therefore, in the skeleton network ResNet of the DBNet model, the convolution in the 3rd stage is replaced, and the 3*3 deformable convolution kernel is replaced with a 2-layer 1*3 convolution kernel. Here, the convolution in the 3rd stage is replaced, and the convolution in the 4th and 5th stages remain unchanged. If all the convolutions in each stage are replaced, the optimized DBNet model can only extract horizontal features, which weakens its generalization. The receptive field of the optimized DBNet model in the horizontal direction is three times that of the original one. Compared with deformable convolution, it can extract horizontal features more directly and accurately, and the number of parameters involved in the DBNet model at this stage is greatly reduced.

[0106] The present invention optimizes the convolution kernels of some stages in the existing DBNet model, which not only improves the horizontal accuracy of the optimized DBNet model, but also improves the processing speed of the optimized DBNet model.

[0107] In one embodiment, in step S103, obtaining a text bounding box corresponding to each text content in the target image according to the binary image includes:

[0108] Obtaining the contracted text area of ​​the text content in the target image according to the binary image;

[0109] The shrunken text area is expanded to obtain a text enclosing box corresponding to each text content in the target image.

[0110] The present disclosure obtains a text bounding box corresponding to each text content in a target image according to a binary image, so as to facilitate the acquisition of text information.

[0111] Based on the same inventive concept, the present disclosure also provides a text information acquisition device, see Figure 7 The device 1300 includes an acquisition module 1301, an execution module 1302, an expansion module 1303, a post-processing module 1304 and a recognition module 1305.

[0112] The acquisition module 1301 is configured to acquire a target image.

[0113] The execution module 1302 is configured to input the target image into the optimized text line detection model to obtain a binary image of the text content in the target image output by the text line detection model.

[0114] The expansion module 1303 is configured to obtain a text bounding box corresponding to each text content in the target image according to the binary image.

[0115] The post-processing module 1304 is configured to perform post-processing on each text enclosing frame to obtain a target text enclosing frame corresponding to the target image.

[0116] The recognition module 1305 is configured to perform text recognition on the text content in the target text enclosing box to obtain the text information of the target image.

[0117] The present invention obtains a binary image of text content in a corresponding target image through an optimized text line detection model, obtains a text bounding box of each text content in the corresponding target image according to the binary image, so as to obtain the text content in the text bounding box, thereby avoiding a non-robust recognition result, and obtains a target text bounding box of the corresponding target image by merging intersecting text bounding boxes with overlapping contents through post-processing of the text bounding boxes, thereby avoiding sentence disconnection due to the presence of long sentences with uneven text spacing in the text of the target image.

[0118] Further, the post-processing module 1304 is configured to traverse each text bounding box and determine whether there is an intersection between two adjacent text bounding boxes;

[0119] When two adjacent text bounding boxes have an intersection, the two adjacent text bounding boxes are screened;

[0120] The two adjacent text bounding boxes after screening are merged to obtain the target text bounding box.

[0121] Further, the post-processing module 1304 is configured to obtain the upper, lower, left, and right boundary values ​​of the first text enclosing box and the second text enclosing box in the coordinate axis Oxy;

[0122] Determine, according to the boundary value, whether the first text enclosing box and the second text enclosing box have an intersection in the X-axis direction or the Y-axis direction of the coordinate axis Oxy;

[0123] When the first text enclosing box intersects the second text enclosing box in the X-axis direction or the Y-axis direction, it is determined that the first text enclosing box and the second text enclosing box have an intersection, and two adjacent text enclosing boxes include the first text enclosing box and the second text enclosing box.

[0124] Further, the post-processing module 1304 is configured to calculate the intersection-and-union ratio of the first text enclosing box and the second text enclosing box;

[0125] When the intersection-over-area ratio is greater than zero, it is determined that the first text enclosing box and the second text enclosing box have an intersection.

[0126] Further, the post-processing module 1304 is configured to obtain a minimum value x1 and a maximum value x2 of the first text enclosing box on the X-axis, and a minimum value x3 and a maximum value x4 of the second text enclosing box on the X-axis;

[0127] In the case of x1<x3, x2<x4 and x3<x2, the first text enclosing box is determined to be the preceding text enclosing box, and the second text enclosing box is determined to be the succeeding text enclosing box;

[0128] A target preceding text enclosing box and a target following text enclosing box are screened out from the preceding text enclosing box and the following text enclosing box, and the target preceding text enclosing box and the target following text enclosing box meet the following conditions: an intersection of coordinate intervals of the target preceding text enclosing box and the target following text enclosing box in the X-axis direction is less than a first preset ratio of a union of coordinate intervals, and a distance between center points of two adjacent sides of the target preceding text enclosing box and the target following text enclosing box in the Y-axis direction does not exceed a second preset ratio of the shortest sides of the two text enclosing boxes.

[0129] Furthermore, the post-processing module 1304 is configured to obtain the coordinates of each vertex of the target preceding text enclosing box and the target following text enclosing box;

[0130] The minimum circumscribed rectangle of the target preceding text enclosing box and the target following text enclosing box is determined according to the coordinates of each vertex, and is used as the target text enclosing box.

[0131] Furthermore, the device 1300 also includes an optimization module, which is configured to replace some 3*3 convolution kernels in the skeleton network ResNet of the DBNet model with multiple 1*3 convolution kernels.

[0132] Further, the expansion module 1303 is configured to obtain a contracted text area of ​​the text content in the target image according to the binary image;

[0133] The shrunken text area is expanded to obtain a text enclosing box corresponding to each text content in the target image.

[0134] In addition, it is worth noting that for the convenience and simplicity of description, the embodiments described in the specification are all preferred embodiments, and the parts involved are not necessarily necessary for the present invention. For example, the acquisition module and the execution module can be independent devices or the same device in specific implementation, and the present disclosure does not limit this.

[0135] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0136] Based on the same inventive concept, the present disclosure also provides an electronic device, including:

[0137] a memory having a computer program stored thereon;

[0138] The processor is used to execute the computer program in the memory to implement the steps of the text information acquisition method.

[0139] Figure 8 FIG. 7 is a block diagram of an electronic device 700 according to an exemplary embodiment. Figure 8 As shown, the electronic device 700 may include: a processor 701 and a memory 702. The electronic device 700 may also include one or more of a multimedia component 703, an input / output (I / O) interface 704, and a communication component 705.

[0140] The processor 701 is used to control the overall operation of the electronic device 700 to complete all or part of the steps in the above-mentioned text information acquisition method. The memory 702 is used to store various types of data to support the operation of the electronic device 700, and these data may include instructions for any application or method used to operate on the electronic device 700, as well as application-related data, such as images, binary images, etc. The memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, disk or optical disk. The multimedia component 703 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 702 or sent through the communication component 705. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 704 provides an interface between the processor 701 and other interface modules, and the above-mentioned other interface modules may be keyboards, mice, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more of them, is not limited here. Therefore, the corresponding communication component 705 may include: Wi-Fi module, Bluetooth module, NFC module, etc.

[0141] In an exemplary embodiment, the electronic device 700 can be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned text information acquisition method.

[0142] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned text information acquisition method are implemented. For example, the computer-readable storage medium can be the above-mentioned memory 702 including program instructions, and the above-mentioned program instructions can be executed by the processor 701 of the electronic device 700 to complete the above-mentioned text information acquisition method.

[0143] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program that can be executed by a programmable device. The computer program has a code portion for executing the above-mentioned text information acquisition method when executed by the programmable device.

[0144] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.

[0145] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0146] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A method for obtaining text information, characterized in that: The method comprises: Get the target image; Inputting the target image into the optimized text line detection model to obtain a binary image corresponding to the text content in the target image output by the text line detection model; Acquire a text bounding box corresponding to each text content in the target image according to the binary image; Post-processing each of the text enclosing frames to obtain a target text enclosing frame corresponding to the target image; Performing text recognition on the text content in the target text enclosing box to obtain text information corresponding to the target image; The post-processing of each of the text enclosing frames to obtain a target text enclosing frame corresponding to the target image includes: Traversing each of the text enclosing boxes to determine whether two adjacent text enclosing boxes have an intersection; When two adjacent text enclosing boxes have an intersection, obtain the minimum value x1 and the maximum value x2 of the first text enclosing box on the X-axis, and the minimum value x3 and the maximum value x4 of the second text enclosing box on the X-axis; In the case of x1<x3, x2<x4 and x3<x2, determining that the first text enclosing box is the preceding text enclosing box and the second text enclosing box is the succeeding text enclosing box; A target preceding text enclosing box and a target following text enclosing box are selected from the preceding text enclosing box and the following text enclosing box, wherein the target preceding text enclosing box and the target following text enclosing box meet the following conditions: the intersection of the coordinate intervals of the target preceding text enclosing box and the target following text enclosing box in the X-axis direction is less than a first preset ratio of the union of the coordinate intervals, and the distance between the center points of the two adjacent sides of the target preceding text enclosing box and the target following text enclosing box in the Y-axis direction does not exceed a second preset ratio of the shortest sides of the two text enclosing boxes; The two adjacent text enclosing frames after screening are merged to obtain a target text enclosing frame.

2. The method according to claim 1, characterized in that The two adjacent text enclosing boxes include a first text enclosing box and a second text enclosing box. The determining whether two adjacent text bounding boxes have an intersection includes: Obtaining the upper, lower, left, and right boundary values ​​of the first text enclosing box and the second text enclosing box on the coordinate axis Oxy; Determine, according to the boundary value, whether the first text enclosing box and the second text enclosing box intersect in the X-axis direction or the Y-axis direction of the coordinate axis Oxy; When the first text enclosing box intersects with the second text enclosing box in the X-axis direction or the Y-axis direction, it is determined that the first text enclosing box and the second text enclosing box have an intersection.

3. The method according to claim 2, characterized in that The method further comprises: When the first text enclosing box and the second text enclosing box do not intersect in the X-axis direction and the Y-axis direction, calculating the intersection-and-combination ratio area of ​​the first text enclosing box and the second text enclosing box; When the intersection-over-area ratio is greater than zero, it is determined that the first text enclosing box and the second text enclosing box have an intersection.

4. The method according to claim 1, characterized in that: The step of merging the two adjacent text enclosing frames after filtering to obtain a target text enclosing frame includes: Obtaining the coordinates of each vertex of the target-before text enclosing box and the target-after text enclosing box; The minimum circumscribed rectangle of the target preceding text enclosing box and the target following text enclosing box is determined according to the vertex coordinates, and is used as the target text enclosing box.

5. The method according to claim 1, characterized in that The text line detection model includes a DBNet model, and the optimized text line detection model is obtained by replacing part of the skeleton network ResNet of the DBNet model with The convolution kernel is replaced by multiple Convolution kernel.

6. The method according to claim 1, characterized in that The step of obtaining a text bounding box corresponding to each text content in the target image according to the binary image includes: Obtaining a contracted text area of ​​text content in the target image according to the binary image; The shrunken text area is expanded to obtain a text enclosing box corresponding to each text content in the target image.

7. A text information acquisition device, characterized in that: The device comprises: An acquisition module, configured to acquire a target image; An execution module is configured to input the target image into the optimized text line detection model to obtain a binary image output by the text line detection model corresponding to the text content in the target image; An expansion module is configured to obtain a text bounding box corresponding to each text content in the target image according to the binary image; A post-processing module is configured to perform post-processing on each of the text enclosing frames to obtain a target text enclosing frame corresponding to the target image; A recognition module is configured to perform text recognition on the text content in the target text enclosing box to obtain text information corresponding to the target image; The post-processing module is configured to: Traversing each of the text enclosing boxes to determine whether two adjacent text enclosing boxes have an intersection; When two adjacent text enclosing boxes have an intersection, obtain the minimum value x1 and the maximum value x2 of the first text enclosing box on the X-axis, and the minimum value x3 and the maximum value x4 of the second text enclosing box on the X-axis; In the case of x1<x3, x2<x4 and x3<x2, determining that the first text enclosing box is the preceding text enclosing box and the second text enclosing box is the succeeding text enclosing box; A target preceding text enclosing box and a target following text enclosing box are selected from the preceding text enclosing box and the following text enclosing box, wherein the target preceding text enclosing box and the target following text enclosing box meet the following conditions: the intersection of the coordinate intervals of the target preceding text enclosing box and the target following text enclosing box in the X-axis direction is less than a first preset ratio of the union of the coordinate intervals, and the distance between the center points of the two adjacent sides of the target preceding text enclosing box and the target following text enclosing box in the Y-axis direction does not exceed a second preset ratio of the shortest sides of the two text enclosing boxes; The two adjacent text enclosing frames after screening are merged to obtain a target text enclosing frame.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

9. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 6.