Text processing method, device, storage medium and computer device

Abstract

The text processing method, device, storage medium and computer equipment provided by this application can obtain the text image and the corresponding Gaussian heat map before performing OCR recognition on the text image. The Gaussian heat map contains the target text area. Therefore, this application can only determine the text boxes and text box types in the target text area without paying attention to other areas in the Gaussian heat map, which can not only accurately correct but also improve the processing efficiency. After determining the text box type corresponding to the text box in the target text area, this application can also make targeted adjustments for different text box types. In this way, the obtained target text not only contains the original text information but also simplifies the text recognition difficulty, thereby effectively improving the text recognition accuracy.

Description

Technical Field

[0001] The present application relates to the field of image processing technologies, and particularly to a text processing method, apparatus, storage medium, and computer device. Background Art

[0002] OCR recognition (Optical Character Recognition) refers to the process of analyzing and recognizing an image file of text materials to obtain text and layout information. That is, recognizing the text in the image and returning it in text form. Due to the characteristics of high accuracy, strong stability, simplicity and ease of use of OCR recognition technology, it is widely used in the fields of education, medical care, industry, etc. For example, the OCR recognition technology can be applied to a PDA (scanning gun) as a supplement to barcode and two-dimensional code recognition.

[0003] Currently, in the process of applying OCR recognition technology to a PDA, due to the unfixed recognition scenario, the recognized text may be curved, of different sizes, etc., resulting in a low accuracy of the final recognition result. Summary of the Invention

[0004] The purpose of the present application aims to solve at least one of the above technical defects, especially the technical defect that in the process of applying OCR recognition technology to a PDA in the prior art, due to the unfixed recognition scenario, the recognized text may be curved, of different sizes, etc., resulting in a low accuracy of the final recognition result.

[0005] The present application provides a text processing method, and the method includes:

[0006] Obtain a text image to be processed and a Gaussian heat map corresponding to the text image, where the Gaussian heat map contains a target text area;

[0007] Determine a text box and a text box type in the target text area, where the text box type includes a curved text box and a straight-line text box;

[0008] If the text box is a curved text box, segment the text box according to the bending degree of the text box, and crop and splice the text in the text image according to the segmented text box to obtain target text;

[0009] If the text box is a straight-line text box, crop the text at the same position in the text image according to the number of text boxes in the target text area and the position of the text box in the Gaussian heat map to obtain target text.

[0010] Optionally, the determining the text box in the target text area includes:

[0011] After performing at least one binarization operation on the Gaussian heat map according to a preset binarization threshold, a binarized image is obtained. When performing more than two binarization operations on the Gaussian heat map, the binarization thresholds used in each binarization operation are different;

[0012] Determine the minimum bounding rectangle corresponding to each text contour located in the target text area in the binarized image, and divide each minimum bounding rectangle according to the text corresponding to it in the text image to obtain text boxes.

[0013] Optionally, the text box in the target text area includes at least one minimum bounding rectangle;

[0014] The text box type for determining the text box in the target text area includes:

[0015] Judge whether the number of minimum bounding rectangles included in the text box in the target text area exceeds a preset value;

[0016] If not, determine that the text box is a straight-line text box;

[0017] If it exceeds, determine the text box type of the text box according to the midpoint positions of the minimum bounding rectangles in the text box.

[0018] Optionally, the determining the text box type of the text box according to the midpoint positions of the minimum bounding rectangles in the text box includes:

[0019] According to the arrangement order of the midpoint positions of the minimum bounding rectangles in the text box, connect the midpoints of the two adjacent minimum bounding rectangles with the earliest arrangement order to obtain a reference line;

[0020] According to the arrangement order, connect the midpoints of the minimum bounding rectangles after the two adjacent minimum bounding rectangles to the midpoint of the minimum bounding rectangle with the later arrangement order among the two adjacent minimum bounding rectangles in turn to obtain at least one measurement line;

[0021] Judge whether the angle between the reference line and each measurement line is greater than a preset angle value;

[0022] If at least one angle is greater than the preset angle value, determine that the text box is a curved text box;

[0023] If all the angles are not greater than the preset angle value, determine that the text box is a straight-line text box.

[0024] Optionally, the dividing the text box according to the bending degree of the text box to obtain the divided text box includes:

[0025] Obtain the measurement line corresponding to when it is first determined that the included angle is greater than the preset angle value;

[0026] Take the minimum bounding rectangle corresponding to the end point with the later arrangement order among the two end points of the measurement line as the first minimum bounding rectangle, and take the minimum bounding rectangle corresponding to the end point with the later sorting order among the two end points of the previous measurement line of the measurement line as the second minimum bounding rectangle;

[0027] According to the positional relationship between the first minimum bounding rectangle and the second minimum bounding rectangle, divide the text box to obtain a text box composed of the minimum bounding rectangles with the earlier arrangement order, and a text box composed of the minimum bounding rectangles with the later arrangement order;

[0028] Judge the number of minimum bounding rectangles in the text box composed of the minimum bounding rectangles with the later arrangement order;

[0029] If the number of the minimum bounding rectangles is less than two, end the splitting operation;

[0030] If the number of the minimum bounding rectangles is at least three, take the text box composed of the minimum bounding rectangles with the later arrangement order as the new text box, and return to execute the step of connecting the midpoints of the two adjacent minimum bounding rectangles with the earliest arrangement order according to the arrangement order of the midpoint positions of the respective minimum bounding rectangles in the text box to obtain the reference line, until the splitting operation ends, to obtain the split text box.

[0031] Optionally, the dividing the text box according to the positional relationship between the first minimum bounding rectangle and the second minimum bounding rectangle includes:

[0032] Connect the midpoints of the first minimum bounding rectangle and the second minimum bounding rectangle to obtain a rectangle connection line;

[0033] Taking the midpoint position of the rectangle connection line as the origin, draw a dividing line parallel to the right rectangle side of the second minimum bounding rectangle, and divide the text box according to the dividing line.

[0034] Optionally, the clipping and splicing the text in the text image according to the divided text box to obtain the target text includes:

[0035] Dilate the divided text box according to a preset dilation multiple to obtain a dilated text box;

[0036] According to the dilated text box, clip the text in the corresponding area of the text image to obtain a plurality of initial texts;

[0037] According to the arrangement order of each initial text in the text image, splice each initial text to obtain the target text.

[0038] Optionally, the method of cropping the text at the same position in the text image according to the number of text boxes in the target text area and the positions of the text boxes in the Gaussian heat map to obtain the target text includes:

[0039] When there is one text box, crop the text at the same position in the text image according to the position of the text box in the Gaussian heat map to obtain the target text;

[0040] When there are multiple text boxes, determine whether there are text boxes in the same row according to the positions of the respective text boxes in the Gaussian heat map;

[0041] If so, merge the text boxes in the same row, and crop the text in the text image according to the merged text box to obtain the target text;

[0042] If not, crop the text at the same position in the text image according to the position of each text box in the Gaussian heat map to obtain the target text.

[0043] Optionally, the method of determining whether there are text boxes in the same row according to the positions of the respective text boxes in the Gaussian heat map includes:

[0044] Select any two text boxes from all the text boxes, and determine the positions of the selected any two text boxes in the Gaussian heat map;

[0045] According to the positions of the selected any two text boxes in the Gaussian heat map, confirm whether the selected any two text boxes are in the same row to obtain a confirmation result;

[0046] Judge whether all the text boxes have been selected;

[0047] If so, determine whether there are text boxes in the same row according to the confirmation result;

[0048] If not, continue to select any two text boxes from all the text boxes. When there is one different text box between the any two text boxes selected this time and the any two text boxes selected last time, return to execute the step of determining the positions of the selected any two text boxes in the Gaussian heat map until all the text boxes have been selected.

[0049] Optionally, the method of confirming whether the selected any two text boxes are in the same row according to the positions of the selected any two text boxes in the Gaussian heat map includes:

[0050] Determine the first inclination angle of each text box in the Gaussian heat map according to the positions of any two of the text boxes in the Gaussian heat map, and determine the second inclination angle of the midpoint line obtained by connecting the midpoints of each text box in the Gaussian heat map;

[0051] Based on the first inclination angle and the second inclination angle, preliminarily determine whether any two of the text boxes are in the same row;

[0052] If so, according to the positions of any two of the text boxes in the Gaussian heat map, determine the upper edge line and the lower edge line of one of the text boxes in the Gaussian heat map, and the perpendicular line between the midpoint of the other text box and the upper edge line and the lower edge line;

[0053] Based on the upper edge line, the lower edge line, and the perpendicular line between the upper edge line and the lower edge line, finally determine whether any two of the text boxes are in the same row.

[0054] Optionally, the merging of the text boxes in the same row to obtain the merged text box includes:

[0055] Mark the text boxes in the same row and the text boxes not in the same row in the Gaussian heat map respectively to obtain a connected graph composed of each marked text box;

[0056] Traverse the connected graph through DFS, and determine the connected components in the connected graph according to the traversal result, and the connected components are the merged text boxes.

[0057] Optionally, before merging the text boxes in the same row, it further includes:

[0058] Determine the aspect ratios of the text boxes in the same row;

[0059] Compare the aspect ratio of each text box with a preset aspect ratio range. If the aspect ratio of at least one text box is not within the aspect ratio range, correct the inclination angle of the at least one text box.

[0060] Optionally, the correction of the inclination angle of the at least one text box includes:

[0061] Select the text box with the longest box length in the Gaussian heat map as the reference text box;

[0062] Correct the inclination angle of the at least one text box according to the inclination angle of the reference text box in the Gaussian heat map.

[0063] Optionally, obtaining the Gaussian heat map corresponding to the text image includes:

[0064] Inputting the text image into a pre-configured deep learning detection model to obtain the Gaussian heat map corresponding to the text image output by the deep learning detection model;

[0065] Wherein, the deep learning detection model is trained with multiple groups of training text images and augmented images obtained by augmenting the multiple groups of training text images as training samples, and the true Gaussian heat maps corresponding to each group of training text images and augmented images as sample labels.

[0066] This application also provides a text processing device, including:

[0067] An image acquisition module, configured to acquire a text image to be processed and a Gaussian heat map corresponding to the text image, where the Gaussian heat map contains a target text area;

[0068] A text box determination module, configured to determine a text box and a text box type in the target text area, where the text box type includes a curved text box and a straight text box;

[0069] A first processing module, configured to, if the text box is a curved text box, segment the text box according to the bending degree of the text box, and crop and splice the text in the text image according to the segmented text box to obtain a target text;

[0070] A second processing module, configured to, if the text box is a straight text box, crop the text at the same position in the text image according to the number of text boxes in the target text area and the positions of the text boxes in the Gaussian heat map to obtain a target text.

[0071] This application also provides a storage medium, in which computer-readable instructions are stored. When the computer-readable instructions are executed by one or more processors, the one or more processors are caused to execute the steps of the text processing method as described in any one of the above embodiments.

[0072] This application also provides a computer device, including: one or more processors and a memory.

[0073] The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the text processing method as described in any one of the above embodiments are executed.

[0074] From the above technical solutions, it can be seen that the embodiments of this application have the following advantages:

[0075] The text processing method, device, storage medium, and computer equipment provided by this application can obtain the text image and the corresponding Gaussian heat map before performing OCR recognition on the text image, so that the background irrelevant to the text in the text image can be filtered out, making it more convenient to process the text. After obtaining the Gaussian heat map, since the target text area is included in the Gaussian heat map, this application can only determine the text boxes and text box types in the target text area without paying attention to other areas in the Gaussian heat map, which can not only correct accurately but also improve the processing efficiency. After determining the text box type corresponding to the text box in the target text area, this application can also make targeted adjustments for different text box types. For example, when the text box is a curved text box, the text box can be segmented according to the curvature of the text box, and the text in the text image can be cropped and spliced according to the segmented text boxes. In this way, the obtained target text not only contains the original text information but also simplifies the text recognition difficulty, thus effectively improving the text recognition accuracy. When the text box is a straight text box, this application can crop the text at the same position in the text image according to the number of text boxes in the target text area and the positions of each text box in the Gaussian heat map. For example, when there is one text box, the text at the same position as the text box in the text image can be directly cropped. When there are multiple text boxes, it can be determined whether there are text boxes in the same row. If so, the text boxes in the same row are merged, and the text in the text image is cropped using the merged text box. This can not only improve the text recognition efficiency but also solve the problem that large and small characters are in the same row in the text box, thereby effectively improving the text recognition accuracy. Brief Description of the Drawings

[0076] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of this application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0077] Figure 1 It is a schematic flowchart of a text processing method provided by an embodiment of this application;

[0078] Figure 2 It is the Gaussian heat map corresponding to the text image in different recognition scenarios provided by an embodiment of this application;

[0079] Figure 3 It is a schematic structural diagram of the segmented text box obtained after performing a segmentation operation on a curved text box provided by an embodiment of this application;

[0080] Figure 4 A structural schematic diagram for initially confirming whether two text boxes are on the same line provided by an embodiment of the present application;

[0081] Figure 5 A structural schematic diagram for finally confirming whether two text boxes are on the same line provided by an embodiment of the present application;

[0082] Figure 6 A structural schematic diagram of a text processing device provided by an embodiment of the present application;

[0083] Figure 7 An internal structural schematic diagram of a computer device provided by an embodiment of the present application. Detailed implementation manners

[0084] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0085] Currently, in the process of applying the OCR recognition technology to the PDA, due to the unfixed recognition scenario, the recognized text may be curved, of different sizes, etc., resulting in a low accuracy of the final recognition result. Based on this, the present application proposes the following technical solutions. For details, please refer to the following:

[0086] In one embodiment, as Figure 1 shown, Figure 1 A flowchart of a text processing method provided by an embodiment of the present application; the present application provides a text processing method, and the method may include:

[0087] S110: Obtain a text image to be processed and a Gaussian heat map corresponding to the text image.

[0088] In this step, after obtaining the text image to be processed, immediately, the present application can obtain the Gaussian heat map corresponding to the text image and adjust the text in the text image through the Gaussian heat map.

[0089] It can be understood that in the non-deep learning field, a heatmap simply aggregates a large amount of data and uses a progressive color band to elegantly represent it, intuitively showing the density or frequency of spatial data. In the deep learning field, however, a heatmap can reflect the location of the detected object. For example, in a deep learning neural network, a heatmap helps to understand which part of an image makes the final classification decision by the deep learning neural network. Among them, there are two ways to generate a heatmap, one is a Gaussian heatmap, and the other is a class activation heatmap generated by Grad-CAM. When this application recognizes a text image, in order to better distinguish the text area and non-text area in the text image, the Gaussian heatmap corresponding to the text image is used to adjust the text in the text image.

[0090] Schematically, as Figure 2 shown, Figure 2 is the Gaussian heatmap corresponding to the text image in different recognition scenarios provided by the embodiments of this application; from Figure 2 the left image in it, it can be seen that this application can obtain text images in different recognition scenarios, such as text images corresponding to parts in a warehouse, text images corresponding to product information, and text images corresponding to license plates, etc. Since the backgrounds in the text images in different recognition scenarios are significantly different, in order to avoid the influence of the background in the text image when recognizing the text subsequently, this application can generate the corresponding Gaussian heatmap for the obtained text image, and use the Gaussian heatmap to weaken the background in the text image and highlight the area where the text in the text image is located, as Figure 2 can be seen from the right image. After generating the Gaussian heatmap, the black area in the Gaussian heatmap is the area corresponding to the background of the text image, and the area circled by the white circle in the Gaussian heatmap is the area corresponding to the text in the text image. The area where the text in the text image is located can be quickly obtained through the Gaussian heatmap.

[0091] Furthermore, the Gaussian heatmap of this application contains a target text area, which is the area corresponding to the target text to be recognized in the text image. It can be understood that this application uses the aiming point positioning as the reference position, and captures several target texts to be recognized that appear in the field of view (FOV) in the text image through the OCR recognition component. After generating the Gaussian heatmap corresponding to the text image, the target text area in the Gaussian heatmap is filtered out according to the area corresponding to the target text of the text image, which can avoid recognizing other interfering texts in the text image, thereby effectively improving the OCR recognition efficiency and recognition accuracy.

[0092] S120: Determine the text boxes and their types in the target text area. The text box types include curved text boxes and straight-line text boxes. If the text box is a curved text box, then execute S130; if the text box is a straight-line text box, then execute S140.

[0093] In this step, after obtaining the text image to be processed and the corresponding Gaussian heat map through S110, the present application can determine the text boxes and their types in the target text area. The text box types can include curved text boxes and straight-line text boxes. Among them, a curved text box can be a text box with one or more curved arcs. When there are multiple curved arcs in the text box, each curved arc can be the same or different; a straight-line text box can be a straight-line text box at any angle, which is not limited here.

[0094] In addition, since there may be one or more text boxes in the target text area, the present application can simultaneously determine one or more text boxes in the target text area and the text box type of each text box.

[0095] S130: Segment the text box according to the degree of curvature of the text box, and crop and splice the text in the text image based on the segmented text box to obtain the target text.

[0096] In this step, after determining the text boxes and their types in the target text area through S120, if the text box is a curved text box, the text box can be segmented according to the degree of curvature of the text box, and the text in the text image can be cropped and spliced based on the segmented text box, so as to obtain the target text.

[0097] Such as Figure 2 the Gaussian heat map corresponding to the first text image. The text box in this Gaussian heat map is a curved text box. For such a curved text box, the present application can segment the text box according to the degree of curvature of the text box. For example, the text box can be segmented into multiple text boxes similar to straight lines, and then the text in the corresponding area of each text box in the text image is cropped to obtain multiple texts. Since each obtained text is an incomplete text, the present application can splice the various texts after obtaining multiple texts, so as to obtain the target text.

[0098] S140: Crop the text at the same position in the text image according to the number of text boxes in the target text area and the positions of the text boxes in the Gaussian heat map to obtain the target text.

[0099] In this step, after determining the text boxes and their types in the target text area through S120, if the text box is a straight-line text box, the text at the same position in the text image can be cropped according to the number of text boxes in the target text area and the positions of the text boxes in the Gaussian heat map, so as to obtain the target text.

[0100] For example, when the text box is a straight-line text box and the number of text boxes is one, the present application can directly crop the text at the same position in the text image according to the position of the text box. When the number of text boxes is multiple, the present application can determine whether there are text boxes in the same row according to the positions of the text boxes. If so, the text boxes in the same row can be merged, and the text at the same position in the text image can be cropped according to the position of the merged text box. This can not only improve the cropping speed, but also crop the text boxes not in the same row separately, solving the problem that large and small characters in the text box are in the same row.

[0101] In the above embodiments, before performing OCR recognition on the text image, the text image and the corresponding Gaussian heat map can be obtained, so that the background irrelevant to the text in the text image can be filtered out, making it more convenient to process the text. After obtaining the Gaussian heat map, since the Gaussian heat map contains the target text area, the present application can only determine the text boxes and their types in the target text area without paying attention to other areas in the Gaussian heat map, which can not only accurately correct but also improve the processing efficiency. After determining the text box type corresponding to the text box in the target text area, the present application can also make targeted adjustments for different text box types. For example, when the text box is a curved text box, the text box can be segmented according to the curvature of the text box, and the text in the text image can be cropped and spliced according to the segmented text box. The obtained target text not only contains the original text information but also simplifies the text recognition difficulty, thus effectively improving the text recognition accuracy. When the text box is a straight-line text box, the present application can crop the text at the same position in the text image according to the number of text boxes in the target text area and the positions of each text box in the Gaussian heat map. For example, when there is one text box, the text at the same position as the text box in the text image can be directly cropped. When there are multiple text boxes, it can be determined whether there are text boxes in the same row. If so, the text boxes in the same row can be merged, and the text in the text image can be cropped using the merged text box. This can not only improve the text recognition efficiency but also solve the problem that large and small characters in the text box are in the same row, thereby effectively improving the text recognition accuracy.

[0102] In one embodiment, determining the text boxes in the target text area in S120 may include:

[0103] S121: Performing at least one binarization operation on the Gaussian heat map according to a preset binarization threshold to obtain a binarized image. When performing more than two binarization operations on the Gaussian heat map, the binarization thresholds used for each binarization operation are different.

[0104] S122: Determining the minimum bounding rectangle corresponding to each text contour located in the target text area in the binarized image, and dividing each minimum bounding rectangle according to the text corresponding to each minimum bounding rectangle in the text image to obtain text boxes.

[0105] In this embodiment, when determining the text boxes in the target text area, a binarized image can be obtained by performing at least one binarization operation on the Gaussian heat map according to a preset binarization threshold. Then, the minimum bounding rectangle corresponding to each text contour located in the target text area in the binarized image is determined. Finally, each minimum bounding rectangle is divided according to the text corresponding to each minimum bounding rectangle in the text image, so as to obtain text boxes. The text boxes can contain one minimum bounding rectangle or multiple minimum bounding rectangles.

[0106] It can be understood that when performing a binarization operation on an image, the larger the binarization threshold selected, the smaller the single-character boxes in the image. Therefore, the present application can use a relatively large binarization threshold to perform a binarization operation on the Gaussian heat map. Of course, the present application can also first select a relatively small binarization threshold to perform a binarization operation on the Gaussian heat map, and determine whether the single characters in the obtained binarized image are separated. If the single characters cannot be separated, a relatively large binarization threshold is continuously used to perform a binarization operation on the Gaussian heat map, so as to obtain a final binarized image in which the single characters are separated, resulting in a higher accuracy when determining the type of text box subsequently.

[0107] Specifically, the present application can first perform a binarization operation on the Gaussian heat map with a threshold of 0.4, and find the contour bounding rectangles corresponding to each character in the Gaussian heat map after the binarization operation. If the obtained contour bounding rectangles cover multiple characters at this time, the Gaussian heat map can be continuously binarized with a threshold of 0.6. The size of each text area in the obtained binarized image is about 0.7 times the size of the original text area. After obtaining the binarized image, the present application can find the minimum bounding rectangle corresponding to each text contour in the binarized image, and finally obtain a series of boxes.

[0108] Further, due to a certain temporal sequence between the characters in the text image, and there are also differences in the character sizes of each region. For example, Figure 2The characters representing the date are unified into a single line and form a separate text box, while the characters representing whether the product is qualified form a separate line, and the character sizes between the two are different. Therefore, in this application, each minimum bounding rectangle can be divided according to the text corresponding to it in the text image, so as to obtain one or more text boxes, and each text box contains at least one minimum bounding rectangle. This is not only beneficial for judging the text box type, but also beneficial for subsequent text recognition.

[0109] In one embodiment, the text box in the target text area contains at least one minimum bounding rectangle; determining the text box type of the text box in the target text area in S120 may include:

[0110] S210: Determine whether the number of minimum bounding rectangles contained in the text box in the target text area exceeds a preset value; if not, execute S211, if so, execute S212.

[0111] S211: Determine that the text box is a straight-line text box.

[0112] S212: Determine the text box type of the text box according to the midpoint positions of the minimum bounding rectangles in the text box.

[0113] In this embodiment, when determining the text box type of the text box in the target text area, it can be determined according to the number of minimum bounding rectangles contained in each text box.

[0114] Specifically, this application can first determine whether the number of minimum bounding rectangles contained in the text box in the target text area exceeds a preset value. If not, the text box is a straight-line text box. If so, the text box type of the text box can be determined according to the midpoint positions of the minimum bounding rectangles in the text box.

[0115] It can be understood that when a certain line segment is short, it can be approximately regarded as a straight line segment. Similarly, when a text box contains only one or a few minimum bounding rectangles, it can also be approximately regarded as a straight-line text box. For example, when a text box contains only one or two minimum bounding rectangles, the text box can be regarded as a straight-line text box. When the number of minimum bounding rectangles contained in the text box exceeds the preset value, it is impossible to directly determine whether the text box is a straight-line text box or a curved text box. At this time, the text box type corresponding to the text box can be determined according to the midpoint positions of the minimum bounding rectangles in the text box.

[0116] In one embodiment, S212 determines the text box type of the text box according to the midpoint positions of the minimum bounding rectangles in the text box, and may include:

[0117] S2121: Connect the midpoints of the two adjacent minimum bounding rectangles with the earliest arrangement order among the midpoint positions of the respective minimum bounding rectangles in the text box to obtain a reference line.

[0118] S2122: According to the arrangement order, connect the midpoints of the minimum bounding rectangles after the two adjacent minimum bounding rectangles to the midpoint of the minimum bounding rectangle with the later arrangement order among the two adjacent minimum bounding rectangles in sequence to obtain at least one measurement line.

[0119] S2123: Determine whether the angle between the reference line and each measurement line is greater than a preset angle value; if at least one angle is greater than the preset angle value, execute S2124; if all angles are not greater than the preset angle value, execute S2125.

[0120] S2124: Determine that the text box is a curved text box.

[0121] S2125: Determine that the text box is a straight text box.

[0122] In this embodiment, when determining the text box type of the text box according to the midpoint positions of the respective minimum bounding rectangles in the text box, the midpoint positions of the respective minimum bounding rectangles in the text box can be sorted first to determine their arrangement order, and then the midpoints of the two adjacent minimum bounding rectangles with the earliest arrangement order can be connected, and the resulting line segment after connection can be used as the reference line.

[0123] Generally, when sorting the central positions of the respective minimum bounding rectangles in the text box, it can be in the order from left to right, or in the order from right to left, or according to the inclination angle of the text box in the binary image, the central positions of multiple minimum bounding rectangles from one end to the other end of the text box can be sorted in sequence. When sorting, it can be sorted according to the abscissa of the central positions of the respective minimum bounding rectangles, or according to the ordinate of the central positions of the respective minimum bounding rectangles, and there is no limitation here.

[0124] After obtaining the sorting order corresponding to the midpoint positions of the respective minimum bounding rectangles in the text box, and connecting the midpoints of the two adjacent minimum bounding rectangles with the earliest arrangement order to obtain the reference line, the present application can also, according to this arrangement order, connect the midpoints of the minimum bounding rectangles after the two adjacent minimum bounding rectangles to the midpoint of the minimum bounding rectangle with the later arrangement order among the two adjacent minimum bounding rectangles in sequence, and obtain at least one measurement line.

[0125] For example, if the text box contains 5 minimum bounding rectangles, after sorting each minimum bounding rectangle, a reference line is obtained by connecting the midpoints of the first and second sorted minimum bounding rectangles. Then, the midpoint of the third sorted minimum bounding rectangle is connected to the midpoint of the second sorted minimum bounding rectangle to obtain a measurement line. The midpoint of the fourth sorted minimum bounding rectangle is connected to the midpoint of the second sorted minimum bounding rectangle to obtain a measurement line. The midpoint of the fifth sorted minimum bounding rectangle is connected to the midpoint of the second sorted minimum bounding rectangle to obtain a measurement line. In this way, three measurement lines can be obtained.

[0126] Immediately afterwards, when at least one measurement line is obtained in the above manner in this application, it can be determined whether the angle between the reference line and each measurement line is greater than a preset angle value. If at least one angle is greater than the preset angle value, it indicates that the text box is a curved text box. If all angles are not greater than the preset angle value, it indicates that the text box is a straight text box.

[0127] It can be understood that after the reference line and the measurement line in this application intersect, an angle not less than zero degrees can be formed. When the angle is small and not greater than the preset angle value, it can be approximately considered that the minimum bounding rectangle corresponding to the current measurement line and the text box formed by all the minimum bounding rectangles before the current measurement line are straight text boxes. When the angles between all the measurement lines and the reference line in the text box are less than the preset angle value, it indicates that the text box is a straight text box; once an angle between the reference line and the measurement line is greater than the preset angle value, it indicates that there is a certain curvature in the text box formed by the minimum bounding rectangle corresponding to the current measurement line and all the minimum bounding rectangles before the current measurement line. At this time, without considering whether the angles between the subsequent measurement lines and the reference line are greater than the preset angle value, it can be directly determined that the current text box is a curved text box.

[0128] Furthermore, the preset angle value here can be set according to the actual situation. For example, the preset angle value can be 3°, 5°, etc., and there is no limitation here.

[0129] In one embodiment, S130 splitting the text box according to the bending degree of the text box to obtain the split text box may include:

[0130] S131: Obtain the measurement line corresponding to the first time when it is determined that the angle is greater than the preset angle value.

[0131] S132: Use the minimum bounding rectangle corresponding to the endpoint with the later arrangement order among the two endpoints of the measurement line as the first minimum bounding rectangle, and use the minimum bounding rectangle corresponding to the endpoint with the later arrangement order among the two endpoints of the previous measurement line of the measurement line as the second minimum bounding rectangle.

[0132] S133: Divide the text box according to the positional relationship between the first minimum bounding rectangle and the second minimum bounding rectangle to obtain a text box composed of the minimum bounding rectangles with the earlier arrangement order, and a text box composed of the minimum bounding rectangles with the later arrangement order.

[0133] S134: Determine the number of minimum bounding rectangles in the text box composed of the minimum bounding rectangles with the later arrangement order; if the number of the minimum bounding rectangles is less than two, execute S135; if the number of the minimum bounding rectangles is at least three, execute S136.

[0134] S135: End the segmentation operation.

[0135] S136: Use the text box composed of the minimum bounding rectangles with the later arrangement order as a new text box, and return to execute the steps of S2121 - S2125 and S131 - S134 until the segmentation operation ends to obtain the segmented text box.

[0136] In this embodiment, when segmenting a curved text box, the segmentation operation can be performed according to the degree of curvature of the text box, so as to obtain the segmented text box.

[0137] Specifically, since this application mainly connects the midpoints of the minimum bounding rectangles after two adjacent minimum bounding rectangles in the arrangement order in sequence with the midpoint of the minimum bounding rectangle with the later arrangement order among the two adjacent minimum bounding rectangles when determining the measurement line. Therefore, each measurement line connects two minimum bounding rectangles, one of which is the second minimum bounding rectangle in the arrangement order, and the other is the minimum bounding rectangle with the later arrangement order.

[0138] Therefore, when the present application needs to split a curved text box, it is possible to obtain a measurement line with an included angle greater than a preset angle value determined for the first time, and use the smallest circumscribed rectangle corresponding to the endpoint with a later arrangement order among the two endpoints of the measurement line as the first smallest circumscribed rectangle, and use the smallest circumscribed rectangle corresponding to the endpoint with a later arrangement order among the two endpoints of the previous measurement line of the measurement line as the second smallest circumscribed rectangle, that is, use the smallest circumscribed rectangle with a higher arrangement order adjacent to the first smallest circumscribed rectangle as the second smallest circumscribed rectangle. Then, according to the positional relationship between the first smallest circumscribed rectangle and the second smallest circumscribed rectangle, the text box can be split to obtain a text box composed of the smallest circumscribed rectangles with a higher arrangement order and a text box composed of the smallest circumscribed rectangles with a later arrangement order.

[0139] Further, after obtaining the text box composed of the smallest circumscribed rectangles with a higher arrangement order and the text box composed of the smallest circumscribed rectangles with a later arrangement order, the present application can continue to determine the number of the smallest circumscribed rectangles in the text box composed of the smallest circumscribed rectangles with a later arrangement order, and determine whether to end the splitting operation according to the number of the smallest circumscribed rectangles. For example, when the number of the smallest circumscribed rectangles is less than two, it indicates that the text box is a text box similar to a straight line. Therefore, there is no need to continue splitting the text box. When the number of the smallest circumscribed rectangles is at least three, it indicates that the text box may be a curved text box. If it is a curved text box, it is necessary to continue splitting the text box according to the degree of curvature of the text box. At this time, the text box composed of the smallest circumscribed rectangles with a later arrangement order can be used as a new text box, and the steps of determining the type of the text box in S2121 to S2125 can be returned for execution. And when it is determined that the text box is a curved text box, the steps of splitting the text box according to the degree of curvature of the text box in S131 to S134 are executed until the splitting operation ends, so that the split text box can be obtained.

[0140] Schematically, as Figure 3 shown, Figure 3 is a schematic structural diagram of the split text box obtained after performing the splitting operation on the curved text box provided by the embodiment of the present application; Figure 3 The text box in is a curved text box. When splitting the text box, the text box is split into three independent text boxes according to the degree of curvature of the text box. Finally, the text in the text image can be cropped and spliced according to these three text boxes.

[0141] In one embodiment, splitting the text box according to the positional relationship between the first smallest circumscribed rectangle and the second smallest circumscribed rectangle in S133 may include:

[0142] S1331: Connect the midpoints of the first minimum bounding rectangle and the second minimum bounding rectangle to obtain a rectangle connection line.

[0143] S1332: Taking the midpoint position of the rectangle connection line as the origin, draw a dividing line parallel to the right rectangle side of the second minimum bounding rectangle, and divide the text box according to the dividing line.

[0144] In this embodiment, after obtaining the first minimum bounding rectangle and the second minimum bounding rectangle, the midpoint of the first minimum bounding rectangle can be connected to the midpoint of the second minimum bounding rectangle to obtain a rectangle connection line. Then, taking the midpoint position of this rectangle connection line as the origin, a dividing line parallel to the right rectangle side of the second minimum bounding rectangle can be drawn, and the text box can be divided according to this dividing line. The text box obtained thereby can be approximately regarded as a straight-line text box.

[0145] In one embodiment, S130 may include, when cropping and splicing the text in the text image according to the divided text box to obtain the target text:

[0146] S310: Dilate the divided text box according to a preset dilation factor to obtain a dilated text box.

[0147] S311: Crop the text in the corresponding area of the text image according to the dilated text box to obtain a plurality of initial texts.

[0148] S312: Splice the respective initial texts according to the arrangement order of each initial text in the text image to obtain the target text.

[0149] In this embodiment, when cropping and splicing the text in the text image according to the divided text box, the divided text box can be dilated according to a preset dilation factor to obtain a dilated text box. Then, the text in the corresponding area of the text image can be cropped according to the dilated text box to obtain a plurality of initial texts. Finally, the respective initial texts are spliced according to the arrangement order of each initial text in the text image to obtain the target text.

[0150] It should be noted that in the process of determining the text box in the target text area, the text box does not necessarily correspond exactly to the text area in the text image, and may only be the central area of the text area. For example, when the present application uses a binarization threshold to perform a binarization operation on the Gaussian heat map to obtain a binarized image and searches for the minimum bounding rectangle corresponding to the text contour in the binarized image, in order to obtain the minimum bounding rectangle corresponding to each text contour, a relatively large binarization threshold is selected to perform the binarization operation on the Gaussian heat map. In this way, the obtained minimum bounding rectangle may be the central part of the corresponding text area in the text image. Therefore, when cropping and splicing the text in the text image according to the segmented text box, the segmented text box can be dilated according to a preset dilation factor. The dilated text box obtained in this way can correspond exactly to the text area in the text image. After processing in this way, it can be ensured that the obtained initial text is a complete text, which is beneficial to subsequent text recognition.

[0151] For example, when the present application performs a binarization operation on the Gaussian heat map with a threshold of 0.6, each minimum bounding rectangle in the obtained binarized image only contains the central part of the text. At this time, a dilation coefficient of 1.6 can be selected to dilate the text box, and the dilated text box can just enclose the text in the corresponding text area in the text image.

[0152] Further, when multiple initial texts are obtained, since the initial text is the segmented text, in order to obtain the complete text, the present application can splice the initial texts according to the arrangement order of each initial text in the text image to obtain the target text.

[0153] In one embodiment, in S140, cropping the text at the same position in the text image according to the number of text boxes in the target text area and the positions of the text boxes in the Gaussian heat map to obtain the target text may include:

[0154] S141: When there is one text box, crop the text at the same position in the text image according to the position of the text box in the Gaussian heat map to obtain the target text.

[0155] S142: When there are multiple text boxes, determine whether there are text boxes located in the same row according to the positions of the respective text boxes in the Gaussian heat map; if so, execute S143, if not, execute S144.

[0156] S143: Merge the text boxes located in the same row, and crop the text in the text image according to the merged text box to obtain the target text.

[0157] S144: Crop the text at the same position in the text image according to the position of each text box in the Gaussian heat map to obtain the target text.

[0158] In this embodiment, when the text box type is a linear text box, the text at the same position in the text image can be cropped according to the number of text boxes and the position of the text boxes in the Gaussian heat map, so as to obtain the target text.

[0159] In a specific implementation, when there is one text box in the target text area, the text at the same position in the text image can be directly cropped according to the position of this text box in the Gaussian heat map to obtain the target text; when there are multiple text boxes in the target text area, it is possible to determine whether there are text boxes in the same row according to the positions of each text box in the Gaussian heat map. If so, the text boxes in the same row are merged, and then the text in the text image is cropped according to the merged text box to obtain the target text. If not, the text at the same position in the text image can be cropped according to the position of each text box in the Gaussian heat map to obtain the target text.

[0160] In one embodiment, determining whether there are text boxes in the same row according to the positions of each text box in the Gaussian heat map in S142 may include:

[0161] S1421: Select any two text boxes from each text box and determine the positions of the selected any two text boxes in the Gaussian heat map.

[0162] S1422: According to the positions of the any two text boxes in the Gaussian heat map, confirm whether the any two text boxes are in the same row to obtain a confirmation result.

[0163] S1423: Determine whether each text box has been selected; if so, execute S1424; if not, execute S1425.

[0164] S1424: Determine whether there are text boxes in the same row according to the confirmation result.

[0165] S1425: Continue to select any two text boxes from each text box. When there is one different text box between the any two text boxes selected this time and the any two text boxes selected last time, return to execute the steps of S1421 - S1423 until each text box has been selected.

[0166] In this embodiment, when there are multiple text boxes in the target text area and each text box is a straight-line text box, the present application can select any two text boxes from each text box, determine the positions of these two text boxes in the Gaussian heat map, and then confirm whether these two text boxes are in the same row according to the positions of these two text boxes in the Gaussian heat map, and obtain a confirmation result. At this time, it can be determined whether each text box has been selected. If so, there is no need to confirm again whether there are text boxes in the same row, and it can be directly determined whether there are text boxes in the same row according to the current confirmation result; if not, it means that there are unselected text boxes. At this time, any two text boxes can be selected from each text box, and at least one of the any two text boxes selected this time is different from the any two text boxes selected last time. After selecting any two text boxes, it can be continued to determine whether these two text boxes are in the same row until all text boxes have been selected.

[0167] In one embodiment, according to the positions of the any two text boxes in the Gaussian heat map in S1422, confirming whether the any two text boxes are in the same row may include:

[0168] S4221: According to the positions of the any two text boxes in the Gaussian heat map, determine the first inclination angle of each text box in the Gaussian heat map, and the second inclination angle of the midpoint line obtained by connecting the midpoints of each text box in the Gaussian heat map.

[0169] S4222: According to the first inclination angle and the second inclination angle, preliminarily determine whether the any two text boxes are in the same row; if so, execute S4223.

[0170] S4223: According to the positions of the any two text boxes in the Gaussian heat map, determine the upper edge line and the lower edge line of one of the text boxes in the Gaussian heat map, and the perpendicular line between the upper edge line and the lower edge line of the midpoint of the other text box.

[0171] S4224: According to the upper edge line, the lower edge line, and the perpendicular line between the upper edge line and the lower edge line, finally determine whether the any two text boxes are in the same row.

[0172] In this embodiment, when confirming whether any two text boxes in the target text area are on the same line, the present application can first determine the first inclination angle of these two text boxes in the Gaussian heat map respectively, and the second inclination angle of the midpoint line obtained by connecting the midpoints of the two text boxes in the Gaussian heat map. Then, based on the first inclination angle and the second inclination angle, it is initially judged whether these two text boxes are on the same line. If the initial judgment result is that they are on the same line, then according to the positions of these two text boxes in the Gaussian heat map, the upper edge line and the lower edge line of one of the text boxes in the Gaussian heat map can be determined, and the perpendicular line from the midpoint of the other text box between the upper edge line and the lower edge line. Then, based on the upper edge line, the lower edge line, and the perpendicular line between the upper edge line and the lower edge line, it is finally judged whether these two text boxes are on the same line.

[0173] In a specific implementation manner, reference can be made to Figure 4 , Figure 4 which is a schematic structural diagram for initially confirming whether two text boxes are on the same line provided by an embodiment of the present application; Figure 4 In [reference figure], taking the upper left corner of the image as the origin, with the right direction as the positive X direction and the downward direction as the positive Y direction to define a coordinate system, the rotation angles of the upper and lower sides of the two text boxes in the image are calculated, and the range is (0, 180) degrees. Then, the present application can determine the rotation angle of the text box according to the average value of the rotation angles of the upper and lower sides. As shown in Figure 4 , L1 and L2 are the inclination angle lines of the two text boxes respectively, and L3 is the midpoint line obtained by connecting the midpoints of the two text boxes. The difference between the second inclination angle corresponding to this midpoint line and the first inclination angles of the two text boxes respectively is calculated, and the angle value with the smaller difference is taken for judgment. If this angle value reaches 5 degrees and reaches 1.2 times the difference between the first inclination angles of the two text boxes, it is considered that the two text boxes must not be on the same line, otherwise, the next step is carried out.

[0174] Furthermore, as shown in Figure 5 , Figure 5 which is a schematic structural diagram for finally confirming whether two text boxes are on the same line provided by an embodiment of the present application; after initially judging that the current two text boxes are on the same line, the present application can further judge whether the two text boxes are on the same line in another way, which can not only avoid the situation of incorrect initial judgment, but also merge the text boxes on the same line to improve the text recognition efficiency. As shown in Figure 5 , the present application can take the upper edge line L1: y = k1x + b1 and the lower edge line L2: y = k2x + b2 of one of the text boxes, and then calculate the perpendicular line L3 from the center point (x0, y0) of the other text box to the upper edge L1, and the intersection coordinates of L3 and L1 are briefly recorded as (x1, y1), and then calculate the intersection coordinates of L3 and the lower edge line L2 Briefly denoted as (x2, y2), finally determine whether (x2 - x0)(x1 - x0) + (y2 - y0)(y1 - y0) is greater than 0. If it is less than 0, it means that the center point of the left text box is between the upper and lower edge lines of the right text box. If it is greater than 0, it means that the center point of the left text box is not between the upper and lower edge lines of the right text box, so as to determine whether the two text boxes are on the same line.

[0175] In one embodiment, in S143, merging the text boxes on the same line to obtain the merged text box may include:

[0176] S1431: Mark the text boxes on the same line and the text boxes not on the same line in the Gaussian heat map respectively to obtain a connected graph composed of each marked text box.

[0177] S1432: Traverse the connected graph through DFS, and determine the connected components in the connected graph according to the traversal result. The connected components are the merged text boxes.

[0178] In this embodiment, after determining the text boxes on the same line in the target text area, the text boxes on the same line can be merged. During the merging process, the text boxes on the same line and the text boxes not on the same line in the Gaussian heat map can be marked respectively to obtain a connected graph composed of the marked text boxes. Then, the present application can traverse the connected graph through DFS and determine the connected components in the connected graph according to the traversal result. The connected components are the merged text boxes on the same line. By merging the text boxes on the same line, the text recognition efficiency and recognition accuracy can be significantly improved.

[0179] In one embodiment, before merging the text boxes on the same line in S143, it may further include:

[0180] S430: Determine the aspect ratio of each text box on the same line.

[0181] S431: Compare the aspect ratio of each text box with a preset aspect ratio interval. If the aspect ratio of at least one text box is not within the aspect ratio interval, correct the inclination angle of the at least one text box.

[0182] In this embodiment, after determining whether two text boxes are on the same line, it is necessary to merge the text on the same line. At this time, some other problems may be encountered. For example, for a text box composed of one or two characters, since the width and height of such a text box are not much different and it completely encloses the text, the angle often appears inclined, such as Figure 5As shown in the left text box, although this text box is on the same line as the right text box, this text box has an obvious angular tilt, which is not conducive to subsequent text recognition.

[0183] Therefore, before merging the text boxes on the same line in this application, the aspect ratios of the individual text boxes on the same line can be determined, and the aspect ratio of each text box can be compared with a preset aspect ratio range. If the aspect ratio of at least one text box is not within this aspect ratio range, the tilt angle of at least one text box is corrected.

[0184] For example, this application can first judge the aspect ratios of the individual text boxes on the same line. If there is at least one text box whose aspect ratio is less than 0.5 and greater than 2, then it can be judged that the tilt angle of this text box has an error. At this time, the tilt angle of this text box can be corrected.

[0185] In one embodiment, correcting the tilt angle of the at least one text box in S431 may include:

[0186] S4311: Select the text box with the longest box length in the Gaussian heat map as the reference text box.

[0187] S4312: Correct the tilt angle of the at least one text box according to the tilt angle of the reference text box in the Gaussian heat map.

[0188] In this embodiment, when it is detected that there is a text box with an aspect ratio not within the aspect ratio range among the text boxes on the same line, that is, when there is a text box with an incorrect tilt angle among the text boxes on the same line, the tilt angle of this text box can be corrected. During the correction process, the text box with the longest box length in the Gaussian heat map can be selected as the reference text box first, and the tilt angle of this text box is corrected according to the tilt angle of this reference text box in the Gaussian heat map.

[0189] Furthermore, when selecting the reference text box in the Gaussian heat map in this application, the text box closest to this text box can also be selected as the reference text box, or other text boxes on the same line as this text box can be selected as the reference text box. It can be specifically selected according to the actual situation and is not limited here.

[0190] In one embodiment, obtaining the Gaussian heat map corresponding to the text image in S110 may include:

[0191] Input the text image into a pre-configured deep learning detection model to obtain the Gaussian heat map corresponding to the text image output by the deep learning detection model.

[0192] Among them, the deep learning detection model is trained with multiple groups of training text images and augmented images obtained by augmenting the multiple groups of training text images as training samples, and the true Gaussian heatmaps corresponding to each group of training text images and augmented images as sample labels.

[0193] In this embodiment, when obtaining the Gaussian heatmap corresponding to the text image, a pre-configured deep learning detection model can be selected and used to obtain the Gaussian heatmap.

[0194] Specifically, the deep learning detection model in this application is configured to use multiple groups of training text images and augmented images obtained by augmenting each group of training images as training data, and the true Gaussian heatmaps corresponding to each group of training text images and augmented images as training samples for training. In the data augmentation process, this application can perform 360-degree rotation operations on multiple groups of collected training text images respectively, and randomly perform pixel value inversion operations on the training text images, so that when the initial deep learning detection model is trained using training samples, it can learn the text information contained in different training samples, thereby obtaining more accurate prediction results.

[0195] Next, the text processing device provided by the embodiments of this application will be described. The text processing device described below can be correspondingly referred to the text processing method described above.

[0196] In one embodiment, as Figure 6 shown, Figure 6 is a schematic structural diagram of a text processing device provided by an embodiment of this application; this application also provides a text processing device, which may include an image acquisition module 210, a text box determination module 220, a first processing module 230, and a second processing module 240, specifically including the following:

[0197] The image acquisition module 210 is used to acquire the text image to be processed and the Gaussian heatmap corresponding to the text image, and the Gaussian heatmap contains the target text area.

[0198] The text box determination module 220 is used to determine the text box and the text box type in the target text area, and the text box type includes a curved text box and a straight text box.

[0199] The first processing module 230 is used to, if the text box is a curved text box, segment the text box according to the bending degree of the text box, and crop and splice the text in the text image according to the segmented text box to obtain the target text.

[0200] The second processing module 240 is configured to, if the text box is a linear text box, crop the text at the same position in the text image according to the number of text boxes in the target text area and the positions of the text boxes in the Gaussian heat map, so as to obtain the target text.

[0201] In the above embodiments, before performing OCR recognition on the text image, the text image and the corresponding Gaussian heat map can be obtained, so that the background irrelevant to the text in the text image can be filtered out, thus facilitating the processing of the text more conveniently; after obtaining the Gaussian heat map, since the Gaussian heat map contains the target text area, therefore, the present application can only determine the text boxes and the text box types in the target text area, without paying attention to other areas in the Gaussian heat map, which can not only correct accurately but also improve the processing efficiency; after determining the text box types corresponding to the text boxes in the target text area, the present application can also make targeted adjustments for different text box types. For example, when the text box is a curved text box, the text box can be segmented according to the curvature of the text box, and the text in the text image can be cropped and spliced according to the segmented text boxes. The obtained target text not only contains the original text information but also simplifies the text recognition difficulty, thereby effectively improving the text recognition accuracy; when the text box is a linear text box, the present application can crop the text at the same position in the text image according to the number of text boxes in the target text area and the positions of each text box in the Gaussian heat map. For example, when there is one text box, the text at the same position as the text box in the text image can be directly cropped, and when there are multiple text boxes, it can be determined whether there are text boxes in the same row. If so, the text boxes in the same row are merged, and the text in the text image is cropped by using the merged text box. This can not only improve the text recognition efficiency but also solve the problem that large and small characters are in the same row in the text box, thereby effectively improving the text recognition accuracy.

[0202] In one embodiment, the present application also provides a storage medium, in which computer-readable instructions are stored. When the computer-readable instructions are executed by one or more processors, the one or more processors are caused to execute the steps of the text processing method according to any one of the above embodiments.

[0203] In one embodiment, the present application also provides a computer device, including: one or more processors, and a memory.

[0204] The memory stores computer-readable instructions. When the computer-readable instructions are executed by the one or more processors, the steps of the text processing method according to any one of the above embodiments are executed.

[0205] Schematically, asFigure 7 As shown Figure 7 is a schematic internal structure diagram of a computer device provided by an embodiment of the present application. The computer device 300 can be provided as a server. Referring to Figure 7 , the computer device 300 includes a processing component 302, which further includes one or more processors, and memory resources represented by a memory 301 for storing instructions executable by the processing component 302, such as application programs. The application programs stored in the memory 301 may include one or more modules each corresponding to a set of instructions. In addition, the processing component 302 is configured to execute instructions to perform the text processing method of any of the above embodiments.

[0206] The computer device 300 may further include a power supply component 303 configured to perform power management of the computer device 300, a wired or wireless network interface 304 configured to connect the computer device 300 to a network, and an input / output (I / O) interface 305. The computer device 300 may operate based on an operating system stored in the memory 301, such as Windows Server TM, Mac OS XTM, Unix TM, Linux TM, Free BSDTM or the like.

[0207] Those skilled in the art can understand that Figure 7 the structure shown in

[0208] is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. A specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0209] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0210] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but rather will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A text processing method, characterized in that, The method includes: Obtaining a text image to be processed and a Gaussian heat map corresponding to the text image, where the Gaussian heat map contains a target text area; Determining a text box and a text box type in the target text area, where the text box type includes a curved text box and a straight text box; the text box in the target text area includes at least one minimum bounding rectangle; If the text box is a curved text box, segmenting the text box according to the bending degree of the text box, and cropping and splicing the text in the text image according to the segmented text box to obtain a target text; If the text box is a straight text box, cropping the text at the same position in the text image according to the number of text boxes in the target text area and the positions of the text boxes in the Gaussian heat map to obtain a target text; The determining the text box type of the text box in the target text area includes: Judging whether the number of minimum bounding rectangles included in the text box in the target text area exceeds a preset value; If not, determining that the text box is a straight text box; If it exceeds, connecting the midpoints of the two adjacent minimum bounding rectangles with the earliest arrangement order among the midpoints of each minimum bounding rectangle in the text box to obtain a reference line; According to the arrangement order, connecting the midpoints of the minimum bounding rectangles after the two adjacent minimum bounding rectangles to the midpoint of the minimum bounding rectangle with the later arrangement order among the two adjacent minimum bounding rectangles in turn to obtain at least one measurement line; Judging whether the angle between the reference line and each measurement line is greater than a preset angle value; If at least one angle is greater than the preset angle value, determining that the text box is a curved text box; If all the angles are not greater than the preset angle value, determining that the text box is a straight text box; The segmenting the text box according to the bending degree of the text box to obtain segmented text boxes includes: Obtaining the measurement line corresponding to the first judgment that the angle is greater than the preset angle value; Taking the minimum bounding rectangle corresponding to the end point with the later arrangement order among the two end points of the measurement line as the first minimum bounding rectangle, and taking the minimum bounding rectangle corresponding to the end point with the later arrangement order among the two end points of the previous measurement line of the measurement line as the second minimum bounding rectangle; Segmenting the text box according to the positional relationship between the first minimum bounding rectangle and the second minimum bounding rectangle to obtain a text box composed of minimum bounding rectangles with the earliest arrangement order and a text box composed of minimum bounding rectangles with the later arrangement order; Judging the number of minimum bounding rectangles in the text box composed of the minimum bounding rectangles with the later arrangement order; If the number of the minimum bounding rectangles is less than two, ending the segmentation operation; If the number of the minimum bounding rectangles is at least three, use the text box formed by the minimum bounding rectangles with the later arrangement order as the new text box, and return to execute the step of obtaining the reference line by connecting the midpoints of the two adjacent minimum bounding rectangles with the earliest arrangement order according to the arrangement order of the midpoint positions of the minimum bounding rectangles in the text box until the splitting operation ends, so as to obtain the split text box.

2. The text processing method according to claim 1, characterized in that The determining the text box in the target text area includes: Performing at least one binarization operation on the Gaussian heat map according to a preset binarization threshold to obtain a binarized image. When performing more than two binarization operations on the Gaussian heat map, the binarization thresholds used for each binarization operation are different; Determining the minimum bounding rectangle corresponding to each character contour located in the target text area in the binarized image, and dividing each minimum bounding rectangle according to the text corresponding to it in the text image to obtain a text box.

3. The text processing method according to claim 1, wherein The splitting the text box according to the positional relationship between the first minimum bounding rectangle and the second minimum bounding rectangle includes: Connecting the midpoints of the first minimum bounding rectangle and the second minimum bounding rectangle to obtain a rectangular connection line; Taking the midpoint position of the rectangular connection line as the origin, drawing a splitting line parallel to the right rectangular side of the second minimum bounding rectangle, and splitting the text box according to the splitting line.

4. The text processing method according to claim 1, characterized in that, The cropping and splicing the text in the text image according to the split text box to obtain the target text includes: Dilating the split text box according to a preset dilation multiple to obtain a dilated text box; Cropping the text in the corresponding area of the text image according to the dilated text box to obtain a plurality of initial texts; Splicing the initial texts according to the arrangement order of each initial text in the text image to obtain the target text.

5. The text processing method according to claim 1, characterized in that The cropping the text at the same position in the text image according to the number of text boxes in the target text area and the positions of the text boxes in the Gaussian heat map to obtain the target text includes: When there is one text box, cropping the text at the same position in the text image according to the position of the text box in the Gaussian heat map to obtain the target text; When there are multiple text boxes, determining whether there are text boxes in the same row according to the positions of the text boxes in the Gaussian heat map; If so, merging the text boxes in the same row, and cropping the text in the text image according to the merged text box to obtain the target text; If not, cropping the text at the same position in the text image according to the position of each text box in the Gaussian heat map to obtain the target text.

6. The text processing method according to claim 5, wherein The determining whether there are text boxes in the same row according to the positions of the text boxes in the Gaussian heat map includes: Selecting any two text boxes from the text boxes, and determining the positions of the selected any two text boxes in the Gaussian heat map; Based on the positions of any two text boxes in the Gaussian heat map, confirm whether the any two text boxes are in the same row to obtain a confirmation result; Determine whether each text box has been selected; If so, determine whether there are text boxes in the same row according to the confirmation result; If not, continue to select any two text boxes from each text box. When there is a different text box between the any two text boxes selected this time and the any two text boxes selected the previous time, return to execute the step of determining the positions of the selected any two text boxes in the Gaussian heat map until each text box has been selected; 7. The text processing method according to claim 6, wherein The confirming whether the any two text boxes are in the same row according to the positions of the any two text boxes in the Gaussian heat map includes: According to the positions of the any two text boxes in the Gaussian heat map, determine the first inclination angle of each text box in the Gaussian heat map, and the second inclination angle of the midpoint line obtained by connecting the midpoints of each text box in the Gaussian heat map; Based on the first inclination angle and the second inclination angle, preliminarily determine whether the any two text boxes are in the same row; If so, according to the positions of the any two text boxes in the Gaussian heat map, determine the upper edge line and the lower edge line of one text box in the Gaussian heat map, and the perpendicular line between the upper edge line and the lower edge line of the midpoint of the other text box; Based on the upper edge line, the lower edge line and the perpendicular line between the upper edge line and the lower edge line, finally determine whether the any two text boxes are in the same row; 8. The text processing method according to claim 5, characterized in that The merging the text boxes in the same row to obtain the merged text box includes: Mark the text boxes in the same row and the text boxes not in the same row in the Gaussian heat map respectively to obtain a connected graph composed of each marked text box; Traverse the connected graph through DFS, and determine the connected components in the connected graph according to the traversal result. The connected components are the merged text boxes; 9. The text processing method according to claim 5, characterized in that Before the merging the text boxes in the same row, it further includes: Determine the aspect ratios of the text boxes in the same row; Compare the aspect ratio of each text box with a preset aspect ratio range. If the aspect ratio of at least one text box is not within the aspect ratio range, correct the inclination angle of the at least one text box; 10. The text processing method according to claim 9, wherein The correcting the inclination angle of the at least one text box includes: Select the text box with the longest box length in the Gaussian heat map as the reference text box; Correct the inclination angle of the at least one text box according to the inclination angle of the reference text box in the Gaussian heat map; 11. The text processing method according to any one of claims 1-10, characterized in that, The obtaining the Gaussian heat map corresponding to the text image includes: Input the text image into a pre-configured deep learning detection model to obtain the Gaussian heat map corresponding to the text image output by the deep learning detection model; Among them, the deep learning detection model is trained with multiple groups of training text images and augmented images obtained by augmenting the multiple groups of training text images as training samples, and the corresponding true Gaussian heatmaps of each group of training text images and augmented images as sample labels.

12. A text processing device, characterized in that, It includes: An image acquisition module, configured to acquire a text image to be processed and a Gaussian heatmap corresponding to the text image, where the Gaussian heatmap contains a target text area; A text box determination module, configured to determine a text box and a text box type in the target text area, where the text box type includes a curved text box and a straight text box; the text box in the target text area includes at least one minimum bounding rectangle; A first processing module, configured to, if the text box is a curved text box, segment the text box according to the bending degree of the text box, and crop and splice the text in the text image according to the segmented text box to obtain a target text; A second processing module, configured to, if the text box is a straight text box, crop the text at the same position in the text image according to the number of text boxes in the target text area and the position of the text box in the Gaussian heatmap to obtain a target text; The text box determination module includes: Judging whether the number of minimum bounding rectangles included in the text box in the target text area exceeds a preset value; If not, determining that the text box is a straight text box; If it exceeds, connect the midpoints of the two adjacent minimum bounding rectangles with the earliest arrangement order among the midpoints of each minimum bounding rectangle in the text box to obtain a reference line; According to the arrangement order, connect the midpoints of the minimum bounding rectangles after the two adjacent minimum bounding rectangles to the midpoint of the minimum bounding rectangle with the later arrangement order among the two adjacent minimum bounding rectangles in turn to obtain at least one measurement line; Judging whether the included angle between the reference line and each measurement line is greater than a preset angle value; If at least one included angle is greater than the preset angle value, determining that the text box is a curved text type; If all included angles are not greater than the preset angle value, determining that the text box is a straight text box; In the first processing module, segmenting the text box according to the bending degree of the text box to obtain the segmented text box includes: Obtaining the measurement line corresponding to the first judgment that the included angle is greater than the preset angle value; Regarding the minimum bounding rectangle corresponding to the end point with the later arrangement order among the two end points of the measurement line as the first minimum bounding rectangle, and regarding the minimum bounding rectangle corresponding to the end point with the later arrangement order among the two end points of the previous measurement line of the measurement line as the second minimum bounding rectangle; Segmenting the text box according to the positional relationship between the first minimum bounding rectangle and the second minimum bounding rectangle to obtain a text box composed of minimum bounding rectangles with the earliest arrangement order and a text box composed of minimum bounding rectangles with the later arrangement order; Determine the number of minimum bounding rectangles in the text box composed of the minimum bounding rectangles with a later arrangement order; If the number of the minimum bounding rectangles is less than two, end the splitting operation; If the number of the minimum bounding rectangles is at least three, use the text box composed of the minimum bounding rectangles with a later arrangement order as a new text box, and return to execute the step of obtaining a reference line by connecting the midpoints of the two adjacent minimum bounding rectangles with the earliest arrangement order according to the arrangement order of the midpoint positions of each minimum bounding rectangle in the text box, until the splitting operation ends, and obtain the split text box.

13. A storage medium, characterized in that: The storage medium stores computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to execute the steps of the text processing method according to any one of claims 1 to 11.

14. A computer device, characterized in that, Including: One or more processors, and a memory; The memory stores computer-readable instructions, which, when executed by the one or more processors, execute the steps of the text processing method according to any one of claims 1 to 11.

Images