A text image processing method and device and a storage medium

Abstract

The application discloses a text image processing method and device and a storage medium, and relates to the technical field of text detection and recognition, and comprises the following steps: acquiring a text image; detecting the text image by using a text detection model to determine a text detection result, wherein the text detection result comprises a plurality of text detection boxes of the text image; determining a principal axis included angle of each text detection box by using a post-processing method based on a principal component analysis algorithm, and the principal axis included angle is used for indicating an included angle between the text detection box and a horizontal direction; when the principal axis included angles of at least two text detection boxes in each text line satisfy a preset condition, performing merging processing on the at least two text detection boxes; determining target detection boxes according to the merged text detection boxes and unmerged text detection boxes; and determining a text recognition result of each target detection box according to each target detection box and a text recognition model, wherein the text recognition result comprises the content and position of each character of the target detection box. The application can improve the recognition efficiency of the text image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of text detection and recognition, and in particular to a text image processing method and device and storage medium. BACKGROUND

[0002] With the rapid development of mobile internet technology, more and more new application scenarios need to use the text information in images. Compared with other content in images / videos, text often contains stronger semantic information, has stronger logicality and more general expression.

[0003] In order to digitize image materials in license, PPT, office document and other scenarios for later use and processing, the simplest method is to use a dedicated person to collect and enter, but this method will consume a large amount of human resources.

[0004] In the prior art, an optical character recognition (OCR) technology can be used to detect and recognize the image of text materials to obtain a recognition result, that is, to extract and convert the characters on an electronic document into computer-readable text data. However, text detection and recognition have high requirements for the quality of collected images. If the brightness of the image is not uniform or the text in the image is inclined when the image is collected, the single word positioning error will be large, and the detection frame will be messy, thereby reducing the recognition efficiency. SUMMARY

[0005] The present application provides a text image processing method, device and storage medium, which can merge the text detection frame in the text image and improve the recognition efficiency of the text image.

[0006] To achieve the above purpose, the present application adopts the following technical solutions:

[0007] In a first aspect, the present application provides a text image processing method, which comprises:

[0008] obtaining a text image to be processed;

[0009] detecting the text image by using a preset text detection model to determine a text detection result, wherein the text detection result includes a plurality of text detection frames in the text image;

[0010] using a post-processing method based on a principal component analysis (PCA) algorithm to determine a principal axis included angle of each text detection frame in the plurality of text detection frames, wherein the principal axis included angle is used to indicate the included angle between the text detection frame and the horizontal direction;

[0011] When the included angles of the principal axes corresponding to at least two text detection boxes in each text line of the text image satisfy a preset condition, the at least two text detection boxes are subjected to a merging process.

[0012] The target detection boxes are determined according to the merged text detection boxes and the text detection boxes that are not subjected to the merging process.

[0013] Text recognition results of each target detection box are determined according to each target detection box and a preset text recognition model, and the text recognition results include the content and position of each character in the target detection box.

[0014] In a possible implementation, the method for post-processing based on the principal component analysis algorithm, the included angle of the principal axis of each text detection box in the plurality of text detection boxes is determined, including:

[0015] The area of each text detection box and the central moment corresponding to each text detection box are determined.

[0016] The covariance matrix corresponding to each text detection box is determined according to the area and the central moment corresponding to each text detection box.

[0017] The singular value decomposition is performed on the covariance matrix to determine the left singular matrix corresponding to each text detection box.

[0018] The included angle of the principal axis of each text detection box is determined according to the first column vector in the left singular matrix corresponding to each text detection box.

[0019] In a possible implementation, the method for processing a text image further includes:

[0020] The target distance between any two text detection boxes in each text line is determined, and the target distance is the shortest distance between the any two text detection boxes.

[0021] When the included angles of the principal axes corresponding to at least two text detection boxes in each text line of the text image satisfy a preset condition, the at least two text detection boxes are subjected to a merging process, including:

[0022] When the difference between the included angles of the principal axes corresponding to any two text detection boxes in the at least two text detection boxes of each text line is less than a first preset value, and the target distance between the any two text detection boxes is less than a second preset value, the at least two text detection boxes are subjected to a merging process.

[0023] In a possible implementation, the method for processing a text image further includes:

[0024] The average included angle corresponding to any two text detection boxes is determined according to the included angles of the principal axes of the any two text detection boxes in each text line.

[0025] determining a distance included angle between a direction of the target distance between any two text detection boxes in each text line and a horizontal direction;

[0026] when the difference of the included angles of the major axes corresponding to any two text detection boxes in the at least two text detection boxes of each text line is less than a first preset value, and the target distance between any two text detection boxes is less than a second preset value, performing merging processing on the at least two text detection boxes, including:

[0027] when the difference of the included angles of the major axes corresponding to any two text detection boxes in the at least two text detection boxes of each text line is less than a first preset value, and the target distance between any two text detection boxes is less than a second preset value, and the difference of the average included angles corresponding to any two text detection boxes and the distance included angle is less than a third preset value, performing merging processing on the at least two text detection boxes.

[0028] In a possible implementation, the above-mentioned merging processing on the at least two text detection boxes includes:

[0029] merging the at least two text detection boxes according to the left and right boundaries of the at least two text detection boxes and the maximum value of the heights of the text detection boxes, to obtain a merged text detection box;

[0030] deleting the at least two text detection boxes contained in the merged text detection box.

[0031] In a possible implementation, the above-mentioned determining, according to each target detection box and a preset text recognition model, of a text recognition result of each target detection box includes:

[0032] inputting each target detection box into the text recognition model after perspective transformation, and identifying the characters in the target detection box by using the text recognition model to obtain the text recognition result.

[0033] In a possible implementation, the above-mentioned processing method of a text image further includes:

[0034] determining a width of a first character of the target detection box, the width being used to indicate a number of pixels occupied by the first character in a horizontal direction;

[0035] offsetting a single-character box corresponding to each character in the target detection box by a unit of pixels to the left by the width;

[0036] supplementing a single-character detection box for a last character of the target detection box;

[0037] deleting a first single-character detection box of the target detection box.

[0038] In a second aspect, the present application provides a text image processing device, comprising:

[0039] an acquisition module configured to acquire a text image to be processed;

[0040] a detection module configured to detect the text image by using a preset text detection model to determine a text detection result, wherein the text detection result comprises a plurality of text detection boxes in the text image;

[0041] a determination module configured to determine a principal axis angle of each text detection box in the plurality of text detection boxes by using a post-processing method based on a PCA algorithm, wherein the principal axis angle is used to indicate an angle between the text detection box and a horizontal direction;

[0042] a merging module configured to perform merging processing on at least two text detection boxes in each text line of the text image when the principal axis angles corresponding to the at least two text detection boxes satisfy a preset condition;

[0043] The determination module is further configured to determine a target detection box according to the merged text detection boxes and the text detection boxes that are not merged.

[0044] an identification module configured to determine a text recognition result of each target detection box according to each target detection box and a preset text recognition model, wherein the text recognition result comprises content and a position of each character in the target detection box.

[0045] In a possible implementation, the determination module is specifically configured to:

[0046] determine an area of each text detection box and a central moment corresponding to each text detection box;

[0047] determine a covariance matrix corresponding to each text detection box according to the area and the central moment corresponding to each text detection box;

[0048] perform singular value decomposition on the covariance matrix to determine a left singular matrix corresponding to each text detection box;

[0049] determine the principal axis angle of each text detection box according to a first column vector in the left singular matrix corresponding to each text detection box.

[0050] In a possible implementation, the determination module is further configured to determine a target distance between any two text detection boxes in each text line, wherein the target distance is a shortest distance between the any two text detection boxes.

[0051] The merging module is specifically configured to:

[0052] When the difference between the principal axis included angles of any two of the at least two text detection boxes of each text line is less than a first preset value, and the target distance between any two of the at least two text detection boxes is less than a second preset value, the at least two text detection boxes are merged.

[0053] In a possible implementation, the determining module is further configured to: determine an average included angle corresponding to any two of the text detection boxes in each text line according to the principal axis included angles of the any two of the text detection boxes; and determine a distance included angle between a direction in which the target distance between any two of the text detection boxes in each text line is located and a horizontal direction.

[0054] The merging module is specifically configured to:

[0055] When the difference between the principal axis included angles of any two of the at least two text detection boxes of each text line is less than a first preset value, and the target distance between any two of the at least two text detection boxes is less than a second preset value, and the difference between the average included angle corresponding to any two of the text detection boxes and the distance included angle is less than a third preset value, the at least two text detection boxes are merged.

[0056] In a possible implementation, the merging module is specifically configured to:

[0057] The at least two text detection boxes are merged according to the left and right boundaries of the at least two text detection boxes and a maximum value of the heights of the text detection boxes, to obtain a merged text detection box.

[0058] The at least two text detection boxes contained in the merged text detection box are deleted.

[0059] In a possible implementation, the identifying module is specifically configured to:

[0060] Each of the target detection boxes is input into the text recognition model after perspective transformation, and the text recognition model is used to recognize the characters in the target detection boxes, to obtain the text recognition result.

[0061] In a possible implementation, the text image processing apparatus further includes a correction module, which is configured to:

[0062] A width of a first character of the target detection box is determined, and the width is used to indicate a number of pixels occupied by the first character in a horizontal direction.

[0063] Each character in the target detection box is offset by a unit of pixels to the left by the width.

[0064] A single-character detection box is supplemented for a last character of the target detection box.

[0065] delete the first single word bounding box of the target bounding box.

[0066] In a third aspect, the present application provides a text image processing device, comprising a processor and a memory. The memory stores at least one instruction or at least one program, which is loaded and executed by the processor to implement the text image processing method according to the first aspect and any possible implementation thereof.

[0067] In a fourth aspect, the present application provides a computer-readable storage medium, which stores at least one instruction or at least one program, which is loaded and executed by a processor to implement the text image processing method according to the first aspect and any possible implementation thereof.

[0068] In a fifth aspect, the present application provides a computer program product, which comprises computer instructions, when the computer instructions run on a text image processing device, cause the text image processing device to perform the text image processing method according to the first aspect and any possible implementation thereof.

[0069] In the text image processing method provided by the embodiments of the present application, when detecting the obtained text image to be processed, first, a preset text detection model is used to detect the text image to determine a text detection result, wherein the text detection result includes a plurality of text detection boxes in the text image; second, a post-processing method based on a principal component analysis algorithm is used to determine a principal axis angle of each text detection box in the plurality of text detection boxes, the principal axis angle being used to indicate an angle between the text detection box and the horizontal direction; third, when the principal axis angles corresponding to at least two text detection boxes in each text line of the text image satisfy a preset condition, the at least two text detection boxes are merged for processing, and a target detection box is determined according to the merged text detection box and the text detection box that is not merged; and finally, according to each target detection box and a preset text recognition model, a text recognition result of each target detection box is determined. As can be seen from the above, in the text image processing method provided by the embodiments of the present application, the principal axis angle between each text detection box in the plurality of text detection boxes and the horizontal direction is determined by using the post-processing method based on the principal component analysis algorithm, when the principal axis angles corresponding to at least two text detection boxes in each text line satisfy the preset condition, the at least two text detection boxes are merged for processing, and the merged target detection box only needs to be recognized once, compared with the recognition of each text detection box in the prior art, the recognition efficiency of the text image is improved. BRIEF DESCRIPTION OF DRAWINGS

[0070] Figure 1 FIG. 1 is a flowchart of a text image processing method according to an embodiment of the present application.

[0071] Figure 2 A schematic diagram of text detection box merging provided for an embodiment of the present application;

[0072] Figure 3 Flowchart II of a text image processing method provided for an embodiment of the present application;

[0073] Figure 4 Flowchart III of a text image processing method provided for an embodiment of the present application;

[0074] Figure 5 Structural schematic diagram I of a text image processing device provided for an embodiment of the present application;

[0075] Figure 6 Structural schematic diagram II of a text image processing device provided for an embodiment of the present application. DETAILED DESCRIPTION

[0076] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0077] Hereinafter, the terms "first" and "second" are used only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first" and "second" can explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, unless otherwise specified, the meaning of "a plurality of" is two or more. In addition, the use of "based on" or "according to" means openness and inclusiveness, because the process, step, calculation or other action "based on" or "according to" one or more stated conditions or values can be based on additional conditions or values beyond the stated values in practice.

[0078] Figure 1 Flowchart I of a text image processing method provided for an embodiment of the present application. As shown in the figure, Figure 1 the text image processing method can include the following steps S101-S106.

[0079] S101, acquiring a text image to be processed.

[0080] In an optional embodiment, the type of the text image to be processed can include: license class, PPT class, office document class, etc. The text image refers to an image whose image content contains text.

[0081] S102, a preset text detection model is used to detect the text image to determine a text detection result, the text detection result including a plurality of text detection boxes in the text image.

[0082] Optionally, in the embodiment of the present application, after obtaining the text image to be processed, the processing device of the text image can pre-process the text image and input the pre-processed text image into the text detection model to obtain a plurality of text detection boxes in the text image. The pre-processing can include but is not limited to noise reduction processing, resize processing (resize processing refers to scaling processing of the text image to change it into an image of a preset size).

[0083] It can be understood that each of the above text detection boxes includes at least one character, and each text line in the text image can include at least one text detection box.

[0084] Optionally, the above text detection model is pre-trained and stored in the processing device of the text image. In some embodiments, the training process of the text detection model can include: first, acquiring images containing text in different scenes through an image acquisition device, and adding labels to the images to construct a training data set. Then, a text detection framework DBnet based on a semantic segmentation method is constructed. The text detection framework DBnet uses MobileNetV3 as the backbone network of DBnet, and the attention idea of MobileNetV3 can make the network pay more attention to the text area in the image and more targetedly extract features. Moreover, a threshold map branch is introduced in DBnet, which adds the step of determining the binary threshold to the network, making it a trainable parameter, so that each pixel point is adaptively binarized, solving the problem of difficulty in determining a suitable binary threshold in the traditional text detection algorithm based on the semantic segmentation method, and improving the robustness of the text detection model. Moreover, by using the loss function Instance-Balanced Cross-Entropy Loss in the text detection framework DBnet, the weight of large text regions is reduced and the weight of small text regions is increased in the training process, thereby balancing the weights between large and small text regions, so that the detection effect of dense and small text regions is better. Finally, the above labeled training samples are input into the constructed text detection framework DBnet for model training to obtain a trained text detection model.

[0085] The calculation formula of the above loss function Instance-Balanced Cross-Entropy Loss is shown in formula (1):

[0086]

[0087] wherein S i represents the area of the i-th text region, S represents the total area of the text regions in the image, and W i = B i / S i

[0088] S103, adopting a post-processing method based on a principal component analysis algorithm, to determine a principal axis angle of each text detection box in the plurality of text detection boxes, the principal axis angle being used to indicate an angle between the text detection box and a horizontal direction.

[0089] Optionally, in a possible implementation, the step of adopting the post-processing method based on the principal component analysis algorithm to determine the principal axis angle of each text detection box in the plurality of text detection boxes can include the following steps a to d:

[0090] Step a, determining an area of each text detection box according to a zero-order moment corresponding to the text detection box.

[0091] In some embodiments, the processing device of the text image can determine a zero-order moment corresponding to each text detection box in the plurality of text detection boxes, and determine an area of the text detection box according to the zero-order moment.

[0092] Step b, determining a covariance matrix corresponding to each text detection box according to a second-order moment matrix corresponding to the text detection box.

[0093] In some embodiments, for each text detection box, the processing device of the text image can obtain the covariance matrix of the text detection box by dividing the second-order moment matrix by the area.

[0094] Step c, performing singular value decomposition on the covariance matrix to determine a left singular matrix corresponding to each text detection box.

[0095] Step d, determining a principal axis angle of each text detection box according to a first column vector in the left singular matrix corresponding to the text detection box.

[0096] In some embodiments, after determining the left singular matrix corresponding to each text detection box, the processing device of the text image can obtain the principal axis angle of each text detection box by taking an inverse tangent of a first column vector in the left singular matrix.

[0097] S104, when the principal axis angles corresponding to at least two text detection boxes in each text line of the text image satisfy a preset condition, performing merging processing on the at least two text detection boxes.

[0098] When the principal axis angles corresponding to any two text detection boxes in each text line do not satisfy the preset condition, the any two text detection boxes are not processed.​

[0099] Optionally, in a possible implementation, the merging of the at least two text detection boxes can include the following step A and step B.

[0100] Step A, merging the at least two text detection boxes according to the maximum value of the left and right boundaries of the at least two text detection boxes and the height of the text detection boxes to obtain a merged text detection box.

[0101] Step B, deleting the at least two text detection boxes contained in the merged text detection box.

[0102] Exemplarily, Figure 2 The schematic diagram of the merging of the text detection boxes provided by the embodiment of the present application is shown in FIG. 1. As shown in FIG. 1, the text detection box 1 and the text detection box 2 in a text line are merged to obtain a merged text detection box 3. The left and right boundaries of the text detection box 3 are obtained according to the left boundary of the text detection box 2 and the right boundary of the text detection box 1, and the height of the text detection box 3 is determined according to the maximum value of the heights of the text detection box 1 and the text detection box 2. Figure 2 As shown in FIG. 1, the maximum value of the heights of the text detection box 1 and the text detection box 2 is the height of the text detection box 1, and thus the height of the merged text detection box 3 is determined according to the height of the text detection box 1. Figure 2

[0103] After obtaining the merged text detection box 3, the text detection box 1 and the text detection box 2 are contained in the merged text detection box 3, and the text detection box 1 and the text detection box 2 are deleted, so that the merging of the text detection boxes is completed.

[0104] S105, determining a target detection box according to the merged text detection box and the text detection box which is not subjected to the merging.

[0105] After the merging of the text detection boxes, the text detection box which is not subjected to the merging and the merged text detection box are the text detection boxes which need to be recognized, and thus the target detection box can be determined according to the merged text detection box and the text detection box which is not subjected to the merging. The text detection box which is not subjected to the merging is the text detection box which does not satisfy the preset condition in step S104.

[0106] S106, determining a text recognition result of each target detection box according to each target detection box and a preset text recognition model, the text recognition result including the content and position of each character in the target detection box.

[0107] ​Optionally, the text recognition model is pre-trained and stored in the text image processing device. In some embodiments, a text recognition model CRNN based on a convolutional neural network (CNN), a BiLSTM, and a loss function (Connectionist temporal Classification, CTC) can be constructed. The backbone network of the text recognition model uses Resnet_vd, and the resnet network avoids the problems of gradient disappearance and network degradation caused by too deep network through residual addition. The BiLSTM can avoid the problems of gradient disappearance, gradient explosion, and poor long-distance dependency information in the training process of the recurrent neural network, and can better learn the sequence relationship in the learned features.

[0108] Optionally, in the embodiments of the present application, the text image processing device can input each target detection frame after perspective transformation into the text recognition model to recognize the characters in each target detection frame using the text recognition model to obtain the text recognition result.

[0109] It can be understood that the text recognition result of each target detection frame can include the content and position of each character in each target detection frame. The position of each character can be the pixel coordinates of each character in the image.

[0110] Optionally, in the embodiments of the present application, the specific process of the text image processing device recognizing the characters in each target detection frame using the text recognition model can be:

[0111] Step I, obtaining non-“blank” index markers in the target detection frame.

[0112] It can be understood that when the text recognition model is used to recognize the input target detection frame, the characters in the target detection frame can be recognized multiple times. The loss function included in the text recognition model introduces a “blank” placeholder to solve the problem of misalignment between input and output.

[0113] For example, if the target detection frame includes hello, the recognition result obtained by the text recognition model can be hhh-eee-ll-lll-ooo, where “-” represents “blank”, and then by taking “-” as a marker, the hhh-eee-ll-lll-ooo can be processed to obtain hello. If there is no “blank” placeholder, the result of the de-duplication is helo. Through the “blank” placeholder, the problem of text error in de-duplication is solved.

[0114] Step II, traversing the markers and taking the larger index marker of each character as the final index marker of the character.

[0115] Understandably, when traversing a single character, the number of pixels can be used as the index of each character. As shown in step I, each character can have multiple consecutive index markers. In this case, the index marker with the largest number of pixels can be used as the larger index marker, and this larger index marker can be used as the final index marker of the character.

[0116] Step III: Define the starting position of each word as the ending position of the previous word, and set the starting position of the first word to 0.

[0117] Step IV: Based on the start and end positions of each character, map back to the input text image to obtain the content and position of each character.

[0118] It is understandable that the position of each character can be represented by the coordinates of the four vertices of the character frame of each character.

[0119] In the text image processing method provided in this embodiment of the invention, when detecting the acquired text image to be processed, a preset text detection model is first used to detect the text image and determine the text detection result, wherein the text detection result includes multiple text detection boxes in the text image; secondly, a post-processing method based on principal component analysis algorithm is used to determine the principal axis angle of each text detection box in the multiple text detection boxes, the principal axis angle is used to indicate the angle between the text detection box and the horizontal direction; then, when the principal axis angles corresponding to at least two text detection boxes in each text line of the text image meet the preset conditions, at least two text detection boxes are merged, and a target detection box is determined based on the merged text detection box and the unmerged text detection box; finally, the text recognition result of each target detection box is determined based on each target detection box and the preset text recognition model. As can be seen from the above, in the text image processing method provided by the embodiments of the present invention, the post-processing method of principal component analysis algorithm is used to determine the angle between each text detection box and the principal axis in the horizontal direction in multiple text detection boxes. When the angle between the principal axes of at least two text detection boxes in each text line meets the preset conditions, the at least two text detection boxes are merged. The merged target detection box only needs to be recognized once. Compared with the prior art of recognizing each text detection box, the recognition efficiency of text images is improved.

[0120] Optional, combined Figure 1 ,like Figure 3 As shown, the text image processing method provided in this embodiment of the invention may further include the following step 301.

[0121] S301. Determine the target distance between any two text detection boxes in each text line. The target distance is the shortest distance between any two text detection boxes.

[0122] In some embodiments, the processing device of the text image can determine the target distance between any two text bounding boxes in each text line according to coordinate information of the any two text bounding boxes.

[0123] For example, assuming that there are two adjacent text bounding boxes without intersection in a text line, the processing device of the text image can determine the shortest distance between the two text bounding boxes according to coordinate information of the right boundary of the left text bounding box and coordinate information of the left boundary of the right text bounding box.

[0124] In this case, as shown in Figure 3 the above step S104 can specifically include the following step S302.

[0125] S302, when the difference between the included angles of the major axes corresponding to any two text bounding boxes in at least two text bounding boxes of each text line is less than a first preset value, and the target distance between the any two text bounding boxes is less than a second preset value, performing merging processing on the at least two text bounding boxes.

[0126] When there are at least two text bounding boxes in each text line, the difference between the included angles of the major axes corresponding to any two text bounding boxes in the at least two text bounding boxes is less than a first preset value, and the target distance between the any two text bounding boxes in the at least two text bounding boxes is less than a second preset value, the processing device of the text image can perform merging processing on the at least two text bounding boxes.

[0127] When the difference between the included angles of the major axes corresponding to any two text bounding boxes in each text line is greater than or equal to the first preset value, or the target distance between the any two text bounding boxes is greater than or equal to the second preset value, the processing device of the text image does not process the any two text bounding boxes.

[0128] In this way, when determining whether to perform merging processing on at least two text bounding boxes in a text line, not only the included angles of the major axes corresponding to any two text bounding boxes are referred to, but also the target distance between the any two text bounding boxes is referred to. Only in the case that the difference between the included angles of the major axes is less than the first preset value, indicating that the inclination angles of the two text bounding boxes are similar, and the target distance is less than the second preset value, indicating that the distance between the two text bounding boxes is close, the merging processing is performed on the two text bounding boxes, so that the result of the merging processing is more accurate.

[0129] Optionally, in combination with Figure 3 as shown in Figure 4 the processing method of the text image provided by the embodiments of the present application can further include the following steps 401-402.

[0130] In step S401, the average included angle of any two text bounding boxes in each text line is determined according to the principal axis included angles of the two text bounding boxes.

[0131] The processing device of the text image can take the average of the two principal axis included angles of any two text bounding boxes in each text line as the average included angle of the two text bounding boxes.

[0132] In step S402, the distance included angle between the target distance direction and the horizontal direction of any two text bounding boxes in each text line is determined.

[0133] In this case, as shown in FIG. 3, the step S302 can specifically include the following step S403. Figure 4

[0134] S403, when the difference between the principal axis included angles of any two text bounding boxes of at least two text bounding boxes in each text line is less than a first preset value, and the target distance between the two text bounding boxes is less than a second preset value, and the difference between the average included angle and the distance included angle of the two text bounding boxes is less than a third preset value, the at least two text bounding boxes are merged.

[0135] When the difference between the two principal axis included angles of any two text bounding boxes in each text line is greater than or equal to the first preset value, or the target distance between the two text bounding boxes is greater than or equal to the second preset value, or the difference between the average included angle and the distance included angle of the two text bounding boxes is greater than or equal to the third preset value, the processing device of the text image does not process the two text bounding boxes.

[0136] In this way, when determining whether to merge at least two text bounding boxes in a text line, the difference between the average included angle and the distance included angle of any two text bounding boxes is further referred to on the basis of referring to the principal axis included angles of the two text bounding boxes and the target distance between the two text bounding boxes, so that the result of the merging processing is more accurate.

[0137] Optionally, in the embodiment of the present application, after the processing device of the text image determines the text recognition result of each target bounding box in the step S106, the positioning error of each single character in each target bounding box can be corrected. In a possible embodiment, taking a target bounding box as an example, the correction process can include the following steps 1-4.

[0138] Step 1, the width of the first character of the target bounding box is determined, and the width is used to indicate the number of pixels occupied by the first character in the horizontal direction.

[0139] ​Step 2, shift each single-character box corresponding to each character in the target detection box to the left by a width of a unit of pixels.

[0140] Step 3, supplement a single-character detection box for the last character of the target detection box.

[0141] Step 4, delete the first single-character detection box of the target detection box.

[0142] The corrected target detection box is the final detection and recognition result of the text image.

[0143] Figure 5 A structure diagram of a text image processing device provided by an embodiment of the present application is shown in FIG. 1. Figure 5 As shown in FIG. 1, the text image processing device 500 can include an acquisition module 501, a detection module 502, a determination module 503, a merging module 504, and an identification module 505.

[0144] The acquisition module 501 is configured to acquire a text image to be processed.

[0145] The detection module 502 is configured to detect the text image by using a preset text detection model, and determine a text detection result, wherein the text detection result includes a plurality of text detection boxes in the text image.

[0146] The determination module 503 is configured to determine a principal axis included angle of each text detection box in the plurality of text detection boxes by using a post-processing method based on a PCA algorithm, wherein the principal axis included angle is used to indicate an included angle of the text detection box with a horizontal direction.

[0147] The merging module 504 is configured to perform merging processing on at least two text detection boxes in each text line of the text image when principal axis included angles corresponding to the at least two text detection boxes satisfy a preset condition.

[0148] The determination module 503 is further configured to determine a target detection box according to the merged text detection boxes and the text detection boxes that are not merged.

[0149] The identification module 505 is configured to determine a text recognition result of each target detection box according to each target detection box and a preset text recognition model, wherein the text recognition result includes content and a position of each character in the target detection box.

[0150] In a possible implementation, the determination module 503 is specifically configured to:

[0151] determine an area of each text detection box and a center moment corresponding to each text detection box.

[0152] determine a covariance matrix corresponding to each text detection box according to the area and the central moment corresponding to each text detection box;

[0153] perform singular value decomposition on the covariance matrix to determine a left singular matrix corresponding to each text detection box;

[0154] determine a principal axis included angle of each text detection box according to a first column vector in the left singular matrix corresponding to each text detection box.

[0155] In a possible implementation, the determining module 503 is further configured to: determine a target distance between any two text detection boxes in each text line, the target distance being a shortest distance between the any two text detection boxes;

[0156] The merging module 504 is specifically configured to:

[0157] perform merging processing on the at least two text detection boxes when the difference between the principal axis included angles corresponding to any two text detection boxes of at least two text detection boxes of each text line is less than a first preset value, and the target distance between the any two text detection boxes is less than a second preset value.

[0158] In a possible implementation, the determining module 503 is further configured to: determine an average included angle corresponding to any two text detection boxes according to the principal axis included angles of the any two text detection boxes in each text line; and determine a distance included angle between a direction in which the target distance between the any two text detection boxes is located and a horizontal direction.

[0159] The merging module 504 is specifically configured to:

[0160] perform merging processing on the at least two text detection boxes when the difference between the principal axis included angles corresponding to any two text detection boxes of at least two text detection boxes of each text line is less than a first preset value, and the target distance between the any two text detection boxes is less than a second preset value, and the difference between the average included angle corresponding to the any two text detection boxes and the distance included angle is less than a third preset value.

[0161] In a possible implementation, the merging module 504 is specifically configured to:

[0162] merge the at least two text detection boxes according to left and right boundaries of the at least two text detection boxes and a maximum value in a height of the text detection boxes to obtain a merged text detection box;

[0163] delete the at least two text detection boxes included in the merged text detection box.

[0164] In a possible implementation, the identifying module 505 is specifically configured to:

[0165] inputting each of the target detection boxes into the text recognition model after perspective transformation, recognizing the words in the target detection boxes by using the text recognition model, and obtaining the text recognition result. In a possible implementation manner, as shown in FIG. 5, the text image processing apparatus 500 can further include a correction module 506. The correction module 506 is configured to: Figure 6 determining the width of the first word of the target detection box, the width being used to indicate the number of pixels occupied by the first word in the horizontal direction;

[0166] offsetting each word in the target detection box by a single word box to the left by the width of units of pixels;

[0167] supplementing a single word detection box for the last word of the target detection box;

[0168] deleting the first single word detection box of the target detection box.

[0169] The embodiment of the present application further provides a text image processing device, which comprises a processor and a memory. The memory stores at least one instruction or at least one program, and the processor loads and executes the at least one instruction or the at least one program to implement the text image processing method in the embodiment of the present application.

[0170] The embodiment of the present application further provides a computer readable storage medium, which stores at least one instruction or at least one program, and the processor loads and executes the at least one instruction or the at least one program to implement the text image processing method in the embodiment of the present application.

[0171]

[0172] ​In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer executes the computer instructions, all or part of the processes or functions according to the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that can be accessed by a computer or include one or more data storage devices such as servers, data centers, etc. that can be integrated with the medium. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0173] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto, any change or replacement within the technical scope disclosed by the present application should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method of processing a text image, characterized by, The method comprises the following steps: acquiring a text image to be processed; detecting the text image by using a preset text detection model to determine a text detection result, wherein the text detection result comprises a plurality of text detection boxes in the text image; determining a principal axis angle of each text detection box in the plurality of text detection boxes by using a post-processing method based on a principal component analysis algorithm, wherein the principal axis angle is used to indicate an angle between the text detection box and a horizontal direction; when the principal axis angles corresponding to at least two text detection boxes in each text line of the text image satisfy a preset condition, performing a merging process on the at least two text detection boxes; determining a target detection box according to the merged text detection boxes and the text detection boxes that are not merged; determining a text recognition result of each target detection box according to each target detection box and a preset text recognition model, wherein the text recognition result comprises content and a position of each character in the target detection box; the method for processing the text image further comprises the following steps: determining an average angle corresponding to any two text detection boxes according to the principal axis angles of the any two text detection boxes in each text line; determining a distance angle between a direction of a target distance between any two text detection boxes in each text line and a horizontal direction; when the principal axis angles corresponding to at least two text detection boxes in each text line of the text image satisfy a preset condition, performing a merging process on the at least two text detection boxes, comprising: when the difference between the principal axis angles corresponding to any two text detection boxes in the at least two text detection boxes in each text line is less than a first preset value, the target distance between the any two text detection boxes is less than a second preset value, and the difference between the average angle corresponding to the any two text detection boxes and the distance angle is less than a third preset value, performing a merging process on the at least two text detection boxes; the target distance is a shortest distance between any two text detection boxes.

2. The method of claim 1, wherein, the method for determining the principal axis angle of each text detection box in the plurality of text detection boxes by using the post-processing method based on the principal component analysis algorithm, comprising: determining an area of each text detection box and a central moment corresponding to each text detection box; determining a covariance matrix corresponding to each text detection box according to the area and the central moment corresponding to each text detection box; performing singular value decomposition on the covariance matrix to determine a left singular matrix corresponding to each text detection box; determining the principal axis angle of each text detection box according to a first column vector in the left singular matrix corresponding to each text detection box.

3. The method of processing a text image according to claim 1 or 2, wherein, the method for processing the text image further comprises the following steps: determining a target distance between any two text detection boxes in each text line, wherein the target distance is a shortest distance between the any two text detection boxes; when the principal axis angles corresponding to at least two text detection boxes in each text line of the text image satisfy a preset condition, performing a merging process on the at least two text detection boxes, comprising: When the difference between the principal axis included angles of any two of the at least two text detection boxes in each text line is less than a first preset value, and the target distance between any two of the at least two text detection boxes is less than a second preset value, the at least two text detection boxes are merged.

4. The method of claim 1 or 2, wherein, The merging of the at least two text detection boxes comprises: merging the at least two text detection boxes according to the maximum of the left and right boundaries of the at least two text detection boxes and the height of the text detection box, to obtain a merged text detection box; deleting the at least two text detection boxes included in the merged text detection box.

5. The method of processing a text image according to claim 1 or 2, wherein, The determination of the text recognition result of each target detection box comprises: inputting each target detection box into the text recognition model after perspective transformation, and recognizing the characters in the target detection box by using the text recognition model to obtain the text recognition result.

6. The method of processing a text image according to claim 1 or 2, wherein, After the text recognition result of each target detection box is determined, the text image processing method further comprises: determining the width of the first character of the target detection box, the width being used to indicate the number of pixels occupied by the first character in the horizontal direction; shifting each character in the target detection box by a single character box to the left by the width of a unit of pixel; supplementing a single character detection box for the last character of the target detection box; deleting the first single character detection box of the target detection box.

7. An apparatus for processing a text image, characterized by comprising: comprises: an acquisition module configured to acquire a text image to be processed; a detection module configured to detect the text image by using a preset text detection model, and determine a text detection result, the text detection result comprising a plurality of text detection boxes in the text image; a determination module configured to determine a principal axis included angle of each text detection box in the plurality of text detection boxes by using a post-processing method based on a principal component analysis algorithm, the principal axis included angle being used to indicate an included angle between the text detection box and a horizontal direction; a merging module configured to, when the principal axis included angles of at least two text detection boxes in each text line of the text image satisfy a preset condition, merge the at least two text detection boxes; the determination module is further configured to determine a target detection box according to a merged text detection box and a text detection box that is not merged; an identification module configured to determine a text recognition result of each target detection box according to each target detection box and a preset text recognition model, the text recognition result comprising the content and position of each character in the target detection box; the apparatus is further configured to: determine an average included angle of any two text detection boxes in each text line according to the principal axis included angles of the any two text detection boxes; determine a distance included angle between a direction of a target distance between any two text detection boxes in each text line and a horizontal direction; the merging module is specifically configured to: When the difference between the principal angles of any two of the at least two text detection boxes of each text line is less than a first preset value, the target distance between any two of the text detection boxes is less than a second preset value, and the difference between the average angle and the distance angle of any two of the text detection boxes is less than a third preset value, the at least two text detection boxes are merged. The target distance is the shortest distance between any two of the text detection boxes.

8. An apparatus for processing a text image, characterized by comprising: The device comprises a processor and a memory, and the memory stores at least one instruction or at least one program, which is loaded and executed by the processor to implement the text image processing method of any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction or at least one program, which is loaded and executed by the processor to implement the text image processing method of any one of claims 1-6.

Images