Image text recognition method, device, computer equipment and computer storage medium

Abstract

The present application discloses an image text recognition method, device and computer storage medium, which relates to the field of computer technology. For images with complex scenes, the text recognition results can be structured to improve the accuracy of image text recognition. The method includes: obtaining an image to be recognized, preprocessing the image to be recognized, and obtaining a target recognition image; determining the position information of the text area in the target recognition image based on a pre-trained text area detection model; inputting the target recognition image and the position information of the text area in the target recognition image into a pre-trained text recognition model to obtain text information in the text area; and performing structured processing on the text information in the text area to obtain a text field with a mapping relationship.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an image text recognition method, device, computer equipment and computer storage medium. Background Art

[0002] With the development of science and technology, images play a great role in information dissemination. In order to better promote the image, more and more images are added with text. For example, in medical transaction platforms, medical institutions require users to upload invoice images, so as to verify the invoice based on the text content in the uploaded invoice images. Therefore, since the text in the image usually contains relatively rich information, the extraction and recognition of the text in the image is of great significance for the analysis, understanding and information retrieval of the image content.

[0003] Existing image text recognition methods generally first detect the text information box in the image, then recognize the detected text information box, and finally return the recognition result to achieve automatic recognition and save human input costs.

[0004] However, in images of actual application scenarios, there are complex and diverse image contents. The text in invoice images or many natural images is usually affected by the irregular image background content, which results in many missed detections and false detections in the existing image text recognition methods, and the recognition accuracy is low, resulting in incomplete final text recognition results and the problem that the recognized fields cannot correspond to the required fields, which seriously affects the subsequent use of the text. Summary of the invention

[0005] In view of this, the present invention provides an image text recognition method, apparatus, computer equipment and computer storage medium, the main purpose of which is to solve the current problem of low accuracy in image text recognition for complex scenes.

[0006] According to one aspect of the present invention, a method for image text recognition is provided, the method comprising:

[0007] Acquire an image to be recognized, and preprocess the image to be recognized to obtain a target recognition image;

[0008] Determining location information of a text region in the target recognition image based on a pre-trained text region detection model;

[0009] Inputting the target recognition image and the position information of the text area in the target recognition image into a pre-trained text recognition model to obtain the text information in the text area;

[0010] The text information in the text area is structured to obtain text fields with mapping relationships.

[0011] Furthermore, the text information in the text area is structured to obtain text fields with mapping relationships, specifically including:

[0012] Selecting a preset field from the text information in the text area as a key field, and obtaining position information of the text area corresponding to the key field;

[0013] Determine a fuzzy area having a mapping relationship with the key field according to the position information of the text area corresponding to the key field;

[0014] The text information identified in the fuzzy area is detected and queried to confirm the text information having a mapping relationship with the key field.

[0015] Furthermore, the determining of the fuzzy area having a mapping relationship with the key field according to the position information of the text area corresponding to the key field specifically includes:

[0016] Move the text area corresponding to the key field in the horizontal and vertical directions by a preset distance, and obtain the position information of the text area after the movement according to the position information of the text area corresponding to the key field;

[0017] Based on the position information of the moved text area, the text area after moving the preset distance is enlarged to determine a fuzzy area having a mapping relationship with the key field.

[0018] Furthermore, the detecting and querying the text information identified in the fuzzy area to confirm the text information having a mapping relationship with the key field specifically includes:

[0019] Detecting the position information of all text areas within the fuzzy area, and extracting the text information of all text areas within the fuzzy area;

[0020] The text information of each text area in the fuzzy area is traversed by using a regular matching method to confirm the text information having a mapping relationship with the key field.

[0021] Furthermore, the method of traversing the text information of each text area in the fuzzy area by using a regular matching method to confirm the text information having a mapping relationship with the key field specifically includes:

[0022] By obtaining the pattern characters applicable to the key field, a regular expression matching the key field is constructed;

[0023] According to the regular expression that matches the key field, the text information of each text area in the fuzzy area is verified to confirm the text information that has a mapping relationship with the key field.

[0024] Furthermore, before determining the position information of the text region in the target recognition image based on the pre-trained text region detection model, the method further includes:

[0025] The collected image sample data is annotated with text regions and then input into the network model for training to obtain a text region detection model;

[0026] The network model includes a multi-layer structure. The collected image sample data is annotated with text regions and then input into the network model for training to obtain a text region detection model, which specifically includes:

[0027] Extracting image region features corresponding to the image sample data through the convolutional layer of the network model;

[0028] Generate horizontal text sequence features according to image region features corresponding to image sample data through the decoding layer of the network model;

[0029] The text area in the image sample data is determined according to the horizontal text sequence features through the prediction layer of the network model, and the text area is processed to obtain candidate text lines.

[0030] Furthermore, the prediction layer of the network model includes a classification part and a regression part, and the prediction layer of the network model determines the text area in the image sample data according to the horizontal text sequence features, and processes the text area to obtain the candidate text line, specifically including:

[0031] Classifying each region in the image sample data according to the horizontal text sequence features by the classification part of the prediction layer of the network model, and determining the text region in the image sample data;

[0032] The text area in the image text data is processed by border regression through the regression part of the prediction layer of the network model to obtain candidate text lines.

[0033] According to another aspect of the present invention, there is provided an image text recognition device, the device comprising:

[0034] An acquisition unit, used for acquiring an image to be recognized, and preprocessing the image to be recognized to obtain a target recognition image;

[0035] A determination unit, configured to determine position information of a text region in the target recognition image based on a pre-trained text region detection model;

[0036] A recognition unit, used for inputting the target recognition image and the position information of the text area in the target recognition image into a pre-trained text recognition model to obtain the text information in the text area;

[0037] The processing unit is used to perform structural processing on the text information in the text area to obtain text fields with mapping relationships.

[0038] Furthermore, the processing unit includes:

[0039] A selection module, used to select a preset field from the text information in the text area as a key field, and obtain position information of the text area corresponding to the key field;

[0040] A determination module, configured to determine a fuzzy area having a mapping relationship with the key field according to position information of a text area corresponding to the key field;

[0041] The detection module is used to detect and query the text information identified in the fuzzy area, and confirm the text information having a mapping relationship with the key field.

[0042] Furthermore, the determination module includes:

[0043] An acquisition submodule, used to move the text area corresponding to the key field in the horizontal and vertical directions by a preset distance, and acquire the position information of the text area after the movement according to the position information of the text area corresponding to the key field;

[0044] The determination submodule is used to enlarge the text area after the text area moves a preset distance based on the position information of the text area after the text area moves, and determine the fuzzy area having a mapping relationship with the key field.

[0045] Furthermore, the detection module includes:

[0046] An extraction submodule, used for detecting the position information of all text areas within the fuzzy area, and extracting the text information of all text areas within the fuzzy area;

[0047] The confirmation submodule is used to traverse the text information of each text area in the fuzzy area by using regular matching, and confirm the text information having a mapping relationship with the key field.

[0048] Furthermore, the confirmation submodule is specifically used to construct a regular expression matching the key field by acquiring a pattern character applicable to the key field;

[0049] The confirmation submodule is further specifically used to verify the text information of each text area in the fuzzy area according to the regular expression matching the key field, and confirm the text information having a mapping relationship with the key field.

[0050] Furthermore, the device also includes:

[0051] A training unit is used to annotate the collected image sample data with text regions and input them into a network model for training before determining the position information of the text region in the target recognition image based on the pre-trained text region detection model to obtain a text region detection model;

[0052] The network model includes a multi-layer structure, and the training unit includes:

[0053] An extraction module, used to extract image region features corresponding to image sample data through the convolutional layer of the network model;

[0054] A generation module, used to generate horizontal text sequence features according to image region features corresponding to image sample data through a decoding layer of the network model;

[0055] A prediction module is used to determine the text area in the image sample data according to the horizontal text sequence features through the prediction layer of the network model, and process the text area to obtain candidate text lines.

[0056] Furthermore, the prediction layer of the network model includes a classification part and a regression part, and the prediction module includes:

[0057] A classification submodule, configured to classify each region in the image sample data according to the horizontal text sequence features through the classification part of the prediction layer of the network model, and determine the text region in the image sample data;

[0058] The processing submodule is used to perform border regression processing on the text area in the image text data through the regression part of the prediction layer of the network model to obtain candidate text lines.

[0059] According to another aspect of the present invention, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the image text recognition method when executing the computer program.

[0060] According to another aspect of the present invention, a computer storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the image text recognition method are implemented.

[0061] By means of the above technical scheme, the present invention provides an image text recognition method and device, which obtains an image to be recognized, pre-processes the image to be recognized, obtains a target recognition image, and determines the position information of the text area in the target recognition image based on a pre-trained text area detection model. After the target recognition image and the position information of the text area in the target recognition image are input into the pre-trained text recognition model, the text information in the text area is recognized, and the text information in the text area is structured into a text field with a mapping relationship. Compared with the image text recognition method in the prior art, the present application can effectively remove the interference information in the image and accurately retain the text information of the image by performing structured processing on the recognized text information, so that the image in the text area detection and text information recognition is not interfered by the background, thereby outputting the corresponding relationship between different fields and improving the accuracy of image text recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0063] Figure 1 A schematic diagram of a flow chart of an image text recognition method provided by an embodiment of the present invention is shown;

[0064] Figure 2 A schematic diagram showing a flow chart of another image text recognition method provided by an embodiment of the present invention;

[0065] Figure 3 A schematic diagram of the structure of an image text recognition device provided by an embodiment of the present invention is shown;

[0066] Figure 4 A schematic diagram of the structure of another image text recognition device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0067] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0068] The embodiment of the present invention provides an image text recognition method, which can perform security authentication on the service caller to ensure the security of the service call. Figure 1 As shown, the method includes:

[0069] 101. Acquire an image to be recognized, and preprocess the image to be recognized to obtain a target recognition image.

[0070] The image to be identified may be an invoice image, an advertisement image, a commodity image, etc. The preprocessing process of the image to be identified may include but is not limited to image rotation correction and image background interference removal.

[0071] It is understandable that when users upload images to be identified, they may not consider the direction of the image and the shooting angle. For example, some users are used to shooting horizontally, while others are used to shooting vertically. In order to facilitate subsequent image processing, the image needs to be angled. Here, a 4-classification model can be pre-trained based on a neural network using the Resnet152 network. After the image to be identified is input, the classification model can give a predicted angle of 0, 90, 180 and 270 degrees. The image to be identified is rotated and corrected according to the predicted angle output by the classification model.

[0072] The training process of the specific classification model can be implemented as follows: first prepare 4 categories of images as training data, namely 0 degree, 90 degree, 180 degree and 270 degree images and corresponding angle labels; then input the training data into the Resnet152 network, the network extracts the features of each image, and predicts the corresponding angle, based on the deviation between the predicted value and the actual value of the angle label as the loss, performs back propagation, updates the parameters in the network, after the classification model training is completed, retains the forward propagation, performs angle prediction on the image to be identified, obtains the predicted angle, and rotates the image to be identified whose corresponding predicted angle is not 0 degree to 0 degree, so that the predicted angle corresponding to the image to be identified is unified to 0 degree, thereby obtaining the image to be identified at the same angle.

[0073] It is understandable that the images to be identified uploaded by users usually contain complex background information. Some background textures are very similar to text textures, and some even contain text. These backgrounds cause great interference to recognition and detection, and it is necessary to remove the image background interference. Here, the network model built based on DeepLab-v3 can be combined with image segmentation technology to segment the background and foreground of the image to be identified, calculate the minimum outer rectangle of the foreground, and cut off the background along the minimum circumscribed matrix.

[0074] Specifically, the implementation process of the network model built based on DeepLab-v3 combined with image segmentation technology to segment the background and foreground of the image to be identified can be as follows: first prepare the training data, image data set and corresponding labels, the pixel size corresponding to the label of each image is consistent with the original image size, the pixel value corresponding to the label is 0, indicating the image background, and the pixel value corresponding to the label is 255, indicating the image foreground, build the DeepLab-v3 network, and input the training data, perform binary classification prediction for each pixel, and calculate the loss between the predicted value and the real pixel value of the label, and then back propagate and update the parameters in the network until the accuracy evaluation index mean-IOU of the network model reaches the preset value. Based on the constructed network model, the pixels in the image to be identified are identified, and the foreground and background labels in the image are obtained, that is, the pixel value of 0 is identified as the image background, and the pixel value of 255 is identified as the image foreground. The image to be identified is segmented based on the background label, and the background of the image to be identified is removed.

[0075] After preprocessing, the image to be identified can better express the image features, and thus can be used as a target image to further identify the text in the image.

[0076] 102. Determine position information of a text region in the target recognition image based on a pre-trained text region detection model.

[0077] Among them, the pre-trained text box detection model can use the open source Detecting Text in Natural Image with Connectionist Text Proposal Network (CTPN) framework. Each target recognition image will have a corresponding output file after passing through the text area detection model. The output file stores the location information of all text area detections. The output file can be used to determine the location information of the text area in the target recognition image.

[0078] The specific process of training the text area detection model can be as follows: first prepare the training data, that is, the image and the label file corresponding to the image. The label file stores the coordinate information of the text area in the image. In order to facilitate the subsequent detection of the text area in the image, before each training data is input into the CTPN network, the coordinate information marked by the text area needs to be converted into a small anchor with a width of 8. The CTPN network structure adopts the form of CNN+BLSTM+RPN. CNN is used to extract the spatial features of the receptive field in the image. The receptive field is the area of ​​the input image corresponding to the response of a certain node of the output feature map (convolved by the convolution kernel). BLSTM can generate horizontal text sequence features based on the spatial features of the receptive field. RPN consists of two parts, anchor classification and bounding box regressing. After the above anchor classification, it can be determined whether each area is a text area. After bounding box regressing, a set of vertical strip candidate text lines will be obtained.

[0079] It should be noted that the output of the pre-trained text region detection model is not directly the text region in the target recognition image, but a group of vertical strip candidate text lines that constitute the text region in the target recognition image. A text line construction algorithm can be used to connect groups of vertical strip candidate text lines into a text region, thereby determining the text region in the target recognition image and the location information of the text region.

[0080] 103. Input the target recognition image and the position information of the text area in the target recognition image into a pre-trained text recognition model to obtain the text information in the text area.

[0081] Among them, the text recognition model can use the An End-to-End Trainable Neural Network for Imaged-based Sequence Recognition and Its Application to Scene Text Recognition (CRNN) algorithm to train the recognition model. After the target recognition image and the position information of the text area in the target recognition image pass through the text recognition model, the text recognition result corresponding to each text area is output.

[0082] The specific process of training the CRNN model can be as follows: First, the training data is stored in the form of labels of images and text information in the text area of ​​the image. The CRNN network structure adopts the form of CNN+RNN+CTC. CNN is used to extract the spatial features of the receptive field in the image. RNN can predict the label distribution of each frame in the image based on the spatial features of the receptive field. CTC can integrate the label distribution of each frame into the final label sequence. For example, the input image is resized to a size of W*32, and the predicted value output by the text recognition model represents the text information corresponding to the text area in the target recognition image.

[0083] 104. Perform structural processing on the text information in the text area to obtain text fields with mapping relationships.

[0084] Since the text information in the text area obtained by the recognition model may contain various types, such as text type, number type, special character type, etc., and there may be a mapping relationship between them, for example, the "payment amount" text area corresponds to the "37.5 yuan" text area, and the "gender" text box corresponds to the "female" text area. By outputting the text information in a structured manner, the mapping relationship between the text information can be clearly seen.

[0085] Specifically, the text information identified in a specific text area in an image can be used as a key field, and the fuzzy area corresponding to the text field having a mapping relationship with the key field can be located based on the key field, and then the field having a mapping relationship with the key field can be locked in an area range, and then the text information within the area range can be verified to confirm the text information having a mapping relationship with the key field. Similarly, the same method can be used for other fields in the image to obtain text fields with a mapping relationship.

[0086] An image text recognition method provided by an embodiment of the present invention obtains an image to be recognized, pre-processes the image to be recognized, obtains a target recognition image, and determines the position information of the text area in the target recognition image based on a pre-trained text area detection model. After the target recognition image and the position information of the text area in the target recognition image are input into the pre-trained text recognition model, the text information in the text area is recognized, and the text information in the text area is structured into a text field with a mapping relationship. Compared with the image text recognition method in the prior art, the present application can effectively remove interference information in the image and accurately retain the text information of the image by performing structured processing on the recognized text information, so that the image in the text area detection and text information recognition is not interfered by the background, thereby outputting the corresponding relationship between different fields and improving the accuracy of image text recognition.

[0087] The embodiment of the present invention provides another image text recognition method, which can perform security authentication on the service caller to ensure the security of the service call. Figure 2 As shown, the method includes:

[0088] 201. Acquire an image to be recognized, and preprocess the image to be recognized to obtain a target recognition image.

[0089] For the embodiment of the present invention, the process of obtaining the image to be recognized, preprocessing the image to be recognized, and obtaining the target recognition image can refer to the contents described in step 101, which will not be described in detail here.

[0090] 202. After the collected image sample data is annotated with text regions, the data is input into a network model for training to obtain a text region detection model.

[0091] Among them, the image sample data can be images collected from different scenes, which can reflect the image features in different scenes. For example, the image features in the medical field are relatively simple or gradual, the image background scenes in industrial sites are relatively complex and the text area is relatively small, and the image background of natural scenes is highly affected by natural factors, and the background complexity is difficult to predict.

[0092] It is understandable that no matter what kind of scene the image is in, in general target detection, each target has a clear closed boundary. However, in the image, since the text line or word is composed of many separate characters or strokes, there may not be such a clearly defined boundary. It is necessary to detect the text area in the image first. Specifically, this can be done by annotating the text area contained in each image in the image sample data, training the annotated image sample data, building a text area detection model, and using the text area detection model to detect the text area in the image, thereby recognizing the text in the image.

[0093] For the embodiment of the present invention, the network model can adopt the CTPN network framework, which includes a three-layer structure. The first layer is a convolutional structure, that is, a CNN structure. The convolutional layer extracts the image area features corresponding to the image sample data, and can learn the spatial information of the receptive field; the second layer is a decoding layer, that is, a BLSTM structure. The decoding layer generates horizontal text sequence features according to the image area features corresponding to the image sample data, and can better cope with the sequence features of horizontal text; the third layer is a prediction layer, that is, an RPN structure. The prediction layer determines the text area in the image sample data according to the horizontal text sequence features, and processes the text area to obtain a candidate text line.

[0094] Specifically, the prediction layer of the network model includes a classification part and a regression part. In the process of determining the text area in the image sample data according to the horizontal text sequence features through the prediction layer of the network model, and processing the text area to obtain the candidate text line, the classification part of the prediction layer of the network model can classify each area in the image sample data according to the horizontal text sequence features to determine the text area in the image sample data; the regression part of the prediction layer of the network model performs border regression processing on the text area in the image text data to obtain the candidate text line.

[0095] In the specific implementation process, in the convolutional layer, CTPN can select the feature maps of conv5 in the VGG model as the final features of the image. At this time, the size of the feature maps is H×W×C; then due to the sequence relationship between texts, a 3×3 sliding window can be used in the decoding layer to extract the 3×3 area around each point on the feature maps as the feature vector representation of the point. At this time, the size of the image becomes H×W×9C, and then each row is used as the length of the sequence and the height as the batch_size, and passed into a 128-dimensional Bi-LSTM, and the output of the decoding layer is W×H×256; finally, the output of the decoding layer is connected to the prediction layer. The prediction layer includes two parts, anchor classification and bounding box regressing. After anchor classification, it can be determined whether each area in the image is a text area. After bounding box regressing, a set of vertical strip candidate text lines will be obtained, and the label of whether the candidate text line is a text area will be carried.

[0096] Furthermore, in order to ensure the accuracy of the prediction of the trained text region detection model, the pre-set loss function can adjust the parameters of the multi-layer structure in the text region detection model based on the deviation between the result output by the text region detection model and the data annotated by the real text region. For the embodiment of the present invention, the pre-trained loss function includes 3 parts, the first part is a loss function for detecting whether the anchor is a text region; the second part is a loss function for detecting the y-coordinate offset regression of the anchor; and the third part is a loss function for detecting the x-coordinate offset regression of the anchor.

[0097] 203. Determine position information of the text region in the target recognition image based on a pre-trained text region detection model.

[0098] It can be understood that each image will have a corresponding output file after passing through the text area detection model. The output file stores the position information of all candidate text lines in the image and the label of whether the candidate text line is a text area. The candidate text line here is equivalent to the vertical strip line split from the text area. By connecting the candidate text lines, the position information of the text area in the target image can be determined.

[0099] 204. Input the target recognition image and the position information of the text area in the target recognition image into a pre-trained text recognition model to obtain text information in the text area.

[0100] It can be understood that the trained text recognition model has the ability to recognize text information in the text area. In the process of training the text recognition model, sample images and position information labels of the text area in the sample images are used, and the parameters of the text recognition model are continuously adjusted through forward propagation and reverse bias correction, so that the image through the text recognition model can accurately recognize the text information in the text area.

[0101] 205. Select a preset field from the text information in the text area as a key field, and obtain position information of the text area corresponding to the key field.

[0102] Since key fields are usually fields with more reference value in the image, for invoice images, the invoice number, total amount, medical insurance type, quantity, date and time and other fields can usually be selected as preset fields as key fields. Here, the coordinate information corresponding to the key fields can be obtained through the text area detection model.

[0103] 206. Determine a fuzzy area having a mapping relationship with the key field according to the position information of the text area corresponding to the key field.

[0104] For an embodiment of the present invention, in order to accurately locate the text information mapped to the key field, since the text information having a mapping relationship with the key field is usually located on one side of the key field, the text information mapped to the key field can be located within a range within a fuzzy area by determining a fuzzy area having a mapping relationship with the key field. For example, the coordinate information of the text area corresponding to the key field can be [(Xmin, Ymin), (Xmax, Ymax)], where (Xmin, Ymin) is the coordinate of the upper left corner, and (Xmax, Ymax) is the coordinate of the lower right corner. Then, the coordinate information of the fuzzy area having a mapping relationship with the key field can be [(Xmin, Ymin+2 / 3(Ymax-Ymin)), (Xmin+1 / 2(Xmax-Xmin),Ymax)], where (Xmin, Ymin+2 / 3(Ymax-Ymin)) is the coordinate of the upper left corner, and (Xmin+1 / 2(Xmax-Xmin),Ymax) is the coordinate of the lower right corner.

[0105] Specifically, the text area corresponding to the key field can be moved by a preset distance in the horizontal and vertical directions, and the position information of the text area after moving is obtained according to the position information of the text area corresponding to the key field. For example, the text area is moved by 1 / 2 (Xmax-Xmin) distance in the horizontal direction and by 2 / 3 (Ymax-Ymin) distance in the vertical direction. Since the text area after moving may not cover the text information that has a mapping relationship with the key field, the text area after moving the preset distance is enlarged based on the position information of the text area after moving, and the fuzzy area that has a mapping relationship with the key field is determined.

[0106] 207. Detect and query the text information identified in the fuzzy area to confirm the text information having a mapping relationship with the key field.

[0107] Specifically, the location information of all text areas within the fuzzy area can be detected, and the text information of all text areas within the fuzzy area can be extracted. Since the text information of all text areas within the fuzzy area is not all text information that has a mapping relationship with the key field, the text information of each text area within the fuzzy area can be further traversed by regular matching to confirm the text information that has a mapping relationship with the key field.

[0108] For the embodiment of the present invention, in the process of traversing the text information of each text area in the fuzzy area by using the regular matching method to confirm the text information having a mapping relationship with the key field, a regular expression matching the key field can be constructed by obtaining the pattern characters applicable to the key field. For example, in the case where the total amount is used as the key field, the pattern characters applicable to the total amount may include zero, one, two, three, four, five, six, seven, eight, nine, ten thousand, hundred, yuan, yuan, etc. The constructed regular expression may be but not be “([whole zero one two three four five "6789]{1,2} ten thousand){0,1}([whole zero one twenty three forty five six seven eighty nine]{1,2} thousand){0,1}([whole zero one twenty three forty five six seven eighty nine]{1,2} hundred){0,1}([whole zero one twenty three forty five six seven eighty nine]{1,2} ten){0,1}zero{0,1}([whole zero one twenty three forty five six seven eighty nine]{0,2}[yuan]){0,1}([whole zero one twenty three forty five six seven eighty nine]{1,2} jiao){0,1}([whole zero one twenty three forty five six seven eighty nine]{1,2} fen){0,1}whole{0,1}". According to the regular expression matching the key field, the text information of each text area in the fuzzy area is verified to confirm the text information having a mapping relationship with the key field, so that the text information that satisfies the regular expression is the text information having a mapping relationship with the key field.

[0109] It should be noted that due to the differences in field formats and types in different text information, some fields are suitable for structured confirmation by regular matching, and other fields can be confirmed by coordinate calculation. Specifically, the coordinates of the lower left corner of the key field can be used as the basis, and the height of the key field text area can be used as the calculation unit for point movement. The text area closest to this point is calculated as the text area of ​​the next key field. Based on the text area of ​​the next key field, the text field that has a mapping relationship with the key field is further confirmed.

[0110] Further, as Figure 1 In a specific implementation of the method, an embodiment of the present invention provides an image text recognition device, such as Figure 3 As shown, the device includes: an acquisition unit 31, a determination unit 32, an identification unit 33, and a processing unit 34.

[0111] The acquisition unit 31 may be used to acquire the image to be recognized, and pre-process the image to be recognized to obtain a target recognition image;

[0112] The determination unit 32 may be used to determine the position information of the text region in the target recognition image based on a pre-trained text region detection model;

[0113] The recognition unit 33 may be used to input the target recognition image and the position information of the text area in the target recognition image into a pre-trained text recognition model to obtain the text information in the text area;

[0114] The processing unit 34 may be used to perform structural processing on the text information in the text area to obtain text fields with mapping relationships.

[0115] An image text recognition device provided by an embodiment of the present invention obtains an image to be recognized, pre-processes the image to be recognized, obtains a target recognition image, and determines the position information of the text area in the target recognition image based on a pre-trained text area detection model. After the target recognition image and the position information of the text area in the target recognition image are input into the pre-trained text recognition model, the text information in the text area is recognized, and the text information in the text area is structured into a text field with a mapping relationship. Compared with the image text recognition method in the prior art, the present application can effectively remove interference information in the image and accurately retain the text information of the image by performing structured processing on the recognized text information, so that the image in the text area detection and text information recognition is not interfered by the background, thereby outputting the corresponding relationship between different fields and improving the accuracy of image text recognition.

[0116] As Figure 3 Further description of the image text recognition device shown in, Figure 4 FIG. 1 is a schematic diagram of the structure of another image text recognition device according to an embodiment of the present invention. Figure 4 As shown, the processing unit 34 includes:

[0117] The selection module 341 may be used to select a preset field from the text information in the text area as a key field, and obtain the position information of the text area corresponding to the key field;

[0118] The determination module 342 may be used to determine the fuzzy area having a mapping relationship with the key field according to the position information of the text area corresponding to the key field;

[0119] The detection module 343 can be used to detect and query the text information identified in the fuzzy area, and confirm the text information having a mapping relationship with the key field.

[0120] Further, the determination module 342 includes:

[0121] The acquisition submodule 3421 may be used to move the text area corresponding to the key field in the horizontal and vertical directions by a preset distance, and acquire the position information of the text area after the movement according to the position information of the text area corresponding to the key field;

[0122] The determination submodule 3422 may be used to amplify the text area after the text area has moved a preset distance based on the position information of the text area after the text area has moved, and determine a fuzzy area having a mapping relationship with the key field.

[0123] Furthermore, the detection module 343 includes:

[0124] The extraction submodule 3431 may be used to detect the position information of all text regions within the fuzzy region and extract the text information of all text regions within the fuzzy region;

[0125] The confirmation submodule 3432 can be used to traverse the text information of each text area in the fuzzy area by using a regular matching method to confirm the text information having a mapping relationship with the key field.

[0126] Furthermore, the confirmation submodule 3432 may be specifically used to construct a regular expression matching the key field by acquiring a pattern character applicable to the key field;

[0127] The confirmation submodule 3432 may be specifically configured to verify the text information of each text area in the fuzzy area according to the regular expression matching the key field, and confirm the text information having a mapping relationship with the key field.

[0128] Furthermore, the device also includes:

[0129] The training unit 35 may be used to annotate the collected image sample data with text regions and input them into the network model for training before determining the position information of the text region in the target recognition image based on the pre-trained text region detection model to obtain a text region detection model;

[0130] The network model includes a multi-layer structure, and the training unit 35 includes:

[0131] The extraction module 351 can be used to extract image region features corresponding to the image sample data through the convolution layer of the network model;

[0132] A generating module 352 may be used to generate horizontal text sequence features according to image region features corresponding to the image sample data through a decoding layer of the network model;

[0133] The prediction module 353 may be used to determine the text region in the image sample data according to the horizontal text sequence features through the prediction layer of the network model, and process the text region to obtain candidate text lines.

[0134] Furthermore, the prediction layer of the network model includes a classification part and a regression part, and the prediction module 353 includes:

[0135] The classification submodule 3531 may be used to classify each region in the image sample data according to the horizontal text sequence feature through the classification part of the prediction layer of the network model, and determine the text region in the image sample data;

[0136] The processing submodule 3532 can be used to perform border regression processing on the text area in the image text data through the regression part of the prediction layer of the network model to obtain candidate text lines.

[0137] It should be noted that for other corresponding descriptions of the functional units involved in the image text recognition device provided in this embodiment, please refer to Figure 1 , Figure 2 The corresponding description in will not be repeated here.

[0138] Based on the above Figure 1 , Figure 2 The method shown in the embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the above-mentioned Figure 1 , Figure 2 The image text recognition method shown.

[0139] Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each implementation scenario of the present application.

[0140] Based on the above Figure 1 , Figure 2 The method shown, and Figure 3 , Figure 4 In order to achieve the above-mentioned purpose, the embodiment of the present application also provides a computer device, which can be a personal computer, a server, a network device, etc. The physical device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above-mentioned Figure 1 , Figure 2 The image text recognition method shown.

[0141] Optionally, the computer device may further include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, etc. The user interface may include a display, an input unit such as a keyboard, etc., and the optional user interface may also include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a Bluetooth interface, a WI-FI interface), etc.

[0142] Those skilled in the art will appreciate that the physical device structure of the image text recognition apparatus provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or a combination of certain components, or different component arrangements.

[0143] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the above-mentioned computer device, and supports the operation of information processing programs and other software and / or programs. The network communication module is used to realize communication between the components inside the storage medium, and communication with other hardware and software in the physical device.

[0144] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform, or by hardware. By applying the technical solution of the present application, compared with the current prior art, the present application can effectively remove interference information in the image by performing structured processing on the recognized text information, accurately retain the text information of the image, and make the image in the text area detection and text information recognition free from background interference, thereby outputting the corresponding relationship between different fields and improving the accuracy of image text recognition.

[0145] Those skilled in the art will appreciate that the accompanying drawings are only schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily necessary for implementing the present application. Those skilled in the art will appreciate that the modules in the devices in the implementation scenario can be distributed in the devices of the implementation scenario according to the description of the implementation scenario, or can be changed accordingly and located in one or more devices different from the present implementation scenario. The modules of the above-mentioned implementation scenario can be combined into one module, or can be further split into multiple submodules.

[0146] The above serial numbers of this application are only for description and do not represent the advantages and disadvantages of the implementation scenarios. The above disclosure is only a few specific implementation scenarios of this application, but this application is not limited to them, and any changes that can be thought of by technicians in this field should fall within the scope of protection of this application.

Claims

1. A method for image text recognition, characterized in that: The method comprises: Acquire an image to be recognized, and preprocess the image to be recognized to obtain a target recognition image; Determining location information of a text region in the target recognition image based on a pre-trained text region detection model; Inputting the target recognition image and the position information of the text area in the target recognition image into a pre-trained text recognition model to obtain the text information in the text area; The text information in the text area is structured to obtain a text field with a mapping relationship, which specifically includes: selecting a preset field as a key field from the text information in the text area, and obtaining the position information of the text area corresponding to the key field; moving the text area corresponding to the key field by a preset distance in the horizontal and vertical directions, and obtaining the position information of the text area after the movement according to the position information of the text area corresponding to the key field; based on the position information of the text area after the movement, enlarging the text area after the preset distance is moved, and determining a fuzzy area with a mapping relationship with the key field; detecting the position information of all text areas in the fuzzy area, and extracting the text information of all text areas in the fuzzy area; traversing the text information of each text area in the fuzzy area by regular matching, and confirming the text information with a mapping relationship with the key field, wherein the preset field is a field with reference value in the image.

2. The method according to claim 1, characterized in that: The method of traversing the text information of each text area in the fuzzy area by using a regular matching method to confirm the text information having a mapping relationship with the key field specifically includes: By obtaining the pattern characters applicable to the key field, a regular expression matching the key field is constructed; According to the regular expression that matches the key field, the text information of each text area in the fuzzy area is verified to confirm the text information that has a mapping relationship with the key field.

3. The method according to any one of claims 1 to 2, characterized in that: Before determining the position information of the text region in the target recognition image based on the pre-trained text region detection model, the method further includes: The collected image sample data is annotated with text regions and then input into the network model for training to obtain a text region detection model; The network model includes a multi-layer structure. The collected image sample data is annotated with text regions and then input into the network model for training to obtain a text region detection model, which specifically includes: Extracting image region features corresponding to the image sample data through the convolutional layer of the network model; Generate horizontal text sequence features according to image region features corresponding to image sample data through the decoding layer of the network model; The text area in the image sample data is determined according to the horizontal text sequence features through the prediction layer of the network model, and the text area is processed to obtain candidate text lines.

4. The method according to claim 3, characterized in that The prediction layer of the network model includes a classification part and a regression part. The prediction layer of the network model determines the text area in the image sample data according to the horizontal text sequence features, and processes the text area to obtain candidate text lines, specifically including: Classifying each region in the image sample data according to the horizontal text sequence features by the classification part of the prediction layer of the network model, and determining the text region in the image sample data; The text area in the image text data is processed by border regression through the regression part of the prediction layer of the network model to obtain candidate text lines.

5. An image text recognition device, characterized in that: The device comprises: An acquisition unit, used for acquiring an image to be recognized, and preprocessing the image to be recognized to obtain a target recognition image; A determination unit, configured to determine position information of a text region in the target recognition image based on a pre-trained text region detection model; A recognition unit, used for inputting the target recognition image and the position information of the text area in the target recognition image into a pre-trained text recognition model to obtain the text information in the text area; The processing unit is used to perform structural processing on the text information in the text area to obtain a text field with a mapping relationship, specifically including: selecting a preset field as a key field from the text information in the text area, and obtaining the position information of the text area corresponding to the key field; moving the text area corresponding to the key field by a preset distance in the horizontal and vertical directions, and obtaining the position information of the text area after the movement according to the position information of the text area corresponding to the key field; based on the position information of the text area after the movement, enlarging the text area after the movement of the preset distance, and determining a fuzzy area with a mapping relationship with the key field; detecting the position information of all text areas in the fuzzy area, and extracting the text information of all text areas in the fuzzy area; traversing the text information of each text area in the fuzzy area by regular matching, and confirming the text information with a mapping relationship with the key field, wherein the preset field is a field with reference value in the image.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

7. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images