Ocr recognition method and apparatus

Abstract

The application relates to the technical field of text recognition, and provides an OCR (Optical Character Recognition) recognition method and device, which comprises the following steps: inputting an image to be recognized into an OCR recognition model to obtain a text line center region predicted by the OCR recognition model, character region center points and pixel categories in the character region; expanding the text line center region to obtain a text line region; acquiring the character region center points in the text line region; determining character categories in the text line region according to the character region center points and the pixel categories in the corresponding character region to obtain a character string result of each text line, wherein the OCR recognition model is obtained based on a sample image, a text line center region label corresponding to the sample image, a character region center point label and a pixel category label in the character region. The application realizes end-to-end OCR recognition, simplifies the OCR recognition process, does not exist error accumulation, guarantees the recognition effect and can reduce the overall model time consumption.

Description

Technical Field

[0001] This invention relates to the field of text recognition technology, and in particular to an OCR recognition method and apparatus. Background Technology

[0002] Optical character recognition (OCR) technology identifies text lines and their content within images. Current OCR technology typically involves two steps: text detection and text recognition. Text detection checks for the presence of text lines in the image, while text recognition identifies the specific text content within those lines. Currently, these two steps are performed separately in two different models, leading to increased computational load and the potential for error accumulation. Summary of the Invention

[0003] This invention provides an OCR recognition method and apparatus to solve the problems of increased model computation and error accumulation caused by dividing OCR recognition into two steps, text detection and text recognition, in the prior art.

[0004] This invention provides an OCR recognition method, comprising:

[0005] The image to be recognized is input into the OCR recognition model to obtain the text line center region, character region center point and pixel category in the character region predicted by the OCR recognition model.

[0006] Expand the center area of ​​the text line to obtain the text line area;

[0007] Get the center point of each character region within the text line area;

[0008] The character categories in the text line region are determined based on the center point of each character region and the pixel category in the corresponding character region, in order to obtain the string result of each line of text.

[0009] The OCR recognition model is trained based on sample images, the text line center region labels corresponding to the sample images, the character region center point labels, and the pixel category labels in the character regions.

[0010] The OCR recognition model is used to predict the center region of the text line, the center point of the character region, and the pixel category in the character region of the image to be recognized based on the image to be recognized.

[0011] According to the OCR recognition method provided by the present invention, the OCR recognition model includes: a feature extraction layer, a first network output layer, and a second network output layer.

[0012] The feature extraction layer is used to extract image features from the image to be identified;

[0013] The first network output layer is used to predict the center point of the character region and the pixel category in the character region based on the image features;

[0014] The second network output layer is used to predict the center region of the text line based on the downsampled image features.

[0015] According to an OCR recognition method provided by the present invention, an image to be recognized is input into an OCR recognition model to obtain the text line center region, character region center point, and pixel category in the character region predicted by the OCR recognition model, including:

[0016] The image to be identified is input into the feature extraction layer to obtain the image features;

[0017] The image features are input into the first network output layer to obtain the character region center point and pixel category in the character region predicted by the first network output layer;

[0018] The image features are downsampled, and the downsampled image features are input into the output layer of the second network to obtain the text line center region predicted by the output layer of the second network.

[0019] According to an OCR recognition method provided by the present invention, before inputting the image to be recognized into the OCR recognition model, the method further includes: training the OCR recognition model, specifically including:

[0020] The sample image is input into the feature extraction layer;

[0021] The feature extraction layer extracts sample features from the sample image;

[0022] The sample features are input into the first network output layer to obtain the character region center point prediction result and the pixel category prediction result in the character region predicted by the first network output layer.

[0023] The sample features are input into the output layer of the second network to obtain the prediction results of the text line center region and the pixel line category of the second network output layer.

[0024] The prediction results of the character region center point, the pixel category prediction results in the character region, the prediction results of the text line center region and the pixel line category prediction results, along with the corresponding labels, are input into the total loss function. Training is complete when the total loss function converges.

[0025] According to an OCR recognition method provided by the present invention, the total loss function is obtained by weighted summation of the character center region probability loss function, the pixel character category loss function, the text line center region probability loss function, and the pixel line category loss function.

[0026] According to an OCR recognition method provided by the present invention, expanding the central region of a text line to obtain a text line region includes:

[0027] The text line region is obtained by expanding the central area of ​​the text line by at least two times.

[0028] According to an OCR recognition method provided by the present invention, the character category in a text line region is determined based on the center point of each character region and the pixel category in the corresponding character region to obtain the string result of each line of text, including:

[0029] Obtain the center point of each character region and the pixel category of the corresponding character region;

[0030] The pixel category percentage is statistically analyzed, and the pixel category with the largest percentage is determined as the character category to obtain the string result for each line of text.

[0031] The present invention also provides an OCR recognition device, comprising:

[0032] The model running module is used to input the image to be recognized into the OCR recognition model and obtain the text line center region, character region center point and pixel category in the character region predicted by the OCR recognition model.

[0033] The text line expansion module is used to expand the central area of ​​a text line to obtain a text line region.

[0034] The inline character acquisition module is used to obtain the center point of each character region within a text line area;

[0035] The character category determination module is used to determine the character category in the text line area based on the center point of each character region and the pixel category in the corresponding character region, so as to obtain the string result of each line of text.

[0036] The OCR recognition model is trained based on sample images, the text line center region labels corresponding to the sample images, the character region center point labels, and the pixel category labels in the character regions.

[0037] The OCR recognition model is used to predict the center region of the text line, the center point of the character region, and the pixel category in the character region of the image to be recognized based on the image to be recognized.

[0038] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the OCR recognition method as described above.

[0039] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the OCR recognition method as described above.

[0040] The OCR recognition method and apparatus provided by this invention perform text detection and text recognition within the same OCR recognition model to obtain the text line center region, character region center points, and pixel categories within the character regions. Then, the text line region is obtained by expanding the text line center region; the center points of each character region within the text line region are acquired; and the character categories within the text line region are determined based on the center points of each character region and the corresponding pixel categories, thus obtaining the string result for each line of text. This achieves end-to-end OCR recognition, simplifies the OCR recognition process, eliminates error accumulation, and ensures recognition effectiveness while reducing overall model time. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0042] Figure 1 This is a flowchart illustrating the OCR recognition method provided by the present invention;

[0043] Figure 2 This is a schematic diagram of the OCR recognition model structure in the OCR recognition method provided by the present invention;

[0044] Figure 3 This is a schematic diagram of the structure of the OCR recognition device provided by the present invention;

[0045] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0047] The OCR recognition method of this invention is as follows: Figure 1 As shown, it includes:

[0048] Step S110: Input the image to be recognized into the OCR recognition model to obtain the text line center region, character region center point, and pixel category in the character region predicted by the OCR recognition model. The OCR recognition model is trained based on a sample image, the corresponding text line center region label, character region center point label, and pixel category label in the character region. The OCR recognition model is used to predict the text line center region, character region center point, and pixel category in the character region of the image to be recognized.

[0049] Step S120: Expand the central area of ​​the text line to obtain the text line area. Specifically, expand the central area of ​​the text line to the upper and lower sides to the width of one text line.

[0050] Step S130: Obtain the center point of each character region within the text line area in order to identify the string in the line.

[0051] Step S140: Determine the character category in the text line area based on the center point of each character area and the pixel category in the corresponding character area to obtain the string result of each line of text. Each character category represents a character; for example, category A also represents character A.

[0052] The OCR recognition method provided in this embodiment performs text detection and text recognition within the same OCR recognition model to obtain the text line center region, character region center points, and pixel categories within the character regions. Then, the text line region is obtained by expanding the text line center region; the center points of each character region within the text line region are acquired; and the character categories within the text line region are determined based on the center points of each character region and the corresponding pixel categories, thus obtaining the string result for each line of text. This achieves end-to-end OCR recognition, simplifies the OCR recognition process, eliminates error accumulation, and ensures recognition quality while reducing overall model time.

[0053] The OCR recognition method in this embodiment is particularly suitable for applications with a limited number of character categories, such as recognizing only English text, where the character categories include only English letters, numbers, and punctuation.

[0054] like Figure 2 As shown, the OCR recognition model includes: a feature extraction layer, a first network output layer, and a second network output layer.

[0055] The feature extraction layer is used to extract image features from the image to be identified. Specifically, the feature extraction layer includes a network backbone layer and an encoding layer. The network backbone layer performs convolution operations on the input image to be identified to extract image features, and then inputs the extracted image features into the encoding layer. The encoding layer is mainly used to upsample the input image features, making the semantic information of the obtained image features richer and the final prediction result more accurate.

[0056] The first network output layer is used to predict the center point of the character region and the pixel category in the character region based on the image features, that is, the first network output layer is used for text recognition.

[0057] The second network output layer is used to predict the central region of the text line based on the downsampled image features; that is, the second network output layer is used for text detection. Since the prediction of the central region of the text line does not require many image features, the second network output layer downsamples the image features extracted by the feature extraction layer before prediction. This reduces the computational load when predicting the central region of the text line while ensuring prediction accuracy, thus improving the model's recognition efficiency.

[0058] The OCR recognition model in this embodiment performs text detection and text recognition prediction in the same model, realizing end-to-end OCR recognition, simplifying the OCR recognition process, ensuring the effect while reducing the overall model time.

[0059] Based on the above structural model, step S110 includes:

[0060] The image to be identified is input into the feature extraction layer to obtain the image features.

[0061] The image features are input into the output layer of the first network to obtain the center point of the character region and the pixel category in the character region predicted by the output layer of the first network.

[0062] The image features are downsampled, and the downsampled image features are input into the output layer of the second network to obtain the text line center region predicted by the output layer of the second network.

[0063] In this embodiment, before step S110, the method further includes: training the OCR recognition model, specifically including:

[0064] The sample image is input into the feature extraction layer.

[0065] The feature extraction layer extracts sample features from the sample image.

[0066] The sample features are input into the output layer of the first network to obtain the prediction results of the character region center point and the pixel category prediction results in the character region.

[0067] The sample features are input into the output layer of the second network to obtain the prediction results of the text line center region and the pixel line category of the second network output layer.

[0068] The prediction results of the character region center point, the pixel category prediction results in the character region, the prediction results of the text line center region and the pixel line category prediction results, along with the corresponding labels, are input into the total loss function. Training is complete when the total loss function converges.

[0069] The total loss function is obtained by weighted summation of the character center region probability loss function, the pixel character category loss function, the text line center region probability loss function, and the pixel line category loss function.

[0070] like Figure 2 As shown, the prediction result of the first network output layer, i.e., text recognition, includes the following two branches:

[0071] The probabilistic graph branch is used to predict the probability of the character center region, that is, to predict the probability that each pixel is located in the character center region. The center region is a Gaussian circle region centered on the center point. The center point can be understood as the center of a character, and the center region can be understood as the area covered by the character.

[0072] The probability loss function for the character center region is the focal loss function, denoted as focal_loss1(pred1, target1). In this loss function, the prediction result pred1 during training is a value between 0 and 1. For the sample image, the pixel located in the character center region is labeled as 1, and the pixels in other regions are labeled as 0. Therefore, the label target1 is a label matrix of 0 or 1. When the predicted value in pred1 corresponding to the element in target1 that is 1 is close to 1 within the allowable error range (0.1), the focal loss function focal_loss1 is considered to have converged.

[0073] Multi-class segmentation branch is used for character category prediction, that is, predicting which character category each pixel belongs to.

[0074] The pixel character category loss function is the cross-entropy loss function, denoted as Cross_Entropy(pred2, target2). For each pixel, target2 is the specific category label of that pixel. The prediction result pred2 during training is a vector representing the probability value or confidence of each category. If the category and category label corresponding to the element with the highest confidence in the vector are the same, then the cross-entropy loss function Cross_Entropy is considered to have converged.

[0075] The prediction results of the second network output layer, i.e., text detection, include the following two branches:

[0076] The probabilistic graph branch is used to predict the probability of the text line's central region. The central region is the area formed by shrinking the text line's rectangular frame (obtained from sample annotations) inward by half. Its value ranges from 0 to 1, with higher probabilities closer to the center (maximum 1). The probability at the edge of the central region is 0.

[0077] The probability loss function for the text line center region is the focal loss function, denoted as focal_loss2(pred3, target3), where the prediction result pred3 during training is a value between 0 and 1. In the sample image, pixels located in the center region of the text line are labeled as 1, and pixels in other regions are labeled as 0. Therefore, target3 is a label matrix of 0s and 1s. The focal loss function focal_loss2 is considered converged when the predicted value of the position corresponding to the element in target3 that is 1 is close to 1 within the allowable error range (0.1).

[0078] The binary classification segmentation branch is used for text line prediction, that is, predicting whether each pixel belongs to a text line or to the background.

[0079] The pixel row category loss function is the binary cross-entropy loss function BCE loss, denoted as BCE(pred4, target4). For each pixel, target4 is the specific category label of that pixel. There are only two categories: text row or background. The prediction result pred4 during training is a vector (containing two elements). The two elements in the vector represent the probability value or confidence of the two categories, respectively. If the category and category label corresponding to the element with the highest confidence in the vector are the same, then the cross-entropy loss function BCE is considered to have converged.

[0080] The total loss function obtained by weighted summation of the above loss functions is: Loss = a·focal_loss1(pred1,target1) + b·Cross_Entropy(pred2,target2)

[0081] +c·focal_loss2(pred3, target3)+d·BCE(pred4, target4)

[0082] In this model, pred1 represents the predicted center point of the character region, target1 represents the label of the center point of the character region, pred2 represents the predicted pixel category of the character region, target2 represents the label of the pixel category of the character region, pred3 represents the predicted center region of the text line, target3 represents the predicted label of the center region of the text line, pred4 represents the predicted pixel category of the pixel line, target4 represents the label of the pixel line category, and a, b, c, and d represent the weights of each loss function, all of which are greater than 0. Based on multiple training tests, the model with a value of 0.5 and values ​​of 1 for a, b, and c exhibits a high recognition rate.

[0083] In this embodiment, step S120 includes: expanding the central region of the text line by at least two times to obtain the text line region. Since the central region of the text line is the region after the text line rectangle is reduced inward by 1 / 2, it needs to be expanded by at least 2 times to ensure that the text line region covers the entire line of text on both the top and bottom sides.

[0084] In this embodiment, step S140 includes:

[0085] Obtain the center point of each character region and the pixel category of the corresponding character region.

[0086] The pixel category percentage is statistically analyzed, and the pixel category with the largest percentage is determined as the character category to obtain the string result for each line of text. Within a character region, the pixel category with the largest percentage indicates that most pixels in that character region belong to one pixel category. For example, if the category is letter A, it means that the character is highly likely to be the letter A; therefore, the character is directly identified as the letter A.

[0087] The OCR recognition device provided by the present invention is described below. The OCR recognition device described below can be referred to in correspondence with the OCR recognition method described above.

[0088] like Figure 3 As shown, the OCR recognition device of the present invention includes:

[0089] The model running module 310 is used to input the image to be recognized into the OCR recognition model to obtain the text line center region, character region center point and pixel category in the character region predicted by the OCR recognition model.

[0090] The text line extension module 320 is used to extend the central area of ​​the text line to obtain the text line area;

[0091] The inline character acquisition module 330 is used to acquire the center point of each character region within the text line area;

[0092] The character category determination module 340 is used to determine the character category in the text line area based on the center point of each character area and the pixel category in the corresponding character area, so as to obtain the string result of each line of text.

[0093] The OCR recognition model is trained based on a sample image, the text line center region label corresponding to the sample image, the character region center point label, and the pixel category label in the character region.

[0094] The OCR recognition model is used to predict the center region of the text line, the center point of the character region, and the pixel category in the character region of the image to be recognized based on the image to be recognized.

[0095] The OCR recognition device provided by this invention performs text detection and text recognition within the same OCR recognition model to obtain the text line center region, character region center points, and pixel categories within the character regions. Then, the text line region is obtained by expanding the text line center region. The center points of each character region within the text line region are acquired. Based on the center points of each character region and the corresponding pixel categories within the character regions, the character categories within the text line region are determined to obtain the string result for each line of text. This achieves end-to-end OCR recognition, simplifies the OCR recognition process, eliminates error accumulation, ensures recognition effectiveness, and reduces overall model time consumption.

[0096] Optionally, the OCR recognition model includes: a feature extraction layer, a first network output layer, and a second network output layer.

[0097] The feature extraction layer is used to extract image features from the image to be identified.

[0098] The first network output layer is used to predict the center point of the character region and the pixel category in the character region based on the image features.

[0099] The second network output layer is used to predict the center region of the text line based on the downsampled image features.

[0100] Optionally, the model running module 310 is specifically used for:

[0101] The image to be identified is input into the feature extraction layer to obtain the image features.

[0102] The image features are input into the output layer of the first network to obtain the center point of the character region and the pixel category in the character region predicted by the output layer of the first network.

[0103] The image features are downsampled, and the downsampled image features are input into the output layer of the second network to obtain the text line center region predicted by the output layer of the second network.

[0104] Optionally, the OCR recognition device of the present invention further includes: a model training module, used to train the OCR recognition model, specifically used for:

[0105] The sample image is input into the feature extraction layer.

[0106] The feature extraction layer extracts sample features from the sample image.

[0107] The sample features are input into the output layer of the first network to obtain the prediction results of the character region center point and the pixel category prediction results in the character region.

[0108] The sample features are input into the output layer of the second network to obtain the prediction results of the text line center region and the pixel line category of the second network output layer.

[0109] The prediction results of the character region center point, the pixel category prediction results in the character region, the prediction results of the text line center region and the pixel line category prediction results, along with the corresponding labels, are input into the total loss function. Training is complete when the total loss function converges.

[0110] Optionally, the total loss function is obtained by weighted summation of the character center region probability loss function, the pixel character category loss function, the text line center region probability loss function, and the pixel line category loss function.

[0111] Optionally, the text line expansion module 320 is specifically used to expand the central area of ​​the text line by at least two times to obtain the text line area.

[0112] Optionally, the character category determination module 340 is specifically used to obtain the center point of each character region and the pixel category in the corresponding character region; to count the proportion of pixel categories, and to determine the pixel category with the largest proportion as the character category, so as to obtain the string result of each line of text.

[0113] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute an OCR recognition method, which includes:

[0114] The image to be recognized is input into the OCR recognition model to obtain the text line center region, character region center point, and pixel category in the character region predicted by the OCR recognition model.

[0115] Expand the center area of ​​the text line to obtain the text line area.

[0116] Get the center point of each character region within the text line area.

[0117] The character categories in the text line area are determined based on the center point of each character area and the pixel category in the corresponding character area, so as to obtain the string result of each line of text.

[0118] The OCR recognition model is trained based on a sample image, the text line center region label corresponding to the sample image, the character region center point label, and the pixel category label in the character region.

[0119] The OCR recognition model is used to predict the center region of the text line, the center point of the character region, and the pixel category in the character region of the image to be recognized based on the image to be recognized.

[0120] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0121] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is able to execute the OCR recognition method provided by the above methods, the method comprising:

[0122] The image to be recognized is input into the OCR recognition model to obtain the text line center region, character region center point, and pixel category in the character region predicted by the OCR recognition model.

[0123] Expand the center area of ​​the text line to obtain the text line area.

[0124] Get the center point of each character region within the text line area.

[0125] The character categories in the text line area are determined based on the center point of each character area and the pixel category in the corresponding character area, so as to obtain the string result of each line of text.

[0126] The OCR recognition model is trained based on a sample image, the text line center region label corresponding to the sample image, the character region center point label, and the pixel category label in the character region.

[0127] The OCR recognition model is used to predict the center region of the text line, the center point of the character region, and the pixel category in the character region of the image to be recognized based on the image to be recognized.

[0128] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the OCR recognition methods provided by the above methods, the method comprising:

[0129] The image to be recognized is input into the OCR recognition model to obtain the text line center region, character region center point, and pixel category in the character region predicted by the OCR recognition model.

[0130] Expand the center area of ​​the text line to obtain the text line area.

[0131] Get the center point of each character region within the text line area.

[0132] The character categories in the text line area are determined based on the center point of each character area and the pixel category in the corresponding character area, so as to obtain the string result of each line of text.

[0133] The OCR recognition model is trained based on a sample image, the text line center region label corresponding to the sample image, the character region center point label, and the pixel category label in the character region.

[0134] The OCR recognition model is used to predict the center region of the text line, the center point of the character region, and the pixel category in the character region of the image to be recognized based on the image to be recognized.

[0135] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0136] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0137] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An OCR recognition method, characterized in that, include: The image to be recognized is input into the OCR recognition model to obtain the text line center region, character region center point and pixel category in the character region predicted by the OCR recognition model. Expand the center area of ​​the text line to obtain the text line area; Get the center point of each character region within the text line area; The character categories in the text line region are determined based on the center point of each character region and the pixel category in the corresponding character region, in order to obtain the string result of each line of text. The OCR recognition model is trained based on sample images, text line center region labels corresponding to the sample images, character region center point labels, pixel line category labels, and pixel category labels within the character regions. The OCR recognition model is used to predict the central region of the text line, the center point of the character region, and the pixel category in the character region of the image to be recognized based on the image to be recognized. The OCR recognition model includes: a feature extraction layer, a first network output layer, and a second network output layer. The feature extraction layer is used to extract image features from the image to be identified; The first network output layer is used to predict the center point of the character region and the pixel category in the character region based on the image features; The second network output layer is used to predict the center region of the text line based on the downsampled image features.

2. The OCR recognition method according to claim 1, characterized in that, The image to be recognized is input into the OCR recognition model to obtain the text line center region, character region center point, and pixel category in the character region predicted by the OCR recognition model, including: The image to be identified is input into the feature extraction layer to obtain the image features; The image features are input into the first network output layer to obtain the character region center point and pixel category in the character region predicted by the first network output layer; The image features are downsampled, and the downsampled image features are input into the output layer of the second network to obtain the text line center region predicted by the output layer of the second network.

3. The OCR recognition method according to claim 1, characterized in that, Before inputting the image to be recognized into the OCR recognition model, the process also includes: training the OCR recognition model, specifically including: The sample image is input into the feature extraction layer; The feature extraction layer extracts sample features from the sample image; The sample features are input into the first network output layer to obtain the character region center point prediction result and the pixel category prediction result in the character region predicted by the first network output layer. The sample features are input into the output layer of the second network to obtain the prediction results of the text line center region and the pixel line category of the second network output layer. The prediction results of the character region center point, the pixel category prediction results in the character region, the prediction results of the text line center region and the pixel line category prediction results, along with the corresponding labels, are input into the total loss function. Training is complete when the total loss function converges.

4. The OCR recognition method according to claim 3, characterized in that, The total loss function is obtained by weighted summation of the character center region probability loss function, the pixel character category loss function, the text line center region probability loss function, and the pixel line category loss function.

5. The OCR recognition method according to any one of claims 1 to 4, characterized in that, Expanding the center area of ​​the text line results in a text line region, including: The text line region is obtained by expanding the central area of ​​the text line by at least two times.

6. The OCR recognition method according to any one of claims 1 to 4, characterized in that, The character categories in the text line region are determined based on the center point of each character region and the pixel category in the corresponding character region, in order to obtain the string result of each line of text, including: Obtain the center point of each character region and the pixel category of the corresponding character region; The percentage of pixel categories is statistically analyzed, and the pixel category with the largest percentage is determined as the character category to obtain the string result for each line of text.

7. An OCR recognition device, characterized in that, include: The model running module is used to input the image to be recognized into the OCR recognition model and obtain the text line center region, character region center point and pixel category in the character region predicted by the OCR recognition model. The text line expansion module is used to expand the central area of ​​a text line to obtain a text line region. The inline character acquisition module is used to obtain the center point of each character region within a text line area; The character category determination module is used to determine the character category in the text line area based on the center point of each character region and the pixel category in the corresponding character region, so as to obtain the string result of each line of text. The OCR recognition model is trained based on sample images, text line center region labels corresponding to the sample images, character region center point labels, pixel line category labels, and pixel category labels within the character regions. The OCR recognition model is used to predict the central region of the text line, the center point of the character region, and the pixel category in the character region of the image to be recognized based on the image to be recognized. The OCR recognition model includes: a feature extraction layer, a first network output layer, and a second network output layer. The feature extraction layer is used to extract image features from the image to be identified; The first network output layer is used to predict the center point of the character region and the pixel category in the character region based on the image features; The second network output layer is used to predict the center region of the text line based on the downsampled image features.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the OCR recognition method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the OCR recognition method as described in any one of claims 1 to 6.

Images