An OCR recognition method and system for text of any angle and any shape

Abstract

The application provides an OCR recognition method and system for text of any angle and any shape, and relates to the field of character recognition.The method comprises the following steps: obtaining a plurality of character images, performing labeling processing and calibration processing on the character images based on a five-tuple representation form to form an image training set, constructing a character recognition neural network model, training the character recognition neural network model based on the image training set, predicting position information and category information of characters in a to-be-recognized character image based on the trained character recognition neural network model, wherein the position information comprises coordinate information, shape information and angle information, and performing character connection based on the position information and category information of the characters in the to-be-recognized character image to output a text recognition result of any angle and any shape.The application solves the problem that the existing OCR recognition technology cannot recognize characters of text of any angle and any shape, and is prone to misrecognition, missed recognition or even cannot recognize.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of character recognition, in particular to an OCR recognition method and system for text of arbitrary angle and arbitrary shape. BACKGROUND

[0002] Optical Character Recognition (OCR) is a process of analyzing and processing a text image to obtain character and layout information.

[0003] OCR recognition technology mainly includes two parts: detection and recognition. The first part is to detect which areas in the image contain text; the second part is to recognize the specific text in the area. Current OCR recognition technology has been mature in the two scenarios of scanning documents and street signs, which have the common feature that the text direction is forward and the text can be read from left to right in sequence. For these two scenarios, the current OCR technology can achieve satisfactory recognition accuracy. However, for other scenarios such as curved text or a certain angle offset between the shooting angle and the text direction, i.e. for text of arbitrary angle and arbitrary shape, the existing OCR recognition technology often has misrecognition, missed recognition, or even cannot recognize. SUMMARY

[0004] The present application provides an OCR recognition model training method, an OCR text recognition method and system, which solves the problem that the existing OCR recognition technology cannot recognize text of arbitrary angle and arbitrary shape, and is prone to misrecognition, missed recognition, or even cannot recognize.

[0005] In a first aspect, an OCR recognition method for text of arbitrary angle and arbitrary shape is provided in the embodiments of the present application, which includes the following steps:

[0006] Obtain a plurality of character images, and perform annotation processing and calibration processing on the character images based on a five-tuple representation to form an image training set;

[0007] Construct a character recognition neural network model, and train the character recognition neural network model based on the image training set;

[0008] Predict the position information and category information of the characters in the to-be-recognized character image based on the trained character recognition neural network model, wherein the position information includes coordinate information, shape information and angle information;

[0009] Connect the characters based on the position information and category information of the characters in the to-be-recognized character image to output a text recognition result of arbitrary angle and arbitrary shape.

[0010] In the above embodiments, first, the character image is labeled and calibrated based on the expression manner of the five-tuple to obtain an image training set containing position information and category information, then the neural network is trained based on the image training set to obtain a character recognition neural network model having position information and category information detection and recognition capability; then the position information and position information of several characters are input into the character recognition neural network model; finally, the character connection is performed based on the position information and position information of the several characters, and then the text recognition result of any angle and any shape can be output, that is, the coordinate information, shape information and angle information of the character are combined to realize the text recognition of any angle and any shape.

[0011] As some optional embodiments of the present application, the process of labeling the character image is as follows:

[0012] All characters in the character image are labeled using four marking points with a sequence, and the four marking points are respectively located at the upper left corner point (x1, y1), the upper right corner point (x2, y2), the lower right corner point (x3, y3) and the lower left corner point (x4, y4) of the character to form a quadrilateral, and the category of the character is labeled.

[0013] All characters in each sentence in the character image are labeled using a plurality of line segments with a sequence, and the line segments are respectively located between the center points of adjacent characters.

[0014] In the above embodiments, by using four marking points with a certain sequence to label the characters, the angles of the characters can be accurately distinguished; by using line segments with a certain sequence to label all characters in the sentence, the shapes formed by all characters in the sentence can be distinguished, which is convenient for subsequent model training.

[0015] As some optional embodiments of the present application, the process of calibrating the character image is as follows:

[0016] The average offset (dx, dy) of the four marking points is calculated, and the coordinate calibration of the four marking points is performed based on the average offset (dx, dy) to form a standard rectangular frame.

[0017] The average offset of the four corner point coordinates is expressed as:

[0018]

[0019]

[0020] The coordinates of the standard rectangular frame are expressed as:

[0021]

[0022] The standard rectangular frame coordinate representation is converted into a five-tuple representation, and the five-tuple representation of the standard rectangular frame is: [x, y, w, h, a], x represents the horizontal coordinate of the center point of the standard rectangular frame, y represents the vertical coordinate of the center point of the standard rectangular frame, w represents the length of the standard rectangular frame, h represents the height of the standard rectangular frame, and a represents the angle of the standard rectangular frame.

[0023] The five-tuple representation is:

[0024]

[0025] The conversion angle f α is:

[0026]

[0027] In the above embodiment, in order to avoid the situation that the model is difficult to converge due to the annotation error in model training, the shape formed by the four corner points is converted into a standard rectangular frame, and the coordinate information, shape information and angle information of the standard rectangular frame are represented in the form of a five-tuple, so that the trained character recognition neural network model can recognize text of any angle and any shape.

[0028] As some optional embodiments of the present application, the process of training the character recognition neural network model based on the image training set is as follows:

[0029] The character image after the marking processing and the calibration processing is input into the character recognition neural network model.

[0030] The character recognition neural network model is trained by using the stochastic gradient descent method until convergence.

[0031] As some optional embodiments of the present application, the process of using the trained character recognition neural network model to predict the position information and the category information of the characters in the to-be-recognized character image is as follows:

[0032] The to-be-recognized character image is input into the trained character recognition neural network model.

[0033] The character neural network model outputs the coordinates of a plurality of standard rectangular frames and the categories of the characters to form a bounding box set S.

[0034] As some optional embodiments of the present application, the process of connecting characters based on the coordinate positions and the category information of the characters in the to-be-recognized character image to output the character recognition result is as follows:

[0035] A standard rectangular frame B i is randomly taken from the bounding box set S i , and the standard rectangular frame B jdistance and reading angle;

[0036] select the standard rectangular frame B j with the minimum distance from the several remaining standard rectangular frames B i,min ;

[0037] If the reading angle of the standard rectangular frame B i,min is less than a fixed threshold θ1, the standard rectangular frame B i,min is considered to be a standard rectangular frame B i After the standard rectangular frame B i,min , if the reading angle is greater than a fixed threshold θ2, the standard rectangular frame B i is considered to be a standard rectangular frame B i,min , and the angle adjustment of the standard rectangular frame B i,min is opposite, i.e., α i,min = α i + π.

[0038] If there is a loop in the connected standard rectangular frames, the distance between any two standard rectangular frames in the loop is calculated, and the two standard rectangular frames with the maximum distance are broken.

[0039] In the above embodiment, the position of the standard rectangular frame is determined to determine the position of the character, and then the text recognition result of any angle, any shape or when multiple characters in the image are in an arc shape is output.

[0040] As some optional embodiments of the present application, the distance dist i between the standard rectangular frame B j and the several remaining standard rectangular frames B i,j is calculated according to the following formula:

[0041]

[0042] where x i represents the horizontal coordinate of the center point of the standard rectangular frame B i , x j represents the vertical coordinate of the center point of the standard rectangular frame B i , x j represents the horizontal coordinate of the center point of the standard rectangular frame B j , and y j represents the vertical coordinate of the center point of the standard rectangular frame B j .

[0043] As some optional embodiments of the present application, the reading angle α i between the standard rectangular frame B j and the several remaining standard rectangular frames B i,j is calculated according to the following formula:

[0044] αi,j = |a i -f αi M|

[0045]

[0046] wherein, a i represents an angle of a standard rectangular frame B i , w i represents a length of a standard rectangular frame B i , h i represents a height of a standard rectangular frame B i , x i represents a horizontal coordinate of a center point of a standard rectangular frame B i , y i represents a vertical coordinate of a center point of a standard rectangular frame B j , x j represents a horizontal coordinate of a center point of a standard rectangular frame B j , y j represents a vertical coordinate of a center point of a standard rectangular frame B j , f αi represents a conversion angle of a standard rectangular frame B i .

[0047] In a second aspect, the present application provides an OCR recognition system for arbitrary angle and arbitrary shape text, the system comprising:

[0048] a training data acquisition unit, the training data acquisition unit being configured to acquire a plurality of character images, and perform annotation processing and calibration processing on the character images based on a five-tuple representation form, to obtain an image training set containing position information and category information;

[0049] a model training unit, the model training unit being configured to construct a character recognition neural network model, and train the character recognition neural network model based on the image training set;

[0050] a model inference unit, the model inference unit being configured to acquire a to-be-recognized character image, and predict position information and category information of characters in the to-be-recognized character image based on the trained character recognition neural network model, the position information including coordinate information, structure information and angle information;

[0051] a character connection unit, the character connection unit being configured to perform character connection based on the position information and the category information of the characters in the to-be-recognized character image, to output a recognition result of the arbitrary angle and arbitrary shape text.

[0052] In a third aspect, the present application provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the OCR recognition method for text of any angle and any shape.

[0053] In a fourth aspect, the present application provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the OCR recognition method for text of any angle and any shape.

[0054] The present application has the following advantages: the present application can recognize text in a character image even when the text in the character image is at an angle, upside down, or in an arc shape; and the text recognition is accurate and efficient. BRIEF DESCRIPTION OF DRAWINGS

[0055] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0056] Figure 1 is a flowchart of the OCR recognition method according to the embodiments of the present application;

[0057] Figure 2 is a character labeling diagram according to the embodiments of the present application;

[0058] Figure 3 is a diagram showing that the text in the character image is at an angle, upside down, multi-line, or in an arc shape according to the embodiments of the present application;

[0059] Figure 4 is a system block diagram of the OCR recognition system according to the embodiments of the present application. DETAILED DESCRIPTION

[0060] In order to better understand the above technical solutions, the following will describe the technical solutions of the present application in detail through the drawings and specific embodiments. It should be understood that the specific features of the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solutions of the present application, and are not limitations on the technical solutions of the present application. In the case of no conflict, the technical features of the embodiments of the present application and the embodiments can be combined with each other.

[0061] It is also to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting; for example, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It must be noted that as used herein:

[0062] Embodiment 1

[0063] The present application provides an OCR recognition method for any angle and any shape text, please refer to Figure 1 , the method comprises the following steps:

[0064] (1) Collect and simulate a number of character images, and label and calibrate the character images based on the five-tuple representation to obtain an image training set containing position information and category information.

[0065] In order to train a better character recognition neural network model later, it is necessary to first collect or simulate a number of character images. For collection and simulation, as much as possible, the rules and fonts of characters appearing in subsequent images to be recognized should be included, the fonts include "Song Ti", "Kai Ti", "Hei Ti" and other fonts, the more types of fonts the better; the rules of character appearance are the arrangement rules of character sentences, such as character arc, multi-line sentence and other character arrangement rules, please refer to Figure 3 .

[0066] Since the simulated character images are generated according to a certain method, the position information of the characters on the character images and the belonging sentences can be known, and for the collected character images, character labeling and sentence labeling are needed.

[0067] Specifically, the process of character labeling and sentence labeling of the character images is as follows:

[0068] (1.1) Character labeling is to label all characters in the character image using a certain order of marking points, the marking points have four points, respectively located at the left upper corner point (x1, y1), the right upper corner point (x2, y2), the right lower corner point (x3, y3) and the left lower corner point (x4, y4) of the character, through the four points, the position information of all characters in the image can be recorded, please refer to Figure 2 .

[0069] ​(1.2) Sentence annotation is to mark all characters in each sentence in the character image with a number of line segments with order, the line segments are respectively located between the center points of adjacent characters, that is, for a plurality of characters existing in a sentence, the center point of the first character in the sentence is recorded, the center point of the second character is recorded in sequence, and so on until the last character.

[0070] In the embodiment of the present application, the purpose of annotating characters and sentences in the image is to be able to train a character recognition neural network model subsequently, and the annotated data still needs to be further calibrated before training; due to annotation errors, the shape formed by the four corner points is not a rectangle, in order to avoid the situation that the model is difficult to converge due to annotation errors in model training, calibration processing is needed.

[0071] Specifically, the calibration process of the character image is as follows:

[0072] (1.3) Calculate the average offset (dx, dy) of the four marking points, and perform coordinate calibration on the four marking points based on the average offset (dx, dy) to form a standard rectangular frame;

[0073] Wherein, the average offset of the four corner point coordinates is represented as:

[0074]

[0075] Wherein, the coordinates of the standard rectangular frame are represented as:

[0076]

[0077] (1.4) Convert the standard rectangular frame coordinate representation into a five-tuple representation, the five-tuple representation of the standard rectangular frame is: [x, y, w, h, a], x represents the horizontal coordinate of the center point of the standard rectangular frame, y represents the vertical coordinate of the center point of the standard rectangular frame, w represents the length of the standard rectangular frame, h represents the height of the standard rectangular frame, and a represents the angle of the standard rectangular frame;

[0078] Wherein, the five-tuple representation is:

[0079]

[0080] Wherein, the conversion angle f α is:

[0081]

[0082] It should be clear that for single character annotation, the annotation information also contains the class information of the character, and the class information refers to numbers, letters, punctuation marks, etc., such as numbers containing 0 to 9, a total of 10 classes.

[0083] (2) constructing a character recognition neural network model, and training the character recognition neural network model based on an image training set;

[0084] Specifically, the character recognition neural network model comprises a feature extraction part, a feature pyramid part, a region proposal part, a rotated region of interest extraction part and a rotated bounding box head part.

[0085] (2.1) the feature extraction part

[0086] The feature extraction part is a neural network module constructed by multiple convolution layers, normalization layers and activation layers, which receives a character image of any resolution size as input.

[0087] It is assumed that the size resolution of the input character image is [H, W, C], wherein H is the width of the character image, W is the length of the character image, and C is the channel number of the character image. The feature extraction part outputs four feature maps of different resolution sizes, which are respectively named as a first feature map, a second feature map, a third feature map and a fourth feature map, and the resolution sizes of the feature maps are respectively: and wherein C1, C2, C3 and C4 represent different channel numbers.

[0088] In the embodiment of the application, if the channel number C1 is 256, C2 is 512, C3 is 1024 and C4 is 2048, the resolution sizes of the feature maps output by the feature extraction part are respectively [256, 256, 256], [128, 128, 512], [64, 64, 1024] and [32, 32, 2048]. That is, the feature extraction part can automatically extract features in the image from the character image by means of the powerful learning ability of the neural network, thereby outputting feature maps of different scales.

[0089] (2.2) the feature pyramid part

[0090] The feature pyramid part is a neural network model constructed by multiple convolution layers and up-sampling layers, which receives the first feature map, the second feature map, the third feature map and the fourth feature map as input, and outputs five feature maps of different resolution sizes, i.e. a fifth feature map, a sixth feature map, a seventh feature map, an eighth feature map and a ninth feature map, and the resolution sizes of the feature maps are respectively and wherein C5 represents a channel number.

[0091] Specifically, in the embodiment of the present application, if the channel number C5 is 256, the resolutions of the feature maps output by the feature pyramid part are [256, 256, 256], [128, 128, 256], [64, 64, 256], [32, 32, 256] and [16, 16, 256] respectively. That is, the semantic feature information in feature maps of different sizes can be fused by means of the powerful learning ability of the neural network, so as to improve the information expression ability of the features and further improve the effect of the overall model.

[0092] (2.3) Region Proposal Part

[0093] The region proposal part includes two sub-modules, an anchor box generation sub-module and an anchor box coordinate prediction sub-module.

[0094] The anchor box generation sub-module is a neural network module composed of multiple convolutional layers, which generates a fixed number of anchor boxes according to the resolution sizes of the fifth to ninth feature maps. The anchor boxes generated by this part are anchor boxes with initial position coordinates, and the anchor boxes are represented in a 5-tuple [cx, cy, mw, mh, mα], which respectively represent the horizontal coordinate, the vertical coordinate, the length, the height and the angle of the anchor. For example, nine anchor boxes with three different angles and three different aspect ratios can be generated for each pixel on the feature map as the center point, and the coordinates of the anchor boxes need to be mapped back to the original image to obtain the anchor boxes in the original coordinate system. Each anchor box with a different aspect ratio is assigned a different angle, i.e. the angles of each three anchor boxes with different aspect ratios are 0 degrees, 60 degrees and 120 degrees respectively. The number of anchor boxes generated by each layer of feature map is related to the resolution size thereof, for example, the fifth layer of feature map generates anchor boxes, and in the embodiment of the present application, the fifth layer of feature Figure 1 generates 589824 anchor boxes.

[0095] The anchor box coordinate prediction sub-module is a neural network module composed of multiple convolutional layers, which receives the fifth feature map, the sixth feature map, the seventh feature map, the eighth feature map and the ninth feature map as input, and outputs the classification confidence and the anchor box coordinate offset corresponding to the anchor boxes; for example, for the fifth feature map, the output shapes of the feature maps are and

[0096] ​Specifically, in the embodiment of the present application, for the input fifth feature map, the output feature map shapes are [589824, 2] and [589824, 5] respectively. Wherein 2 represents the 2 classification probability of whether each anchor frame is a character predicted by the sub-module, and 5 represents the offset of the 5-tuple information in each anchor frame, that is, the horizontal and vertical coordinate offset dcx, dcy of the corresponding anchor frame center point, the length and width offset dmw, dmh and the angle offset dmα. In order to make the model easy to converge, the angle range of the anchor frame is [0, 180], that is, the minimum included angle is always set to the anchor frame angle; the anchor frame coordinate prediction sub-module will output the N1 anchor frames with the largest probability value and the anchor frame character probability greater than a fixed threshold.

[0097] For the model training stage, when the area intersection over union (IOU) of the anchor frame and the standard rectangular frame is greater than a threshold, the offset calculation is performed, and the offset of the anchor frame five-tuple information [cx, cy, mw, mh, mα] based on the standard rectangular frame [x, y, w, h, α] is:

[0098]

[0099]

[0100]

[0101]

[0102]

[0103] For the model inference stage, through the offset predicted by the model and the anchor frame five-tuple information [cx, cy, mw, mh, mα], the character bounding box information can be obtained as:

[0104] cx = cx + mw * dcx

[0105] cy = cy + mh * dcy

[0106] mw = mw * e dmw

[0107] mh = mh * e dmh

[0108]

[0109] Wherein, mod(x, y) represents the remainder of x divided by y.

[0110] (2.4) Rotating region of interest extraction part

[0111] The rotation region of interest extraction part is a neural network module composed of multiple convolution layers, which receives the output of the previous step as input, i.e., N1 anchor boxes and coordinate information, and outputs N1 cropped region feature maps. For each anchor box in the N1 anchor boxes, according to its coordinate information and the feature map to which it belongs, the corresponding feature map region is rotated and cropped, and a fixed size feature map is generated by scaling. A total of N1 fixed size feature maps are output.

[0112] In an embodiment of the present application, N1 is 2000, and the fixed size is [7, 7], i.e., this step will fix the output of 2000 feature maps with a size of [7, 7].

[0113] (2.5) Rotated bounding box head part

[0114] The rotated bounding box head module is a neural network module composed of multiple convolution layers, which receives N1 fixed size feature maps as input and outputs feature maps with shapes [N1, NC+1] and [N1, 5], where NC represents the category of characters. For example, if you want to recognize numbers 0-9, NC categories are ten, and 1 represents the background category. Similar to the anchor box coordinate prediction submodule, the method of predicting anchor box center point horizontal and vertical coordinate offsets dx, dy, length and width offsets dw, dh is the same. It should be noted that the angle range of the anchor box at this time is [0, 360].

[0115] For the training phase, when the angle rotation sensitive area intersection over union (RIOU) of the anchor box and the standard rectangular box is greater than a certain threshold, the offset calculation is performed, where RIOU is:

[0116]

[0117] The angle offset should be:

[0118]

[0119] For the model inference phase, the angle should be:

[0120] mα=mod(mα+dα,2π)

[0121] In an embodiment of the present application, training the character recognition neural network model based on the image training set is to input the character image after label processing and calibration processing into the character recognition neural network model, and to train the character recognition neural network model to convergence by using the stochastic gradient descent method.

[0122] (3) Obtain the character image to be recognized, and predict the position information and category information of the character in the character image to be recognized based on the trained character recognition neural network model; that is, input the character image to be recognized into the trained character recognition neural network model; output the coordinates of a plurality of standard rectangular frames and the category of the character through the character neural network model to form a frame set S.

[0123] (4) Perform character connection based on the position information and category information of the character in the character image to be recognized to output a text recognition result of any angle and any shape;

[0124] Since the frame obtained by model inference contains coordinate position information and category information, and the arrangement and combination order of characters is not known, a character connection method is needed to arrange and combine all characters in a certain way to obtain the reading method of the characters.

[0125] Specifically, the character connection method is as follows:

[0126] (4.1) For a standard rectangular frame B i ∈S,i=1,2,…N, wherein N is the number of predicted standard rectangular frames, B i =[x i ,y i ,w i ,h i ,α i ,C i ]; define the categories of numbers “0”, “6”, “8”, “9” and the like as a reversible set S C , reversibility means that when a character is rotated by 180°, the reversed character also belongs to another legal category, for example, “6” is reversed to “9”, “0” is also reversed to “0”, but “7” is not reversed to any category.

[0127] (4.2) Traverse and take a non-reversed standard rectangular frame B i from the frame set S, and find its subsequent standard rectangular frame according to the following steps. The non-reversed frame is defined as C i ∈S\S C .

[0128] (4.21) Traverse and take a standard rectangular frame B j from the frame set S, wherein i≠j.

[0129] ① Calculate the distance between the standard rectangular frame B i and B j :

[0130]

[0131] ② Calculate the distance between the standard rectangular frame Bi with B j :

[0132] α i,j = |α i -f αi M

[0133]

[0134] (4.22) Calculate B i the minimum distance standard rectangular frame, that is

[0135] B i,min = argmindist i,j

[0136] (4.23) If the reading angle of the standard rectangular frame B i,min is less than a fixed threshold θ1, it is considered that the standard rectangular frame B i,min After the standard rectangular frame B i , if the reading angle is greater than a fixed threshold θ2, and C i,min ∈ S C , it is considered that the standard rectangular frame B i,min After the standard rectangular frame B i , and the angle adjustment of the standard rectangular frame B i,min is the opposite angle, that is, α i,min = α i,min + π, in the embodiment of the present application, θ1 is 90°, and θ2 is 270°.

[0137] (4.3) Traverse and take out the reversible frame B' from the frame set S i , repeat steps (4.22) to (4.23).

[0138] (4.4) Determine whether all connected standard rectangular frames form a ring, if there is a ring frame in the connected standard rectangular frames, calculate the distance of any two standard rectangular frames in the ring, and break the ring by taking the two standard rectangular frames with the maximum distance, that is, the two standard rectangular frames with the maximum distance are not considered to constitute a front and rear dependent relationship.

[0139] Embodiment 2

[0140] The present application provides an OCR recognition system for any angle and any shape text, please refer to Figure 4 , the system and the method of embodiment 1 one by one, the system comprises:

[0141] A training data acquisition unit, the training data acquisition unit is used for acquiring a plurality of character images, and based on the representation form of five tuples, the character images are labeled and calibrated to obtain an image training set containing position information and category information;

[0142] a model training unit configured to construct a character recognition neural network model and train the character recognition neural network model based on an image training set;

[0143] a model inference unit configured to predict position information and category information of characters in a to-be-recognized character image based on the character recognition neural network model after training, the position information including coordinate information, shape information and angle information;

[0144] a character connection unit configured to perform character connection based on the position information and the category information of the characters in the to-be-recognized character image to output a text recognition result in an arbitrary angle and an arbitrary shape.

[0145] Embodiment 3

[0146] The present application provides a computer device comprising a memory and a processor, the memory storing a computer program, the computer program being executed on the processor to implement the OCR recognition method for the text in an arbitrary angle and an arbitrary shape according to the embodiment 1.

[0147] The computer device provided in the present embodiment can implement the method according to the embodiment 1, and thus the details are not repeated here.

[0148] Embodiment 4

[0149] The present application provides a computer readable storage medium, the computer readable storage medium storing a computer program, the computer program being executed on the processor to implement the OCR recognition method for the text in an arbitrary angle and an arbitrary shape according to the embodiment 1.

[0150] The computer readable storage medium provided in the present embodiment can implement the method according to the embodiment 1, and thus the details are not repeated here.

[0151] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0152] The memory can be used to store the computer program and / or modules, and the processor realizes various functions of the OCR recognition system for text of any angle and any shape by running or executing the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), and the like. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, for example, a hard disk, a memory, a plug-in hard disk, an intelligent memory card, a secure digital card, a flash memory card, at least one disk storage device, a flash memory device, or other volatile solid-state memory devices.

[0153] The OCR recognition system for text of any angle and any shape can be stored in a computer readable storage medium if it is realized in the form of a software function unit and sold or used as an independent product. Based on this understanding, all or part of the processes in the above-mentioned embodiment methods can be realized by a computer program which can be stored in a computer readable storage medium and can realize the steps of each method embodiment when executed by a processor. The computer program includes computer program code, object code form, executable files, or some intermediate forms, etc. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, point carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the contents contained in the computer readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0154] The basic concept has been described in the present application, and it is obvious that the above detailed disclosure is only used as an example and does not constitute a limitation on the present specification. Although it is not explicitly stated here, those skilled in the art can make various modifications, improvements and corrections to the present specification. Such modifications, improvements and corrections are suggested in the present specification, so such modifications, improvements and corrections still belong to the spirit and scope of the exemplary embodiments of the present specification.

Claims

1. An OCR recognition method for text of arbitrary angle and shape, characterized in that, The method includes the following steps: Acquire several character images and perform annotation and calibration processing on the character images based on the representation of quintuples to obtain an image training set containing positional and category information; Construct a character recognition neural network model and train the character recognition neural network model based on an image training set; Based on the trained character recognition neural network model, the position information and category information of the characters in the image to be recognized are predicted. The position information includes coordinate information, shape information and angle information. The process of using a trained character recognition neural network model to predict the position and category information of characters in an image to be recognized is as follows: Input the image of the character to be recognized into the trained character recognition neural network model; The character neural network model outputs the coordinates of several standard rectangular boxes and the character category to form a set of bounding boxes. ; Based on the positional and category information of the characters in the image to be recognized, characters are connected to output text recognition results of arbitrary angles and shapes; The process of connecting characters based on their coordinate positions and category information in the image to be recognized is as follows: From the border set Take any standard rectangle And calculate the standard rectangular frame in sequence. With several remaining standard rectangular frames Distance and reading angle ; From several remaining standard rectangular boxes Select the standard rectangle with the smallest distance. ; If a standard rectangular frame The reading angle is less than a fixed threshold Then the standard rectangular frame is determined. In a standard rectangular frame Then, if the reading angle is greater than a fixed threshold Then the standard rectangular frame is determined. In a standard rectangular frame After that, and the standard rectangular frame The angle is adjusted to the opposite angle, that is... ; Determine whether any of the connected standard rectangles form a loop. If a loop exists within the connected standard rectangles, calculate the distance between any two standard rectangles forming the loop, and break the loop by selecting the two standard rectangles with the largest distance.

2. The OCR recognition method for text of arbitrary angle and shape according to claim 1, characterized in that, The process of annotating character images is as follows: All characters in the character image are marked with four sequential markers, each located at the top left corner of a character. Top right corner bottom right corner , lower left corner point To form quadrilaterals and to label the categories of the characters; All characters in each statement in the character image are marked with a series of line segments arranged in a specific order, with each line segment located between the center points of adjacent characters.

3. The OCR recognition method for text of arbitrary angle and shape according to claim 2, characterized in that, The process for calibrating character images is as follows: Calculate the average offset of the four marker points And based on the average offset The coordinates of the four marker points are calibrated to form a standard rectangular frame; The average offset of the four corner coordinates is expressed as: ； The coordinates of the standard rectangle are represented as follows: ； The standard rectangular bounding box coordinate representation is converted into a 5-tuple representation, and the 5-tuple representation of the standard rectangular bounding box is as follows: , This represents the x-coordinate of the center point of the standard rectangle. This represents the ordinate of the center point of a standard rectangular frame. Indicates the length of a standard rectangle. Indicates the height of a standard rectangle. Indicates the angle of a standard rectangle; The quintuple is represented as: ； Among them, the angle of conversion for: 。 4. The OCR recognition method for text of arbitrary angle and shape according to claim 1, characterized in that, The process of training a character recognition neural network model based on an image training set is as follows: The character images after labeling and calibration are input into the character recognition neural network model; The character recognition neural network model was trained until convergence using stochastic gradient descent.

5. The OCR recognition method for text of arbitrary angle and shape according to claim 1, characterized in that, Standard rectangular frame With several remaining standard rectangular frames distance The calculation formula is: ； in, Represents a standard rectangular box The x-coordinate of the center point, Represents a standard rectangular box The ordinate of the center point, Represents a standard rectangular box The x-coordinate of the center point, Represents a standard rectangular box The ordinate of the center point.

6. The OCR recognition method for text of arbitrary angle and shape according to claim 1, characterized in that, Standard rectangular frame With several remaining standard rectangular frames reading angle The calculation formula is: ； in, Represents a standard rectangular box Angle, Represents a standard rectangular box Length, Represents a standard rectangular box high, Represents a standard rectangular box The x-coordinate of the center point, Represents a standard rectangular box The ordinate of the center point, Represents a standard rectangular box The x-coordinate of the center point, Represents a standard rectangular box The ordinate of the center point represents the standard rectangular frame. The angle of change.

7. An OCR recognition system for text of arbitrary angle and shape, the system comprising: The training data acquisition unit is used to acquire several character images and perform annotation and calibration processing on the character images based on the representation of quintuples to obtain an image training set containing position information and category information. A model training unit is used to construct a character recognition neural network model and train the character recognition neural network model based on an image training set. The model inference unit predicts the position information and category information of characters in the character image to be recognized based on the trained character recognition neural network model. The position information includes coordinate information, shape information and angle information. The process of using a trained character recognition neural network model to predict the position and category information of characters in an image to be recognized is as follows: Input the image of the character to be recognized into the trained character recognition neural network model; The character neural network model outputs the coordinates of several standard rectangular boxes and the character category to form a set of bounding boxes. ; A character connection unit connects characters based on their position and category information in the image of the character to be recognized, so as to output text recognition results of any angle and any shape. The process of connecting characters based on their coordinate positions and category information in the image to be recognized is as follows: From the border set Take any standard rectangle And calculate the standard rectangular frame in sequence. With several remaining standard rectangular frames Distance and reading angle ; From several remaining standard rectangular boxes Select the standard rectangle with the smallest distance. ; If a standard rectangular frame The reading angle is less than a fixed threshold Then the standard rectangular frame is determined. In a standard rectangular frame Then, if the reading angle is greater than a fixed threshold Then the standard rectangular frame is determined. In a standard rectangular frame After that, and the standard rectangular frame The angle is adjusted to the opposite angle, that is... ; Determine whether any of the connected standard rectangles form a loop. If a loop exists within the connected standard rectangles, calculate the distance between any two standard rectangles forming the loop, and break the loop by selecting the two standard rectangles with the largest distance.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements an OCR recognition method for text of any angle and shape as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the OCR recognition method for text of any one of claims 1-6.

Images