OCR (Optical Character Recognition) image-text recognition and statistical method for data collector

Abstract

The invention relates to the technical field of image-text recognition, in particular to an OCR (Optical Character Recognition) image-text recognition and statistical method for a data acquirer, which comprises the following steps of: acquiring an image-text image through the data acquirer to obtain an image-text grey-scale map, and acquiring each character detection frame; extracting each trend direction and a trend direction saliency value thereof; determining the dense ink stroke direction of each growth region according to the trend of the dense ink stroke, constructing the horizontal and vertical motion blur direction confidence of the image-text grey-scale map, and estimating the motion blur direction of the image-text grey-scale map; further constructing a motion blur characteristic value of each angular point relative to each gradient direction so as to screen key points, constructing a characteristic descriptor of the key points in the motion blur direction, performing image-text recognition by adopting a key point matching algorithm, and counting the number of characters. The image-text recognition precision and efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of image and text recognition technology, and in particular to an OCR image and text recognition and statistical method for a data collector. Background Art

[0002] Data collectors are widely used in logistics, commerce, inventory management and other fields. They can not only scan and identify barcodes and QR codes, but also perform image and text recognition and character count when encountering characters, dates and text that are not barcoded. This can save labor, avoid manual entry errors, and achieve rapid recognition and reading of image and text information.

[0003] The OCR recognition method based on deep learning automatically extracts the deep features of images and texts through a neural network model. It has wide applicability to complex scenarios and has high recognition accuracy. However, the implementation of OCR recognition requires the collection of a large number of image and text data samples, and it is highly dependent on the data collector hardware, resulting in high technical implementation costs.

[0004] The SIFT (Scale-Invariant Feature Transform) algorithm extracts key points and feature descriptors of text features in images and texts, and implements OCR image and text recognition through feature matching. However, the use scenarios of data collectors are complex, and the collected images are prone to motion blur. The SIFT algorithm detects too few key points, resulting in a high error rate in image and text recognition. In addition, the SIFT algorithm generates 128-dimensional feature descriptors for feature key points, which has high computational complexity and cannot meet the real-time performance of OCR image and text recognition in data collectors. Summary of the invention

[0005] In order to solve the above technical problems, the present application provides an OCR image and text recognition and statistical method for a data collector to solve the existing problems.

[0006] The OCR image and text recognition and statistical method for data collector of this application adopts the following technical solutions:

[0007] An embodiment of the present application provides an OCR image and text recognition and statistics method for a data collector, comprising the following steps:

[0008] The image is collected by a data collector to obtain a grayscale image of the image, and the image and text grayscale image is segmented into characters to obtain each character detection frame;

[0009] Extract edge lines and corner points in the grayscale image of the text and image, analyze the distribution of edge lines and corner points in each text detection frame, and obtain each trend direction and its trend direction significance value;

[0010] Threshold segmentation is performed on the grayscale image to obtain thick ink pixels, and thick ink strokes are extracted by region growing algorithm. The direction of thick ink strokes in each growing region is determined according to the direction of the thick ink strokes.

[0011] According to the difference in the number of growing regions in the horizontal and vertical directions of thick ink strokes, the confidence of the horizontal and vertical motion blur directions of the image and text grayscale image is constructed, and the motion blur direction of the image and text grayscale image is estimated by combining the number of growing regions in each thick ink stroke direction;

[0012] According to the motion blur direction estimation result of the image grayscale image, and combined with the gradient direction, gradient amplitude and path information of the dark ink pixel in the motion blur direction of each corner point, the motion blur feature value of each corner point with respect to each gradient direction is constructed;

[0013] According to the motion blur feature value and the distribution of dark ink pixels in the neighborhood of each corner point, the corner points are screened to obtain the key points for feature matching of each text detection frame, and combined with the motion blur direction and the number of dark ink pixels in the neighborhood of the key point, the feature descriptor of the key point in the motion blur direction is constructed, and the key point matching algorithm is used to automatically recognize the OCR images and texts of the data collector, and the number of characters is counted through the number of text detection frames.

[0014] Preferably, the acquisition of each trend direction further includes:

[0015] Count the ratio of the number of edge pixels in each text detection frame to the total number of pixels in the text detection frame, and arrange the text detection frames in ascending order according to the ratio, and obtain a preset number of text detection frames, which are recorded as the detection frames to be analyzed;

[0016] The edge line in the detection frame to be analyzed is segmented by the corner points in the detection frame to be analyzed to obtain edge line segments, and each edge line segment is fitted with a straight line, and the angle between each fitted straight line and the horizontal direction is calculated as each trend direction.

[0017] Preferably, the method for acquiring the trend direction significance value of each trend direction is: counting the number of fitted straight lines under the same trend direction and using the number as the trend direction significance value of the same trend direction.

[0018] Preferably, the acquisition of the thick ink pixel further comprises:

[0019] The first threshold segmentation is performed on the grayscale image of the text and image, and the first segmentation threshold is output. The pixels with grayscale values ​​lower than the first segmentation threshold are regarded as stroke pixels. The grayscale values ​​of all stroke pixels are subjected to a second threshold segmentation, and the second segmentation threshold is output. The pixels with grayscale values ​​lower than the second segmentation threshold are regarded as thick ink pixels.

[0020] Preferably, the method for determining the direction of the thick ink strokes is: extracting the skeleton of each growth region, performing straight line fitting on the skeleton, and determining the angle between the fitting straight line and the horizontal direction as the direction of the thick ink strokes of each growth region.

[0021] Preferably, the expression of the confidence of the horizontal and vertical motion blur direction of the image and text grayscale image is:

[0022] Where T is the confidence of the horizontal and vertical motion blur direction of the grayscale image, μ1 is the number of growth areas with a thick ink stroke direction of 90°, μ2 is the number of growth areas with a thick ink stroke direction of 180°, max() is the maximum value function, and M is the number of growth areas.

[0023] Preferably, estimating the motion blur direction of the image-text grayscale image further comprises:

[0024] When the confidence of the horizontal and vertical motion blur directions is greater than or equal to the preset confidence threshold, the direction containing the largest number of growth areas is selected from the horizontal and vertical directions as the motion blur direction of the image grayscale image;

[0025] When the confidence of the horizontal and vertical motion blur direction is less than the preset confidence threshold, the motion blur direction confidence values ​​of all trend directions except the horizontal and vertical directions are calculated, and the trend direction corresponding to the maximum motion blur direction confidence value is used as the motion blur direction of the text grayscale image;

[0026] The expression of the motion blur direction confidence value is: In the formula, R θ 、n θ They are respectively the motion blur direction confidence value and trend direction significance value of the trend direction θ. It is the motion blur direction weight when the thick ink stroke direction is θ, wherein the proportion of the number of growth areas in each thick ink stroke direction in the total number of growth areas in all thick ink stroke directions is used as the motion blur direction weight of each thick ink stroke direction.

[0027] Preferably, the expression of the motion blur characteristic value of each corner point with respect to each gradient direction is:

[0028] In the formula, v q,x is the motion blur eigenvalue of the corner point q about the gradient direction x, φ is the motion blur direction of the grayscale image, Cos() is the cosine value, J is the number of gradient directions, and h q,x is the gradient magnitude of the corner point q in the gradient direction x, h q,j is the gradient magnitude of corner point q in gradient direction j.

[0029] Preferably, the screening of key points includes:

[0030] Obtain a straight line in the preset neighborhood of each corner point in the direction of motion blur and passing through the corner point, and count the number of dark ink pixels that the straight line passing through the corner point passes through in the preset neighborhood of the corner point as the dark ink pixel length of the corner point;

[0031] The product of the maximum value of the motion blur eigenvalue of each corner point and the length of the thick ink pixel is used as the screening robust strength of each corner point. The screening robust strength of all corner points in the grayscale image of the image is arranged from large to small according to the screening robust strength, and a preset number of corner points are selected as key points.

[0032] Preferably, the construction of the feature descriptor of the key point in the motion blur direction includes:

[0033] The motion blur direction of the grayscale image, the blur representation direction of each key point, the motion blur feature value and the length of thick ink pixel are normalized respectively, and the vector composed of the normalized result values ​​is used as the feature descriptor of each key point in the motion blur direction.

[0034] This application has at least the following beneficial effects:

[0035] The present application estimates the motion blur direction of the image and text grayscale image based on the edge and ink pixel growth area and multi-scale analysis. The beneficial effect is that the printed text is horizontal and vertical. In addition to the edge lines with the same trend direction and motion blur direction, there are also a large number of horizontal and vertical edge lines. The influence of the horizontal and vertical edge lines on the motion blur direction estimation is eliminated, thereby improving the accuracy of the motion blur direction estimation.

[0036] At the same time, the present application screens the corner points that meet the motion blur direction characteristics based on the motion blur direction estimation results of the image grayscale image, and based on the gradient characteristics and stroke trend characteristics in the motion blur direction within the corner point neighborhood, to achieve automatic OCR image and text recognition and character quantity statistics of the data collector. Its beneficial effect is to expand the key points of the SIFI algorithm, reduce the false detection rate, and at the same time mine the key points with significant features in the motion blur direction. It has high applicability to the motion blur scenes of the data collector, and the length of the feature descriptor is reduced, which improves the speed of the SIFI algorithm in key point matching. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0038] Figure 1 A flowchart of the steps of the OCR image and text recognition and statistical method for the data collector provided in this application. DETAILED DESCRIPTION

[0039] In order to further explain the technical means and effects adopted by the present application to achieve the predetermined invention purpose, the following is a detailed description of the OCR image recognition and statistical method for data collectors proposed in the present application, its specific implementation method, structure, features and effects, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0040] Unless otherwise defined, terms such as "comprises", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a circuit structure, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such articles or devices. In the absence of further restrictions, an element defined by the statement "comprising a ..." does not exclude the presence of other identical elements in the article or device comprising the element. In addition, the term "and\or" used herein includes any and all combinations of one or more related listed items. All technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present application belongs.

[0041] The specific scheme of the OCR image and text recognition and statistical method for a data collector provided by the present application is described in detail below with reference to the accompanying drawings.

[0042] An embodiment of the present application provides an OCR image and text recognition and statistics method for a data collector. For details, please refer to Figure 1 , including the following steps:

[0043] Step 1: Collect text and image data through a data collector to obtain a text and image grayscale image, and perform text and image character segmentation on the text and image grayscale image to obtain each text detection frame.

[0044] This embodiment uses a data collector to acquire images, in which the rear high-definition camera of the data collector can be used to acquire images of the text to be detected, or the wireless communication method of the data collector can be used to upload images. The image acquired by the data collector is subjected to bilateral filtering to eliminate the interference of noise on the image, improve the accuracy of OCR image and text recognition, and convert the grayscale of the denoised image into a grayscale image of the image and text.

[0045] Data collectors are widely used in logistics, commerce, inventory management and other fields. They usually perform OCR automatic text recognition on item labels. The text on item labels belongs to the category of printed characters with black text on a white background, and has the characteristics of high standardization and neat arrangement. Therefore, this embodiment first performs rotation correction on the grayscale image of the image, and then uses the MSER (Maximally Stable Extremal Regions) algorithm to achieve preliminary detection frame calibration for the grayscale image of the image. The preliminary detection frame calibration is specifically implemented using cv2.MSER in the Python language. The rotation correction and MSER algorithm are both well-known technologies, and the specific process will not be repeated.

[0046] Due to the different forms of text, the detection boxes obtained by the MSER algorithm are also irregular, and there are many overlaps in the detection boxes. Some text detection boxes may only circle the radicals or strokes of Chinese characters. Therefore, this embodiment uses cv2.boundingRect in the Python language to obtain the minimum bounding rectangle of each text detection box in the text grayscale image, and then uses the NMS (Non Maximum Suppression) algorithm to suppress the boxes that are not of the maximum size and remove the repeated areas. The overlapping area (IoU) in this embodiment is 0.3, and the text detection box of each text in the text grayscale image is output. Each text detection box includes a printed character in black and white, so as to achieve text character segmentation.

[0047] Step 2: Extract edge lines and corner points in the grayscale image of the text and image, analyze the distribution of edge lines and corner points in each text detection frame, and obtain each trend direction and its trend direction significance value.

[0048] In the process of acquiring images with the data collector, due to the complex use scenarios of the data collector, it is easy to have factors such as hand-held shaking and object movement during shooting, which will cause motion blur in the acquired images, resulting in unclear outlines of graphic characters and partial loss of details. The motion blur direction of the graphic grayscale image reflects the relative movement direction of the object. The strokes of printed fonts are regular and the direction is clear. The traditional motion blur direction estimation method has poor applicability. Motion blur can cause ghosting of clear graphic strokes, and the background of graphic characters is usually white label paper. These ghost strokes have a high contrast with the white label paper, which easily forms edge lines with the same trend direction and motion blur direction.

[0049] This embodiment uses the image grayscale image as the input of the Canny edge detection algorithm and outputs all the edge lines of the image grayscale image. As the more edge lines there are in a graphic character, the more complex the structural features of the text strokes are, and the strokes between different trend directions are more likely to interfere with each other, disturbing the subsequent estimation of the motion blur direction. In this embodiment, the ratio of the number of edge line pixels in each text detection frame in the image grayscale image to the total number of pixels in the text detection frame is counted, and the text detection frames are arranged in ascending order according to the ratio, and the text detection frames with a preset number of text detection frames are obtained, which are recorded as the detection frames to be analyzed. The preset number of text detection frames in this embodiment is 10. In actual application scenarios, the implementer can select them by himself, and no special limitation is made in this embodiment.

[0050] Since the edge line may include the inflection point and the folding point of the stroke, this embodiment uses the Harris corner point detection algorithm to obtain all the corner points in the grayscale image of the image and text. The edge line in the detection frame to be analyzed is segmented by the corner points in the detection frame to be analyzed to obtain each edge line segment, and each edge line segment is fitted with a straight line, and the angle value between each straight line after fitting and the horizontal direction is calculated as each trend direction. This embodiment can perform straight line fitting by the least squares method. Furthermore, for each trend direction, the number of straight lines under the same trend direction is counted, and used as the trend direction significance value of the corresponding trend direction.

[0051] Step 3: Perform threshold segmentation on the grayscale image to obtain thick ink pixels, extract thick ink strokes through region growing algorithm, and determine the direction of thick ink strokes in each growing region according to the direction of the thick ink strokes.

[0052] However, in the grayscale image, the printed text is highly standardized, horizontal and vertical in shape, and in addition to the edge lines that are consistent with the direction of movement and motion blur, there are also a large number of horizontal (motion blur direction is 180°) and vertical (motion blur direction is 90°) edge lines. In the grayscale image, the stroke part that is consistent with the motion blur direction does not have too much ghosting, the printed ink is thick, and the grayscale value is low, while the ghosting part of the text stroke is blurred and the grayscale value is slightly higher.

[0053] In this embodiment, the grayscale image of the text is used as the input of the first Ostu algorithm, and the first segmentation threshold is output. The pixels with grayscale values ​​higher than the first segmentation threshold are white background pixels, and the pixels with grayscale values ​​lower than the first segmentation threshold are stroke pixels. The grayscale values ​​of all pixels lower than the first segmentation threshold are used as the input of the second Ostu algorithm, and the second segmentation threshold is output. The pixels with grayscale values ​​higher than the second segmentation threshold are ghost pixels, and the pixels with grayscale values ​​lower than the second segmentation threshold are thick ink pixels.

[0054] Furthermore, in this embodiment, strokes in which the printed ink is still relatively thick after motion blur are extracted and recorded as thick ink strokes. Thick ink pixels are pixels on thick ink strokes, and adjacent thick ink pixels together constitute thick ink strokes. In this embodiment, the specific acquisition process of thick ink strokes is: obtain each text detection frame containing the detection frame to be analyzed, and use region growing to obtain each growth region to represent the thick ink strokes in the text grayscale image. The specific region growing process in this embodiment is:

[0055] S1: In each text detection box containing the detection box to be analyzed, randomly select a thick ink pixel point as the initial center of region growth;

[0056] S2: Using the region growing algorithm, if there are dark ink pixels in the initial center eight neighborhoods, merge and grow them, and finally get the growing region;

[0057] S3: In each detection frame to be analyzed, a thick ink pixel outside the growing area is reselected as the initial center of the region growing, and step S2 is repeated to continue the region growing to obtain a new round of growing area;

[0058] S4: Repeat the above steps until all the thick ink pixels in the to-be-analyzed detection frame are within the growing region, or when the number of growing regions is equal to M, the region growing is also stopped, where M in this embodiment is 15. It should be noted that the region growing algorithm is a prior art, and the specific growth process is not described in detail in this embodiment.

[0059] Furthermore, the direction of the thick ink strokes is obtained. Specifically, in this embodiment, the growth area is used as the input of the Zhang-Suen backbone extraction algorithm. Since the growth area is approximately a connected domain of thick ink strokes, the skeleton of the growth area is output, and a straight line fitting is performed on the skeleton. The angle between the fitting line and the horizontal direction is used as the direction of the thick ink strokes of the growth area.

[0060] Step 4: According to the difference in the number of growth areas in the horizontal and vertical directions of the thick ink strokes, the confidence of the horizontal and vertical motion blur direction of the grayscale image is constructed, and the motion blur direction of the grayscale image is estimated by combining the number of growth areas in each thick ink stroke direction.

[0061] Because the stroke parts in the grayscale image that are consistent with the motion blur direction belong to dark ink pixels only after motion blur, and there are more horizontal and vertical strokes in the grayscale image, if the motion blur direction is 90° or 180°, there should also be a large number of growth areas with dark ink stroke directions of 90° or 180°.

[0062] Therefore, according to the above analysis, the confidence T of the horizontal and vertical motion blur direction of the image and text grayscale image is constructed. In this embodiment, the calculation formula is:

[0063] Where, μ1 is the number of growth regions with a thick ink stroke direction of 90°, μ2 is the number of growth regions with a thick ink stroke direction of 180°, max() is the maximum value function, and M is the number of growth regions. If the motion blur direction of the text grayscale image is 90° or 180°, the text strokes in the other direction will cause ghosting. When |μ1-μ2| is larger, it means that the direction of the thick ink stroke in the growth region is only biased towards one of 90° or 180°. At the same time, when max(μ1,μ2) is larger, it means that the direction of the thick ink stroke of 90° or 180° is the most complete, and the motion blur direction of the text grayscale image is more likely to be horizontal or vertical.

[0064] This embodiment calculates the proportion of the number of growth areas in each dark ink stroke direction in the total number of growth areas in all dark ink stroke directions as the motion blur direction weight of each dark ink stroke direction. The larger the motion blur direction weight, the more likely it is that the dark ink strokes composed of dark ink pixels in the grayscale image of the image and text are consistent with the motion blur direction.

[0065] At this point, this embodiment obtains the trend direction significance value n of each trend direction of the image and text grayscale image, the horizontal and vertical motion blur direction confidence T of the image and text grayscale image, and the motion blur direction weight of each thick ink stroke direction

[0066] When the confidence T of the horizontal and vertical motion blur direction of the image and text grayscale image is greater than or equal to the confidence threshold t, the motion blur direction of the image and text grayscale image is more likely to be horizontal or vertical, where the confidence threshold is 0.7 in this embodiment. The direction containing the largest number of growth areas is selected from the horizontal and vertical directions as the motion blur direction of the image and text grayscale image.

[0067] When the confidence T of the horizontal and vertical motion blur direction of the image grayscale image is less than the confidence threshold t, the motion blur direction of the image grayscale image is less likely to be horizontal or vertical, and the interference of the horizontal and vertical edge lines in the trend direction histogram should be excluded. After excluding the horizontal and vertical directions, any trend direction is recorded as θ, and the motion blur direction confidence value R of the trend direction θ is calculated as follows θ In this embodiment, the specific calculation formula is:

[0068] In the formula, is the motion blur direction weight when the direction of the thick ink stroke is θ, n θ is the trend direction significance value of the trend direction θ. θ The larger the value, the more likely it is that the edge line trend direction of the grayscale image is consistent with the motion blur direction. The larger it is, the more likely it is that the dark ink strokes composed of dark ink pixels are consistent with the motion blur direction, excluding the horizontal and vertical directions, and θ is more likely to be the motion blur direction of the grayscale image.

[0069] According to the above process, the motion blur direction confidence values ​​of all other trend directions are calculated after excluding the horizontal and vertical directions, and the trend direction corresponding to the maximum motion blur direction confidence value is used as the motion blur direction of the text and image grayscale image.

[0070] So far, according to the above process of this embodiment, the estimation of the motion blur direction of the image and text grayscale image is achieved.

[0071] Step 5: Based on the motion blur direction estimation result of the grayscale image, and combined with the gradient direction, gradient amplitude and path information of the dark ink pixel in the motion blur direction of each corner point, the motion blur feature value of each corner point with respect to each gradient direction is constructed.

[0072] Motion blur causes unclear character outlines and loss of details in the grayscale image of the image acquired by the data collector, which can easily cause the SIFT algorithm to output too few key points, resulting in the misjudgment of text characters as similar-structured image characters, and low OCR image recognition accuracy. The corner points in the grayscale image of the image have a certain robustness under the conditions of scaling and lighting changes, which can make up for the lack of key points in the SIFI algorithm, but it is easy to misdetect redundant corner points, and further screening of the corner points is required.

[0073] Considering that key points should be highly robust to motion blur, the text strokes that are consistent with the motion blur direction can retain a large amount of text stroke detail information. This embodiment uses the Sobel operator to calculate the gradient direction and gradient amplitude of any pixel in the text grayscale image. Further, combining the gradient direction, gradient amplitude of each corner point and the path information of the thick ink pixel point in the motion blur direction, the motion blur feature value of any corner point about the gradient direction x can be obtained. In this embodiment, the specific calculation formula is:

[0074] In the formula, v q,x is the motion blur feature value of the corner point q about the gradient direction x, φ is the motion blur direction of the image grayscale image, and since the gradient direction is the direction in which the brightness of the pixel changes fastest, in this embodiment, the complementary angle of the gradient direction is the stroke direction, Cos() is the cosine value, and h q,x is the gradient magnitude of the corner point q in the gradient direction x, h q,j is the gradient amplitude of corner point q in gradient direction j, the lower limit of j is 1°, and J is the number of gradient directions.

[0075] Understandably, The larger it is, the more significant the edge trend characteristics of the corner point q in the gradient direction x are. At the same time, when Cos(180-x,φ) is larger, the gradient direction of the text strokes in the neighborhood of the corner point q is more likely to be perpendicular to the motion blur direction, the more consistent the text stroke direction is with the motion blur direction, the less the gradient direction x is disturbed by motion blur, the more the gradient direction x can represent the motion blur direction, the closer the edge trend characteristics of the gradient direction x are to the template text, and the more suitable the corner point q is as the key point for feature matching.

[0076] Step six: According to the motion blur feature value and the distribution of dark ink pixels in the neighborhood of each corner point, the corner points are screened to obtain the key points for feature matching of each text detection frame, and the feature descriptor of the key point in the motion blur direction is constructed in combination with the motion blur direction and the number of dark ink pixels in the neighborhood of the key point. The key point matching algorithm is used to automatically recognize the OCR images and texts of the data collector, and the number of characters is counted through the number of text detection frames.

[0077] In the grayscale image of the text, the blur degree of the text strokes that are consistent with the motion blur direction is low. Therefore, in the preset neighborhood of the corner point, this embodiment obtains a straight line that is in the motion blur direction and passes through the corner point, and counts the number of thick ink pixels that the straight line passes through in the local neighborhood of the corner point as the thick ink pixel length of the corner point. The preset neighborhood size is 9*9 in this embodiment.

[0078] Furthermore, in this embodiment, the product of the maximum value of the motion blur eigenvalue of each corner point and the length of the thick ink pixel is used as the screening robustness strength of each corner point. Specifically, for the convenience of understanding and expression, in this embodiment, the gradient direction with the maximum motion blur eigenvalue of each corner point is recorded as the blur representation direction of each corner point. It can be understood that the larger the motion blur eigenvalue, the less the blur representation direction of the corner point is disturbed by motion blur, and the more suitable it is as a key point for feature matching. At the same time, the larger the length of the thick ink pixel, the higher the length of the pixel point of the text stroke with high robustness in the motion blur process, the more it can represent the local information of the stroke of the text template in the motion blur direction, and the more it should be screened as a key point for feature matching.

[0079] At this point, the screening robust strength of all corner points in the image and text grayscale image is obtained, and the diagonal points are arranged from large to small according to the screening robust strength, and the first preset number of corner points are selected as key points for image and text character feature matching, where the preset number in this embodiment is 20. The motion blur direction of the image and text grayscale image, the blur representation direction of each key point, the motion blur feature value, and the thick ink pixel length are normalized respectively, and the vector composed of the normalized result values ​​is used as the feature descriptor of each key point in the motion blur direction.

[0080] In order to realize the character quantity statistics of the automatic image and text recognition machine, this embodiment first establishes a text template library, obtains the corresponding key points and the feature descriptors of each key point in any motion blur direction for each text template, and further uses the SIFI algorithm to match the key points in the image and text grayscale image. In actual application scenarios, the implementer can also use the FLANN fast approximate nearest neighbor search algorithm for key point matching. Through key point matching, the text information in the image and text grayscale image can be obtained, that is, each text detection box in the image and text grayscale image obtains a text template with the highest matching degree, realizes OCR image and text automatic recognition, and counts the total number of all text detection boxes in the image and text grayscale image to perform character quantity statistics. Among them, the SIFI algorithm and the FLANN algorithm to realize key point matching are well-known technologies, and the specific process will not be repeated.

[0081] It is understood that references to "one embodiment" or "some embodiments" etc. described in the specification of the present application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, if "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appear in different places in this specification, they do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0082] It should be noted that the sequence of the above-mentioned embodiments of the present application is only for description and does not represent the advantages and disadvantages of the embodiments. And the above-mentioned specific embodiments of this specification are described. In addition, the process depicted in the accompanying drawings does not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-tasking and parallel processing are also possible or may be advantageous. At the same time, the size of the sequence number of each step in the embodiment does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments in this specification.

[0083] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. An OCR image and text recognition and statistics method for a data collector, characterized in that: The following steps are involved: The image is collected by a data collector to obtain a grayscale image of the image, and the image and text grayscale image is segmented into characters to obtain each character detection frame; Extract edge lines and corner points in the grayscale image of the text and image, analyze the distribution of edge lines and corner points in each text detection frame, and obtain each trend direction and its trend direction significance value; Threshold segmentation is performed on the grayscale image to obtain thick ink pixels, and thick ink strokes are extracted by region growing algorithm. The direction of thick ink strokes in each growing region is determined according to the direction of the thick ink strokes. According to the difference in the number of growing regions in the horizontal and vertical directions of thick ink strokes, the confidence of the horizontal and vertical motion blur directions of the image and text grayscale image is constructed, and the motion blur direction of the image and text grayscale image is estimated by combining the number of growing regions in each thick ink stroke direction; According to the motion blur direction estimation result of the image grayscale image, and combined with the gradient direction, gradient amplitude and path information of the dark ink pixel in the motion blur direction of each corner point, the motion blur feature value of each corner point with respect to each gradient direction is constructed; According to the motion blur feature value and the distribution of dark ink pixels in the neighborhood of each corner point, the corner points are screened to obtain the key points for feature matching of each text detection frame, and combined with the motion blur direction and the number of dark ink pixels in the neighborhood of the key point, the feature descriptor of the key point in the motion blur direction is constructed, and the key point matching algorithm is used to automatically recognize the OCR images and texts of the data collector, and the number of characters is counted through the number of text detection frames.

2. The OCR image and text recognition and statistics method for a data collector as claimed in claim 1, characterized in that: The acquisition of each trend direction further includes: Count the ratio of the number of edge pixels in each text detection frame to the total number of pixels in the text detection frame, and arrange the text detection frames in ascending order according to the ratio, and obtain a preset number of text detection frames, which are recorded as the detection frames to be analyzed; The edge line in the detection frame to be analyzed is segmented by the corner points in the detection frame to be analyzed to obtain edge line segments, and each edge line segment is fitted with a straight line, and the angle between each fitted straight line and the horizontal direction is calculated as each trend direction.

3. The OCR image and text recognition and statistics method for a data collector as claimed in claim 2, characterized in that: The method for acquiring the trend direction significance value of each trend direction is: counting the number of fitted straight lines under the same trend direction and using it as the trend direction significance value of the same trend direction.

4. The OCR image and text recognition and statistics method for a data collector as claimed in claim 1, characterized in that: The acquisition of thick ink pixels further includes: The first threshold segmentation is performed on the grayscale image of the text and image, and the first segmentation threshold is output. The pixels with grayscale values ​​lower than the first segmentation threshold are regarded as stroke pixels. The grayscale values ​​of all stroke pixels are subjected to a second threshold segmentation, and the second segmentation threshold is output. The pixels with grayscale values ​​lower than the second segmentation threshold are regarded as thick ink pixels.

5. The OCR image and text recognition and statistics method for a data collector as claimed in claim 1, characterized in that: The method for determining the direction of the thick ink strokes is: extracting the skeleton of each growth region, performing straight line fitting on the skeleton, and determining the angle between the fitting straight line and the horizontal direction as the direction of the thick ink strokes of each growth region.

6. The OCR image and text recognition and statistics method for a data collector as claimed in claim 1, characterized in that: The expression of the confidence of the horizontal and vertical motion blur direction of the image grayscale image is: Where T is the confidence of the horizontal and vertical motion blur direction of the grayscale image, μ1 is the number of growth areas with a thick ink stroke direction of 90°, μ2 is the number of growth areas with a thick ink stroke direction of 180°, max() is the maximum value function, and M is the number of growth areas.

7. The OCR image and text recognition and statistics method for a data collector as claimed in claim 1, characterized in that: The estimating of the motion blur direction of the image-text grayscale image further comprises: When the confidence of the horizontal and vertical motion blur directions is greater than or equal to the preset confidence threshold, the direction containing the largest number of growth areas is selected from the horizontal and vertical directions as the motion blur direction of the image grayscale image; When the confidence of the horizontal and vertical motion blur direction is less than the preset confidence threshold, the motion blur direction confidence values ​​of all trend directions except the horizontal and vertical directions are calculated, and the trend direction corresponding to the maximum motion blur direction confidence value is used as the motion blur direction of the text grayscale image; The expression of the motion blur direction confidence value is: In the formula, R θ 、n θ They are respectively the motion blur direction confidence value and trend direction significance value of the trend direction θ. It is the motion blur direction weight when the thick ink stroke direction is θ, wherein the proportion of the number of growth areas in each thick ink stroke direction in the total number of growth areas in all thick ink stroke directions is used as the motion blur direction weight of each thick ink stroke direction.

8. The OCR image and text recognition and statistics method for a data collector as claimed in claim 1, characterized in that: The expression of the motion blur characteristic value of each corner point with respect to each gradient direction is: In the formula, v q,x is the motion blur eigenvalue of the corner point q about the gradient direction x, φ is the motion blur direction of the grayscale image, Cos() is the cosine value, J is the number of gradient directions, and h q,x is the gradient magnitude of the corner point q in the gradient direction x, h q,j is the gradient magnitude of corner point q in gradient direction j.

9. The OCR image and text recognition and statistics method for a data collector as claimed in claim 1, characterized in that: The screening of key points includes: Obtain a straight line in the preset neighborhood of each corner point in the direction of motion blur and passing through the corner point, and count the number of dark ink pixels that the straight line passing through the corner point passes through in the preset neighborhood of the corner point as the dark ink pixel length of the corner point; The product of the maximum value of the motion blur eigenvalue of each corner point and the length of the thick ink pixel is used as the screening robust strength of each corner point. The screening robust strength of all corner points in the grayscale image of the image is arranged from large to small according to the screening robust strength, and a preset number of corner points are selected as key points.

10. The OCR image and text recognition and statistics method for a data collector as claimed in claim 9, characterized in that: The construction of the feature descriptor of the key point in the motion blur direction includes: The motion blur direction of the grayscale image, the blur representation direction of each key point, the motion blur feature value and the length of thick ink pixel are normalized respectively, and the vector composed of the normalized result values ​​is used as the feature descriptor of each key point in the motion blur direction.

Images