A digital instrument reading recognition method based on improved threading method

Abstract

The application relates to a digital instrument reading recognition method based on an improved threading method, and the method is characterized in that the method comprises the following steps: acquiring a digital instrument image; pre-processing the digital instrument image; image correction: based on contour and polygon fitting, and referring to a Ransac algorithm, effective samples of the pre-processed digital instrument image are extracted, and the upper and lower boundaries and the left and right boundaries of the digital region of the digital instrument are respectively located, and a perspective transformation is used to correct a quadrilateral region containing numbers, so that the strokes of the nixie tube are horizontal and vertical; digital recognition: the corrected image is segmented into single characters by using a vertical projection method, and the improved threading method is used to complete the recognition of each character, so that the reading value of the digital instrument is obtained. Compared with the prior art, the application has the advantages of accurate recognition and good stability.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image recognition, in particular to a digital instrument reading recognition method based on improved threading method. BACKGROUND

[0002] There are a large number of digital tube display digital instruments in industrial machine rooms, and regular inspection of these instruments helps the safe and stable operation of enterprises. Under the background of deep integration of informatization and industrialization, the traditional manual meter reading inspection system cannot meet the needs of smart industry. At present, the recognition method of digital instrument is mostly based on template matching and CRNN network, wherein the former requires high image quality of the instrument, requires shooting the instrument plane directly, has low resistance to disturbance, poor stability, and is only suitable for digital instruments similar in style, and has small scope of application; the latter needs to make a large digital instrument dataset, and the network training is time-consuming and laborious, and the inference speed is slow, and the real-time performance is not high. The "threading method" is a good digital tube recognition method, which has small calculation amount, high real-time performance and strong anti-interference ability, but the disadvantage is that the influence of decimal point on recognition is not considered, and the light and dark of the decimal point cannot be judged when recognizing the digital instrument of eight segment digital tube, and the digital "1" such narrow character requires the digital to fall in the right area of the segmentation graph, otherwise it will be recognized as digital "0" or "8", which brings inconvenience to the segmentation of the digital, in addition, for some digital instruments with large digital tube spacing, the digital "7" will be segmented into upper line and digital "1", and the traditional threading method cannot correctly recognize it. Based on the above recognition defects, the existing threading method is difficult to realize high-precision digital instrument reading recognition. SUMMARY

[0003] The purpose of the present application is to provide a digital instrument reading recognition method based on improved threading method, improve recognition accuracy and recognition efficiency.

[0004] The purpose of the present application can be realized by the following technical solutions:

[0005] A digital instrument reading recognition method based on improved threading method, comprising the following steps:

[0006] Step 1) obtaining a digital instrument image;

[0007] Step 2) preprocessing the digital instrument image;

[0008] Step 3) image correction: based on contour and polygon fitting, and referring to Ransac algorithm, the effective samples of the preprocessed digital instrument image are extracted, and the upper and lower boundaries and the left and right boundaries of the digital instrument digital region are located respectively, and the perspective transformation is used to correct the quadrilateral region containing the digital, so that the digital tube stroke is horizontal and vertical;

[0009] Step 4) Digital recognition: using the method of vertical projection, the corrected image is segmented into single characters, and the improved threading method is used to complete the recognition of each character to obtain the reading value of the digital instrument.

[0010] The step 2) comprises the following steps:

[0011] Step 2-1) Check the picture format: determine whether the digital instrument image is a color image, if yes, execute step 2-2), otherwise report an error;

[0012] Step 2-2) BGR color channel separation is performed on the digital instrument image;

[0013] Step 2-3) The halo effect of light-emitting diode is suppressed by using channel fusion, and the channel fusion calculation method is:

[0014] Value(i,j)=λ red R(i,j)+λ green G(i,j)+λ blue B(i,j)

[0015] Wherein, R(i,j), G(i,j), B(i,j) represent the pixel values of the separated red, green and blue channels respectively, λ red , λ green , λ blue are the channel fusion weights;

[0016] Step 2-4) Based on OTSU algorithm, the fused gray image is binarized to facilitate contour searching;

[0017] Step 2-5) Based on morphological opening operation, the noise in the binarized image is removed to complete image preprocessing.

[0018] The step 3) comprises the following steps:

[0019] Step 3-1) Based on contour searching and polygon fitting, the upper and lower boundaries of the digital region of the preprocessed digital instrument image are extracted;

[0020] Step 3-2) Based on Ransac algorithm, the left and right boundaries of the digital region of the preprocessed digital instrument image are extracted;

[0021] Step 3-3) Based on the upper and lower boundaries and the left and right boundaries, the quadrilateral region containing the numbers is determined, and perspective transformation correction is performed.

[0022] The step 3-1) comprises the following steps:

[0023] Step 3-1-1) Determine all the outer contours of the digital region of the digital instrument by contour searching;

[0024] Step 3-1-2) sort the contours by length, keep all contours whose length is larger than a first length threshold, and judge the decimal point for the contours whose length is smaller than the first length threshold but larger than a second length threshold, if the contour is a possible decimal point, keep it, otherwise, remove it, wherein the first length threshold is larger than the second length threshold;

[0025] Step 3-1-3) calculate the outer bounding polygon of the contours based on the kept contours and filter the horizontal edges of the polygon;

[0026] Step 3-1-4) classify the horizontal edges into candidate upper and lower boundaries according to their vertical positions;

[0027] Step 3-1-5) select the longest edge from the candidate upper and lower boundaries as the upper and lower boundaries of the digital region respectively.

[0028] The method of judging the decimal point is that, for the contours whose length is smaller than the first length threshold but larger than the second length threshold, judge their positions, if they are located in the lower part of the image, keep them as possible contours of the decimal point.

[0029] The step 3-2) comprises the following steps:

[0030] Step 3-2-1) perform Sobel filtering on the preprocessed digital instrument image to extract the longitudinal edge information;

[0031] Step 3-2-2) detect the straight lines in the longitudinal edge information of the obtained image by using Hough transform, and keep the detected longitudinal straight lines;

[0032] Step 3-2-3) extract the effective data in the kept longitudinal straight lines by using Ransac algorithm, and calculate the tilt angle θ of the digital number, and the deviation of the inliers and the outliers is the included angle between the two straight lines l1, l2:

[0033]

[0034] Step 3-2-4) determine the left and right boundary points of the digital contour based on the tilt angle, and form the left and right boundaries of the digital region based on the left and right boundary points, wherein the method of determining the left and right boundary points is:

[0035]

[0036] wherein (x i ,y i ) is the image coordinate.

[0037] The step 3-3) is specifically: calculating the intersection of the upper and lower boundaries and the left and right boundaries, taking the four vertices of the original graph as target points, correcting the digital region by using perspective transformation, and if the image contains a decimal point, simultaneously calculating the position of the decimal point after perspective transformation; the perspective transformation calculation method in the homogeneous coordinates is:

[0038]

[0039] Wherein, M is the perspective transformation matrix, (x', y', z') is the homogeneous coordinates of the center of the decimal point after perspective transformation.

[0040] The step 4) includes the following steps:

[0041] Step 4-1) using the vertical projection method on the digital image obtained in step 3), the corrected image is segmented into single characters to obtain a plurality of segmented regions;

[0042] Step 4-2) for each segmented region, based on the improved threading method, the center vertical line, the lower horizontal line, the upper horizontal line and the right vertical line are processed respectively, and the processing results are matched based on the pre-configured improved threading method value table, and the value of the recognized number is output;

[0043] Step 4-3) judging whether there is a situation that the decimal point is recognized in the correction process but the decimal point is not recognized by the improved threading method, if yes, correcting the output number according to the position of the recognized decimal point in the correction process, and if no, turning to step 4-4);

[0044] Step 4-4) integrating the integer part of the number value and the decimal part of the number value, and outputting the digital instrument reading recognition result.

[0045] The improved threading method is improved on the basis of the traditional threading method as follows:

[0046] a) To solve the problem of decimal point adhesion, a threading line is added at the position of 11 / 12 width, which is used for decimal point recognition;

[0047] b) To solve the recognition error of the number 1, two state change flags toNull and toFull are added based on the original number tube segments a to g, which respectively represent the two state changes of the pixel from having to no and from no to having in the upper horizontal line detection process, so as to distinguish the hollow numbers 0 and 8 from the non-hollow number 1;

[0048] c) To solve the separation misplacement problem of the number 7, a memory mechanism is introduced to judge whether the two continuous characters are related, and the final correct character is obtained according to the combination relationship of the two characters.

[0049] The center vertical line processing includes the following steps:

[0050] Step 4-2-1-1) traverse the center vertical line, and determine whether the foreground color occupies more than a preconfigured percentage in a preconfigured number of continuous pixels, if yes, execute step 4-2-1-2), otherwise, go to step 4-2-1-3);

[0051] Step 4-2-1-2) determine whether the current position is in the upper third, if yes, the a segment is bright, otherwise, determine whether the current position is in the lower third, if yes, the d segment is bright, otherwise, the g segment is bright;

[0052] Step 4-2-1-3) determine whether the traversal is completed, if yes, complete the center vertical line processing, if not, return to step 4-2-1-1) to continue the traversal;

[0053] The lower horizontal line processing includes the following steps:

[0054] Step 4-2-2-1) traverse the lower horizontal line, and determine whether the foreground color occupies more than a preconfigured percentage in a preconfigured number of continuous pixels, if yes, execute step 4-2-2-2), otherwise, go to step 4-2-2-3);

[0055] Step 4-2-2-2) determine whether the current position is in the left half, if yes, the e segment is bright, if not, the c segment is bright;

[0056] Step 4-2-2-3) determine whether the traversal is completed, if yes, complete the lower horizontal line processing, if not, return to step 4-2-2-1) to continue the traversal;

[0057] The upper horizontal line processing includes the following steps:

[0058] Step 4-2-3-1) traverse the upper horizontal line, and determine whether the foreground color occupies more than a preconfigured percentage in a preconfigured number of continuous pixels, if yes, execute step 4-2-3-2), otherwise, go to step 4-2-3-3);

[0059] Step 4-2-3-2) determine whether the current position is in the left half, if the current position is in the left half, the b segment is bright, and determine whether toNull is equal to 1, if toNull = 1, set toFull = 1 and go to step 4-2-3-4), if toNull ≠ 1, go to step 4-2-3-4) directly, if the current position is not in the left half, the f segment is bright, and go to step 4-2-3-4);

[0060] Step 4-2-3-3) determine whether the f segment is bright, if yes, set toNull = 1 and go to step 4-2-3-4), if not, go to step 4-2-3-4) directly;

[0061] Step 4-2-3-4) determine whether the traversal is completed, if yes, complete the upper horizontal line processing, if not, return to step 4-2-3-1) to continue the traversal;

[0062] The right vertical line processing includes the following steps:

[0063] Step 4-2-4-1) judging whether a decimal point has not appeared so far, if yes, traversing the right vertical line, if no, completing the right vertical line processing;

[0064] Step 4-2-4-2) judging whether the foreground color occupies more than a preconfigured percentage in a preconfigured number of continuous pixels, if yes, executing step 4-2-4-4), otherwise, turning to step 4-2-4-3);

[0065] Step 4-2-4-3) judging whether the current position is the lower third, if yes, obtaining the decimal point and turning to step 4-2-4-4), if no, directly turning to step 4-2-4-4);

[0066] Step 4-2-4-4) judging whether the traversal is completed, if yes, completing the right vertical line processing, if no, returning to step 4-2-4-1) to continue the traversal.

[0067] Compared with the prior art, the present application has the following beneficial effects:

[0068] (1) The improved threading method of the present application solves the problem that the traditional threading method cannot recognize the decimal point and the problem of recognition error of the number "1" and the number "7", and improves the recognition accuracy.

[0069] (2) Compared with the template matching method, the digital instrument recognition method of the present application can be used for the recognition of a variety of different style digital instruments, has a wider application range, and can resist character distortion caused by the shooting angle to a certain extent, and has higher stability.

[0070] (3) Compared with the method of the CRNN network, the digital instrument recognition method of the present application has an identification speed improved by more than 30 times on the basis of supporting indefinite length number recognition, has improved real-time performance, does not need to prepare a data set or train a model, is simpler and easier to use, and has very low requirements for hardware computing power.

[0071] (4) In the image preprocessing, the method of channel fusion of the present application suppresses the halo effect of the light emitting diode, and provides a basis for subsequent high-precision recognition. BRIEF DESCRIPTION OF DRAWINGS

[0072] Figure 1 The overall method flowchart of the present application.

[0073] Figure 2 The digital instrument image preprocessing flowchart.

[0074] Figure 3 The digital instrument upper and lower boundary positioning algorithm flowchart.

[0075] Figure 4 Flow chart of left and right boundary positioning algorithm for digital meter.

[0076] Figure 5 Schematic diagram and value reference table of traditional threading method, wherein (5a) is a schematic diagram of traditional threading method, and (5b) is a value reference table.

[0077] Figure 6 Problem explanation diagram of traditional threading method, wherein (6a) is a decimal point sticking situation, (6b) is a "1" identification error situation, and (6c) is a "7" separation error situation.

[0078] Figure 7 Schematic diagram and value reference table of improved threading method, wherein (7a) is a schematic diagram of improved threading method, and (7b) is a value reference table.

[0079] Figure 8 Flow chart of identification algorithm of improved threading method.

[0080] Figure 9 Flow chart of horizontal and vertical line processing method of improved threading method.

[0081] Figure 10 Digital meter preprocessing result diagram in an embodiment, wherein (10a) is an original diagram, (10b), (10c) and (10d) are red channel, green channel and blue channel images respectively, (10e) is a channel fusion image, and (10f) is a binary image.

[0082] Figure 11 Digital meter correction result diagram in an embodiment, wherein (11a) is a digital contour diagram, (11b) is a polygon outer envelope diagram of the contour, (11c) is a Sobel longitudinal filter edge diagram, (11d) is a longitudinal stroke straight line diagram extracted by Hough straight line detection, (11e) is a quadrilateral outer envelope diagram of a digital region, and (11f) is a binary image of the digital region after perspective transformation correction.

[0083] Figure 12 Digital meter character projection and segmentation result diagram in an embodiment, wherein (12a) is a corrected image, (12b) is a character projection image, and (12c) is a segmented image. DETAILED DESCRIPTION

[0084] The present application will be described in detail below in conjunction with the accompanying drawings and specific embodiments. The embodiments are implemented on the premise of the technical solution of the present application, and detailed implementation modes and specific operation processes are given, but the protection scope of the present application is not limited to the following embodiments.

[0085] The embodiment provides a digital instrument reading recognition method based on an improved threading method, as shown in the figure, which comprises three parts of image preprocessing, digital part perspective transformation correction and digital recognition. Figure 1 The input of the application is a digital instrument digital area part image, and the output is an instrument reading, which contains an instrument reading integer part, a decimal part and a decimal point flag bit, so that indefinite length micro-deformation multi-style seven-segment digital recognition can be realized. Meanwhile, key step state codes are returned, including six kinds of non-color image error, contour search error, straight line search error, vertical straight line search error, character segmentation error and reading error, so that different solutions can be taken by the system. Specifically, the following steps are included:

[0086] Step 1) acquiring a digital instrument image.

[0087] Step 2) preprocessing the digital instrument image, and the flowchart is as shown in the figure. Figure 2

[0088] Step 2-1) checking the picture format: judging whether the digital instrument image is a color image, if yes, executing step 2-2), otherwise reporting a non-color image error. This step is used to ensure that the image format is correct.

[0089] Step 2-2) performing BGR color channel separation on the digital instrument image.

[0090] Step 2-3) suppressing the halo effect of light emitting diodes by using channel fusion, and the channel fusion calculation method is as follows:

[0091] Value(i,j) = λ red R(i,j) + λ green G(i,j) + λ blue B(i,j)

[0092] Wherein, R(i,j), G(i,j) and B(i,j) represent the pixel values of the separated red, green and blue channels respectively, λ red , λ green and λ blue are channel fusion weights.

[0093] The processing of steps 2-2) and 2-3) is aimed at the problem that the high brightness of part of the light emitting diodes leads to serious image halo, which affects binarization and contour search. A channel fusion method is proposed, which effectively weakens the halo effect and improves the system stability.

[0094] Step 2-4) performing binarization processing on the fused gray image based on the OTSU algorithm, so as to perform contour search.

[0095] ​OTSU is a self-adaptive threshold image binarization method, which calculates the optimal segmentation threshold by maximizing the inter-class variance of foreground and background, and uses it to binarize the fused gray image so as to perform contour search.

[0096] Step 2-5) The binarized image may have noise interference and be easily mistaken as a decimal point, so further morphological opening operation is used to remove the noise in the binarized image to complete image preprocessing.

[0097] Step 3) Image correction: based on contour and polygon fitting, and referring to Ransac algorithm, effective samples of the preprocessed digital instrument image are extracted, and the upper and lower boundaries and the left and right boundaries of the digital instrument digital region are located respectively, and perspective transformation is used to correct the quadrilateral region containing the digital, so that the digital tube stroke is horizontal and vertical.

[0098] Step 3-1) Based on contour search and polygon fitting, the upper and lower boundaries of the digital region of the preprocessed digital instrument image are extracted.

[0099] The upper and lower boundary positioning process is shown in Figure 3 Since the number "7" does not have a straight left boundary, the positioning of the upper and lower boundaries and the left and right boundaries of the digital instrument is not suitable for the same algorithm.

[0100] Step 3-1-1) All outer contours of the digital instrument digital region are determined by contour search.

[0101] Step 3-1-2) The contours found are sorted by length, and all contours with a length greater than a first length threshold are retained, and the contours with a length less than the first length threshold but greater than a second length threshold are judged as decimal points, and if judged as possible decimal points, they are retained, otherwise they are removed, wherein the first length threshold is greater than the second length threshold.

[0102] In this embodiment, the method of decimal point judgment is: for the contours with a length less than the first length threshold but greater than the second length threshold, the position is judged, and if it is located in the lower region of the image, it is retained as a possible contour of the decimal point.

[0103] That is, the judgment method of the retained contour is:

[0104]

[0105] Wherein, size is the contour length, pos_y is the mean value of the contour longitudinal coordinate, high_th is the first length threshold, low_th is the second length threshold, and y0 is the longitudinal coordinate threshold. In a computer image, the lower the position, the greater the longitudinal coordinate of the pixel, so pos_y>y0 represents the points below y0.

[0106] Step 3-1-3) Calculate the outer polygon of the contour based on the reserved contour and screen the horizontal edges of the polygon.

[0107] Step 3-1-4) Classify the horizontal edges as candidate upper and lower boundaries according to their vertical positions. In this embodiment, the horizontal edges of the polygon within the range of -45° to 45° are regarded as the candidate upper and lower boundaries of the to-be-recognized digit.

[0108] Step 3-1-5) Screen the longest edge from the candidate upper and lower boundaries as the upper and lower boundaries of the digit region, respectively.

[0109] Step 3-2) Extract the left and right boundaries of the digit region of the preprocessed digital instrument image based on the Ransac algorithm, and the positioning process of the left and right boundaries is shown in Figure 4

[0110] Step 3-2-1) Perform Sobel filtering on the preprocessed digital instrument image to extract the longitudinal edge information.

[0111] Step 3-2-2) Detect the straight lines in the longitudinal edge information of the obtained image by using Hough transform, and reserve the detected longitudinal straight lines.

[0112] Step 3-2-3) Extract the effective data in the reserved longitudinal straight lines by using the Ransac algorithm, and calculate the tilt angle θ of the instrument digit, and the deviation of the inlier and the outlier is the included angle between the two straight lines l1 and l2:

[0113]

[0114] Step 3-2-4) Determine the left and right boundary points of the digit based on the tilt angle, and form the left and right boundaries of the digit region based on the left and right boundary points, wherein the method for determining the left and right boundary points is as follows:

[0115]

[0116] wherein (x i ,y i ) is the image coordinate.

[0117] Step 3-3) Determine the quadrilateral region containing the digit based on the upper and lower boundaries and the left and right boundaries, and perform perspective transformation correction on the quadrilateral region.

[0118] Specifically, the intersection of the upper and lower boundaries and the left and right boundaries is calculated, the four vertices of the original image are taken as the target points, and the perspective transformation of the digit region is corrected, and if the image contains a decimal point, the position of the decimal point after the perspective transformation is also calculated. The perspective transformation calculation method in the homogeneous coordinates is as follows:

[0119]

[0120] ​Wherein, M is a perspective transformation matrix, (x', y', z') is the center of the decimal point homogeneous coordinates after perspective transformation.

[0121] Step 4) Digital recognition: the corrected image is segmented into single characters by using the vertical projection method, and each character is recognized by using the improved threading method to obtain the reading value of the digital instrument.

[0122] The traditional threading method is shown in Figure 5 The three dashed lines are traversed respectively, the bright and dark states of the numeral tube intersecting with them are inquired, and the corresponding character of the digital bit is obtained by looking up the table. This method has small calculation amount, high real-time performance and strong anti-interference ability, but has the disadvantage that the influence of the decimal point on recognition is not considered, and the segmentation of the narrow character such as the numeral '1' requires the numeral to fall on the right side of the segmented image, otherwise it will be recognized as the numeral '0' or '8', which brings inconvenience to the segmentation of the numeral. In addition, the numeral '7' will be segmented into an upper horizontal line and the numeral '1' for some instruments with large spacing between numeral tubes, which is also not considered by the traditional method.

[0123] There are a large number of decimal point sticking situations in real detection, in which the decimal point character cutting is adversely affected due to the design structure of the numeral tube itself, the spread of the numeral tube halo, or the character tilt, so that the decimal point and the numeral character cannot be successfully separated, thereby hindering the recognition of the numeral by the threading method, and even causing recognition errors. The numeral '1' cannot be successfully recognized by the algorithm due to the particularity of its width, so that the correct position of the a, d, e, f, and g segments of the numeral tube cannot be determined, resulting in a judgment error. In the case of non-sticking strokes, it is recognized as '0', and in the case of sticking strokes, it is recognized as '8', both of which are incorrect results, as shown in Figure 6 .

[0124] Therefore, the traditional threading method is improved to solve the above problems, and the recognition is shown in Figure 7 .

[0125] Specifically, the traditional threading method is improved as follows:

[0126] a) To solve the problem of decimal point sticking, a threading line is added at the position of 11 / 12 width, which is used for decimal point recognition.

[0127] b) To solve the recognition error of the numeral '1', two state change flags toNull and toFull are added based on the original numeral tube segment variables a to g, which respectively represent the two state changes of the pixel from having to none and from none to having in the upper horizontal line detection process, so as to distinguish the hollow numerals '0' and '8' from the non-hollow numeral '1'.

[0128] c) To solve the problem of the separation of the digital "7", a memory mechanism is introduced to determine whether two consecutive characters are related, and the final correct character is obtained according to the combination of the two characters.

[0129] In addition, the problem of character tilt caused by incomplete correction is also considered, and the value correspondence table is enriched. The whole digital recognition algorithm flow is as shown in Figure 8 , including the following steps:

[0130] Step 4-1) The digital image obtained in step 3) is segmented into single characters by using the vertical projection method, and a plurality of segmentation regions are obtained.

[0131] Step 4-2) For each segmentation region, the center vertical line, the lower horizontal line, the upper horizontal line and the right vertical line are processed based on the improved threading method respectively, and the processing results are matched based on the pre-configured improved threading method value table. The value of the recognized number is output.

[0132] The processing method of each horizontal or vertical line is similar, as shown in Figure 9 , starting from the first pixel on the line, traversing each pixel point on the line in the horizontal or vertical direction one by one, saving a number of consecutive pixel values, and if the proportion of foreground color in all saved pixels reaches the threshold, the current position is recorded as the light-emitting state of the digital tube. In this way, the interference of noise is suppressed, and the recognition rate is improved. The toNull and toFull flag bits introduced by the upper horizontal line processing. Based on the detection of the light-emitting segment, if the horizontal line pixel changes from foreground to background, toNull=1 is recorded; under the condition that the toNull=1 is met, if the pixel on the horizontal line changes from background to foreground, toFull=1 is recorded, so as to distinguish the digital "1" from the digital "0" and "8", and reduce the requirement for the segmentation algorithm.

[0133] Specifically, the center vertical line processing includes the following steps:

[0134] Step 4-2-1-1) Traverse the center vertical line and determine whether the proportion of foreground color in the continuous 5 pixels exceeds 80%, if yes, execute step 4-2-1-2), otherwise, go to step 4-2-1-3);

[0135] Step 4-2-1-2) Determine whether the current position is in the upper third, if yes, the a segment is bright, otherwise, determine whether the current position is in the lower third, if yes, the d segment is bright, otherwise, the g segment is bright;

[0136] Step 4-2-1-3) Determine whether the traversal is completed, if yes, the center vertical line processing is completed, if not, return to step 4-2-1-1) to continue traversal.

[0137] The lower horizontal line processing includes the following steps:

[0138] Step 4-2-2-1) traverse the lower horizontal line, and determine whether the foreground color accounts for more than 80% in 5 continuous pixels, if yes, execute step 4-2-2-2), otherwise, go to step 4-2-2-3);

[0139] Step 4-2-2-2) determine whether the current position is the left half, if yes, the e segment is bright, if not, the c segment is bright;

[0140] Step 4-2-2-3) determine whether the traversal is completed, if yes, complete the lower horizontal line processing, if not, return to step 4-2-2-1) to continue the traversal.

[0141] The upper horizontal line processing includes the following steps:

[0142] Step 4-2-3-1) traverse the upper horizontal line, and determine whether the foreground color accounts for more than 80% in 5 continuous pixels, if yes, execute step 4-2-3-2), otherwise, go to step 4-2-3-3);

[0143] Step 4-2-3-2) determine whether the current position is the left half, if the current position is the left half, the b segment is bright, and determine whether toNull is equal to 1, if toNull = 1, set toFull = 1 and go to step 4-2-3-4), if toNull ≠ 1, go to step 4-2-3-4) directly, if the current position is not the left half, the f segment is bright, and go to step 4-2-3-4);

[0144] Step 4-2-3-3) determine whether the f segment is bright, if yes, set toNull = 1 and go to step 4-2-3-4), if not, go to step 4-2-3-4) directly;

[0145] Step 4-2-3-4) determine whether the traversal is completed, if yes, complete the upper horizontal line processing, if not, return to step 4-2-3-1) to continue the traversal.

[0146] The right vertical line processing includes the following steps:

[0147] Step 4-2-4-1) determine whether the decimal point has not appeared so far, if yes, traverse the right vertical line, if not, complete the right vertical line processing;

[0148] Step 4-2-4-2) determine whether the foreground color accounts for more than 80% in 5 continuous pixels, if yes, execute step 4-2-4-4), otherwise, go to step 4-2-4-3);

[0149] Step 4-2-4-3) determine whether the current position is the lower third, if yes, get the decimal point and go to step 4-2-4-4), if not, go to step 4-2-4-4) directly;

[0150] Step 4-2-4-4) Determine if the traversal is complete. If yes, complete the processing of the right vertical line. Otherwise, return to step 4-2-4-1) to continue the traversal.

[0151] Step 4-3) Determine if there is a situation where the decimal point was identified during the correction process but the mouse point was not identified by the improved threading method. If so, correct the output number according to the position of the decimal point identified during the correction process; otherwise, go to step 4-4.

[0152] Step 4-4) Integrate the identified integer and fractional digit values ​​and output the digital instrument reading recognition result.

[0153] The improved algorithm takes as input the segmented image obtained from digital instrument correction and character segmentation, as well as the presence or absence of a decimal point and its position after perspective transformation, determined based on the contour. The algorithm iterates through all segmented characters, processing each character with a central vertical line, an upper horizontal line, and a lower horizontal line. It then matches the corresponding character using a lookup table, while simultaneously correcting the output character based on the previous character. Considering that the recognized digital instrument will have at most one decimal point, once a decimal point is matched, subsequent characters do not need to have the 11 / 12-width vertical line on the right processed, thus improving algorithm efficiency.

[0154] Figure 10 This is a preprocessed image of the digital instruments identified in the experiment. The experimental results show that some digital instrument LED displays have excessive brightness or severe halo due to insufficient ambient light. Directly using the red channel data for binarization makes it difficult to correctly extract the edges of the LED displays, thus failing to complete the digital correction.

[0155] Depend on Figure 10 The first row of instruments shows that the blue channel (10d) has very little difference in whether the digital tube is emitting light or not, therefore it is not suitable for separating red emitting digital tubes. Furthermore, since the extracted digits are red, using only the green channel (10c) data is clearly unreasonable. The third row of instruments shows that using only the red channel (10b) data will cause binarization difficulties, making it impossible to successfully extract the digit outline. Experiments show that fusing the red and green channels can obtain a clear digital tube image, facilitating subsequent processing. In this embodiment, the fusion parameter λ is chosen. red =0.5,λ green =0.5,λ blue =0, the fusion effect is shown in Figure (10e).

[0156] The OTSU algorithm binarization does not require a fixed threshold. The algorithm uses the threshold that maximizes the variance between the foreground and background classes to complete the binarization. It is applicable to different lighting conditions and instrument brightness, and can improve the robustness of instrument recognition. Its effect is shown in Figure (10f).

[0157] Experimental results of each step of image correction are as followsFigure 11 As the left boundary of the number "7" is not flat, it is difficult to match the left boundary of the number directly when the number starts with "7". The overall tilt angle can only be estimated by the angle of the vertical strokes of other numbers. However, the camera plane is not necessarily parallel to the plane of the number display, so the number display can only be roughly corrected based on the stroke angle. When the distortion between the two planes is severe, the corrected number is usually not completely regular. The last row of Fig. (11f) confirms this.

[0158] However, in general, the camera plane and the digital instrument plane are basically parallel in the target detection scene, there is no large distortion, and the subsequent threading method further enriches the value correspondence table to solve the problem of number tilt, trying to consider different tilt modes. Therefore, this contour-based perspective transformation correction can still be applied to the digital instrument recognition task of the present application.

[0159] After obtaining the corrected image of the number, all number characters can be segmented by vertical projection. The experimental results are shown in Fig. (12). Figure 12 As can be seen from the experimental results, this method can accommodate a certain correction error. In addition, due to the improvement of the threading method, the accuracy requirement of the decimal point separation is reduced, and the occurrence of decimal point adhesion is allowed, so the character segmentation algorithm meets the requirements.

[0160] Finally, the improved threading method can complete the number recognition of the target machine room digital instrument. The CRNN network was initially used to complete the digital instrument recognition in this embodiment, but due to the insufficient computing power of the hardware platform, at most one target detection network was deployed to complete the positioning and classification of the machine room instrument, so the traditional image processing algorithm was used instead of the CRNN network to complete the digital instrument recognition. The improved threading method of this embodiment takes an average of 20.95 ms, which is 30 times faster than the average time of 697.1 ms of the CRNN network, and at the same time, the accuracy rate of 98.6% is achieved for the recognition of 220 digital instruments.

[0161] The preferred embodiments of the present application are described in detail above. It should be understood that those skilled in the art can make many modifications and changes without creative labor based on the concept of the present application. Therefore, any technical solutions obtained by logical analysis, reasoning, or limited experiments based on the prior art according to the concept of the present application shall be within the scope of protection determined by the claims.

Claims

1. A method for digital meter reading based on improved thread method, characterized in that, The method comprises the following steps: Step 1) acquiring a digital instrument image; Step 2) pre-processing the digital instrument image; Step 3) image correction: based on contour and polygon fitting, and referring to the Ransac algorithm, effective samples of the pre-processed digital instrument image are extracted, and the upper and lower boundaries and the left and right boundaries of the digital instrument digital region are located respectively, and the perspective transformation is used to correct the quadrilateral region containing the numbers, so that the cathode ray tube strokes are horizontal and vertical; Step 4) number recognition: the corrected image is segmented into single characters by using the vertical projection method, and the improved threading method is used to complete the recognition of each character to obtain the reading value of the digital instrument; wherein the improved threading method is improved on the basis of the traditional threading method as follows: a) To solve the problem of decimal point adhesion, a threading line is added at a position of 11 / 12 width, which is used for decimal point recognition; b) To solve the recognition error of the number 1, two state change markers toNull and toFull are added on the basis of the original cathode ray tube segment variables a to g, which represent two state changes of the pixel from having to not having and from not having to having in the upper horizontal line detection process, so as to distinguish the hollow numbers 0 and 8 from the non-hollow number 1; c) To solve the separation misplacement problem of the number 7, a memory mechanism is introduced to judge whether two continuous characters are related, and the final correct character is obtained according to the combination relationship of the two characters.

2. The digital meter reading method based on improved thread method according to claim 1, characterized in that, The step 2) comprises the following steps: Step 2-1) checking the picture format: judging whether the digital instrument image is a color picture, if yes, step 2-2) is executed, otherwise an error is reported; Step 2-2) separating the digital instrument image by BGR color channel; Step 2-3) using channel fusion to suppress the halo effect of light-emitting diodes, and the channel fusion calculation method is: wherein, respectively represent the separated red, green and blue channel pixel values, are the channel fusion weights; Step 2-4) based on the OTSU algorithm, the fused gray image is binarized to facilitate contour searching; Step 2-5) based on morphological opening operation, the noise in the binarized image is removed to complete the image preprocessing.

3. The method of claim 1, wherein the method is based on an improved thread method. The step 3) comprises the following steps: Step 3-1) based on contour searching and polygon fitting, the upper and lower boundaries of the digital region of the pre-processed digital instrument image are extracted; Step 3-2) based on the Ransac algorithm, the left and right boundaries of the digital region of the pre-processed digital instrument image are extracted; Step 3-3) based on the upper and lower boundaries and the left and right boundaries, the quadrilateral region containing the numbers is determined and perspective transformation is performed on it.

4. The digital meter reading method based on improved thread method according to claim 3, characterized in that, The step 3-1) comprises the following steps: Step 3-1-1) all the outer contours of the digital instrument digital region are determined by contour searching; Step 3-1-2) the found contours are sorted by length, and all the contours with a length greater than a first length threshold are retained, and the contours with a length less than the first length threshold but greater than a second length threshold are subjected to decimal point judgment, if it is determined that it may be a decimal point, it is retained, otherwise it is removed, wherein the first length threshold is greater than the second length threshold; Step 3-1-3) based on the retained contours, the outer contour polygon is calculated and the horizontal edges of the polygon are screened; Step 3-1-4) the horizontal edges are classified into candidate upper boundaries and lower boundaries according to their vertical positions; Step 3-1-5) screen the longest side from the alternative upper and lower boundaries as the upper and lower boundaries respectively.

5. The digital meter reading method based on improved thread method according to claim 4, characterized in that, The method for judging the decimal point is that, for the contour with a length less than the first length threshold but more than the second length threshold, the position of the contour is judged, and if the contour is located in the lower region of the image, the contour is retained as a possible contour of the decimal point.

6. The digital meter reading method based on improved thread method according to claim 3, characterized in that, The step 3-2) comprises the following steps: Step 3-2-1) performing Sobel filtering on the preprocessed digital instrument image to extract longitudinal edge information; Step 3-2-2) detecting straight lines in the longitudinal edge information of the obtained image by using Hough transform, and retaining the detected longitudinal straight lines; Step 3-2-3) Extract the effective data in the reserved longitudinal straight line using Ransac algorithm, and calculate the tilt angle of the instrument digital The deviation of inliers and outliers is the included angle of two straight lines : Step 3-2-4) determining the left and right boundary points of the digital contour based on the tilt angle, and forming the left and right boundaries of the digital region based on the left and right boundary points, wherein the method for determining the left and right boundary points is as follows: wherein are image coordinates.

7. The method as claimed in claim 3, wherein, The step 3-3) is specifically as follows: calculating the intersection of the upper and lower boundaries and the left and right boundaries, taking the four vertices of the original image as target points, and correcting the digital region by using perspective transformation, and if the image contains a decimal point, the position of the decimal point after the perspective transformation is also calculated, and the method for calculating the perspective transformation in the homogeneous coordinates is as follows: wherein, is the perspective transformation matrix, is the decimal point center homogenous coordinate after perspective transformation.

8. The method of claim 1, wherein, The step 4) comprises the following steps: Step 4-1) dividing the corrected image into single characters by using the vertical projection method to segment the corrected image into multiple segmentation regions; Step 4-2) for each segmentation region, performing center vertical line, lower horizontal line, upper horizontal line and right vertical line processing based on the improved threading method respectively, and matching the processing results based on a preconfigured improved threading method value table to output the value of the recognized number; Step 4-3) judging whether there is a case that the decimal point is recognized in the correction process but the improved threading method does not recognize the decimal point, and if yes, correcting the output number according to the position of the recognized decimal point in the correction process, and if no, proceeding to step 4-4); Step 4-4) integrating the integer part of the recognized number value and the decimal part of the number value to output the digital instrument reading recognition result.

9. The digital meter reading method based on improved thread method according to claim 8, characterized in that, The center vertical line processing comprises the following steps: Step 4-2-1-1) traversing the center vertical line, and judging whether the proportion of the foreground color in a continuous preconfigured number of pixels exceeds a preconfigured percentage, if yes, proceeding to step 4-2-1-2), otherwise, proceeding to step 4-2-1-3); Step 4-2-1-2) judging whether the current position is in the upper third, if yes, the a segment is bright, otherwise, judging whether the current position is in the lower third, if yes, the d segment is bright, otherwise, the g segment is bright; Step 4-2-1-3) judging whether the traversal is completed, if yes, completing the center vertical line processing, if no, returning to step 4-2-1-1) to continue the traversal; The lower horizontal line processing comprises the following steps: Step 4-2-2-1) traversing the lower horizontal line, and judging whether the proportion of the foreground color in a continuous preconfigured number of pixels exceeds a preconfigured percentage, if yes, proceeding to step 4-2-2-2), otherwise, proceeding to step 4-2-2-3); Step 4-2-2-2) judging whether the current position is in the left half, if yes, the e segment is bright, if no, the c segment is bright; Step 4-2-2-3) judging whether the traversal is completed, if yes, completing the lower horizontal line processing, if not, returning to step 4-2-2-1) to continue the traversal; The upper horizontal line processing comprises the following steps: Step 4-2-3-1) traversing the upper horizontal line and judging whether the proportion of the foreground color in the preconfigured number of continuous pixels exceeds the preconfigured percentage, if yes, executing step 4-2-3-2), otherwise, transferring to step 4-2-3-3); Step 4-2-3-2) judging whether the current position is the left half, if the current position is the left half, then the b segment is bright, and judging whether toNull is equal to 1, if toNull=1, setting toFull=1 and transferring to step 4-2-3-4), if toNull≠1, directly transferring to step 4-2-3-4), if the current position is not the left half, then the f segment is bright, and transferring to step 4-2-3-4); Step 4-2-3-3) judging whether the f segment is bright, if yes, setting toNull=1 and transferring to step 4-2-3-4), if not, directly transferring to step 4-2-3-4); Step 4-2-3-4) judging whether the traversal is completed, if yes, completing the upper horizontal line processing, if not, returning to step 4-2-3-1) to continue the traversal; The right vertical line processing comprises the following steps: Step 4-2-4-1) judging whether the decimal point has not appeared so far, if yes, traversing the right vertical line, if not, completing the right vertical line processing; Step 4-2-4-2) judging whether the proportion of the foreground color in the preconfigured number of continuous pixels exceeds the preconfigured percentage, if yes, executing step 4-2-4-4), otherwise, transferring to step 4-2-4-3); Step 4-2-4-3) judging whether the current position is the lower third, if yes, obtaining the decimal point and transferring to step 4-2-4-4), if not, directly transferring to step 4-2-4-4); Step 4-2-4-4) judging whether the traversal is completed, if yes, completing the right vertical line processing, if not, returning to step 4-2-4-1) to continue the traversal.

Images