An automatic positioning and identification method for instruments used in wheeled inspection robots

Abstract

The present invention discloses an automatic positioning and identification method for an instrument applied to a wheeled inspection robot, relates to the technical field of instrument reading detection, and solves the technical problem of low efficiency of manual inspection. The method comprises: step S1. collecting images of instrument detection points, performing labelless positioning through a deep learning algorithm, and obtaining an instrument area; step S2. preprocessing the instrument area using a threshold method and a morphological preprocessing method to obtain an instrument pointer area; step S3. judging the front and rear ends of the pointer by comparing the number of pixels at both ends of the instrument pointer area; step S4. determining the rotation angle through the slope of the straight line of the pointer and the front vertex of the pointer, and finally calculating the reading. Compared with manual detection, the present invention can detect continuously and uninterruptedly for 24 hours, and can timely feedback instrument information; it can realize remote automatic monitoring, save time and manpower, improve work efficiency, and ensure the personal safety of staff; the present invention is not affected by complex background interference, has strong robustness, and is highly portable.

Description

Technical Field

[0001] The invention relates to the technical field of instrument reading detection, and more specifically, to an automatic positioning and identification method for an instrument applied to a wheeled inspection robot. Background Art

[0002] Instrument identification is an effective way to control the production process and product quality of enterprises. At present, the identification of instrument readings is mainly done manually, which not only consumes a lot of manpower costs and has low inspection efficiency, but also produces gross errors and random errors due to human factors, affecting the production quality of enterprises, and may even cause safety accidents due to untimely detection. Summary of the invention

[0003] The technical problem to be solved by the present invention is to address the above-mentioned deficiencies in the prior art. The purpose of the present invention is to provide an automatic instrument positioning and identification method for a wheeled inspection robot that can replace manual inspection.

[0004] The technical solution of the present invention is: an automatic positioning and identification method for an instrument applied to a wheeled inspection robot, comprising:

[0005] Step S1. Collect images of instrument detection points, perform signless positioning through deep learning algorithms, and obtain instrument areas;

[0006] Step S2. Preprocessing the instrument area using a threshold method and a morphological preprocessing method to obtain an instrument pointer area;

[0007] Step S3. Determine the front and rear ends of the pointer by comparing the number of pixels at both ends of the instrument pointer area;

[0008] Step S4. Determine the rotation angle through the straight line slope of the pointer and the front vertex of the pointer, and finally calculate the reading.

[0009] As a further improvement, the step S2 includes:

[0010] Step S21. Perform mean filtering and stretching on the instrument area, obtain the grayscale histogram of all pixels in the instrument area, obtain the trough grayscale value, the peak grayscale value, the minimum grayscale value of the number of non-zero pixels, and the maximum grayscale value of the number of non-zero pixels;

[0011] Step S22. Using the trough grayscale value as the initial value of the binarization threshold of the instrument area;

[0012] Step S23. Using the binary threshold as a threshold to threshold the instrument area, obtain a threshold image, eliminate noise interference through morphological processing, and count the actual number of pixels with a gray value of 255;

[0013] Step S24. If the ratio of the actual number to the number of pixels in the instrument area is less than the set threshold, the binarization threshold is increased by (maximum gray value - minimum gray value) * set threshold + minimum gray value, and step S23 is executed; otherwise, the current binarization threshold is assigned to the filtering threshold;

[0014] Step S25. Perform morphological calculation on the threshold image obtained in step S23, remove interference noise, calculate the outline of the threshold image and the minimum area enclosing rectangle, and determine whether the aspect ratio and length of the longest side rectangle meet the requirements; if yes, take the longest side rectangle as the instrument pointer area and execute step S3; otherwise, execute step S26;

[0015] Step S26. Perform a secondary calculation to determine whether the second longest side rectangle meets the requirements. If it does, execute step S3; otherwise, execute step S1.

[0016] Further, whether the aspect ratio and the length of the longest side rectangle meet the requirements is determined as follows: the aspect ratio of the longest side rectangle is greater than 4 and the length of the longest side rectangle is greater than the instrument area*0.66.

[0017] Furthermore, the secondary calculation is specifically as follows: if at least one of the aspect ratios of the second longest side rectangle and the longest side rectangle is greater than 3.5, compare the aspect ratios of the two rectangles and select the rectangle with the larger aspect ratio as the instrument pointer area; otherwise; if at least one of the lengths of the second longest side rectangle and the longest side rectangle is greater than the instrument area * 0.5, compare the aspect ratios of the two rectangles and select the larger one as the instrument pointer area.

[0018] Furthermore, the set threshold is 0.035-0.05.

[0019] Furthermore, the step S3 comprises:

[0020] Step S31. fitting a straight line according to the pointer pixel points in the instrument pointer area;

[0021] Step S32. Obtain four vertices P0, P1, P2, and P3 of the two short sides of the instrument pointer area, and the midpoints centerP2 and centerP3 of the two long sides, and record the rectangle formed by P0, P1, centerP2, and centerP3 as the first end, and record the rectangle formed by P2, P3, centerP2, and centerP3 as the second end;

[0022] Step S33. Obtain the intersection point X1 of the straight line and the short side of the first end, the intersection point X2 of the straight line and the short side of the second end, the midpoint centerP0 of the short side of the first end, and the midpoint centerP1 of the short side of the second end; count the number of white pixel points at the first end, denoted as whiteNum1, and count the number of white pixel points at the second end, denoted as whiteNum2;

[0023] Step S34. Calculate the distance from centerP0 to the image center centerP of the instrument pointer area, denoted as len1, and calculate the distance from centerP1 to the image center centerP of the instrument pointer area, denoted as len2;

[0024] If len1 > 2 * len2, calculate the distance D1 between X1 and centerP0, and calculate the distance D2 between X2 and centerP0; if D1 < D2, then take X1 as the front end of the pointer; if D1 > D2, then take X2 as the front end of the pointer;

[0025] If len2 > 2 * len1, calculate the distance D3 between X1 and centerP1, and calculate the distance D4 between X2 and centerP1; if D3 < D4, then take X1 as the front end of the pointer; if D3 > D4, then take X2 as the front end of the pointer;

[0026] If whiteNum1 < whiteNum2, then take X1 as the front end of the pointer;

[0027] If whiteNum1 > whiteNum2, then take X2 as the front end of the pointer;

[0028] Otherwise, the determination of the front and rear ends of the pointer fails.

[0029] Further, the specific content of the step S4 is as follows:

[0030] Calculate the angle α through the slope of the straight line;

[0031] If the abscissa of the front end of the pointer is less than the abscissa of the center point of the pointer and the slope of the straight line is less than 0, then the pointer rotation angle = 90 - α;

[0032] If the abscissa of the front end of the pointer is less than the abscissa of the center point of the pointer and the slope of the straight line is greater than or equal to 0, then the pointer rotation angle = 90 + α;

[0033] If the abscissa of the front end of the pointer is greater than or equal to the abscissa of the center point of the pointer and the slope of the straight line is less than 0, then the pointer rotation angle = 270 - α;

[0034] Otherwise, the pointer rotation angle = 270 + α.

[0035] Beneficial Effects

[0036] Compared with the prior art, the present invention has the following advantages:

[0037] 1. Compared with manual detection, the present invention can detect continuously for 24 hours and can feedback instrument information in time.

[0038] 2. The present invention can realize remote automatic monitoring, save time and manpower, improve work efficiency, and ensure the personal safety of staff.

[0039] 3. The present invention is not affected by complex backgrounds, has strong robustness and high portability. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 Flowchart for using deep learning algorithm to perform signless localization on collected images;

[0041] Figure 2 Flowchart for obtaining the instrument pointer area for the thresholding method and morphological preprocessing method;

[0042] Figure 3 Flowchart for calculating the proportion of white pixels in the image and the distance from the center point to determine the front and rear ends of the pointer;

[0043] Figure 4 This is a schematic diagram of the instrument pointer;

[0044] Figure 5 This is a schematic diagram for judging the front and back ends of the pointer. DETAILED DESCRIPTION

[0045] The present invention will be further described below with reference to the specific embodiments in the accompanying drawings.

[0046] See also Figure 1-5 , an automatic positioning and identification method for an instrument applied to a wheeled inspection robot, comprising:

[0047] Step S1. Collect images of instrument detection points, perform non-signboard positioning through deep learning algorithms, and obtain instrument area ROI;

[0048] Step S2. Preprocess the instrument region roi using a threshold method and a morphological preprocessing method to obtain an instrument pointer region;

[0049] Step S3. Determine the front and rear ends of the pointer by comparing the number of pixels at both ends of the pointer area of ​​the instrument;

[0050] Step S4. Determine the rotation angle through the straight line slope of the pointer and the front vertex of the pointer, and finally calculate the reading.

[0051] Step S2 includes:

[0052] Step S21. Perform mean filtering and stretching on the instrument area roi, obtain the grayscale histogram of all pixels in the instrument area roi, obtain the trough grayscale value lowest_value, the peak grayscale value highest_value, the minimum grayscale value min_value with a non-zero number of pixels, and the maximum grayscale value max_value with a non-zero number of pixels;

[0053] Step S22. Using the valley gray value lowest_value as the initial value of the binarization threshold of the instrument region roi;

[0054] Step S23. Use the binary threshold as the threshold to threshold the instrument area roi to obtain a threshold image, eliminate noise interference through morphological processing, and count the actual number of pixels white_nums with a gray value of 255;

[0055] Step S24. If the ratio of the actual number white_nums to the number of pixels in the instrument area roi is less than the set threshold ratio_thresh, the binarization threshold is increased by (max_value-min_value)

[0056] *ratio_thresh+min_value, execute step S23; otherwise, assign the current binarization threshold to the filtering threshold;

[0057] Step S25. Perform morphological calculation on the threshold image obtained in step S23, remove interference noise, calculate the outline of the threshold image and the minimum area enclosing rectangle, and determine whether the aspect ratio and length of the longest side rectangle meet the requirements; if yes, take the longest side rectangle as the instrument pointer area and execute step S3; otherwise, execute step S26;

[0058] Step S26. Perform a secondary calculation to determine whether the second longest side rectangle meets the requirements. If it does, execute step S3; otherwise, execute step S1.

[0059] Whether the aspect ratio and length of the longest side rectangle meet the requirements is determined as follows: the aspect ratio of the longest side rectangle is greater than 4 and the length of the longest side rectangle is greater than the instrument area roi*0.66.

[0060] The secondary calculation is as follows: if at least one of the aspect ratios of the second longest side rectangle and the longest side rectangle is greater than 3.5, compare the aspect ratios of the two rectangles and select the rectangle with the larger aspect ratio as the instrument pointer area; otherwise; if at least one of the lengths of the second longest side rectangle and the longest side rectangle is greater than the instrument area roi*0.5, compare the aspect ratios of the two rectangles and select the larger one as the instrument pointer area.

[0061] Set the threshold ratio_thresh to 0.035 to 0.05.

[0062] Step S3 includes:

[0063] Step S31. Fit a straight line based on the pointer pixel points within the instrument pointer area;

[0064] Step S32. Obtain the four vertices P0, P1, P2, P3 of the two short sides of the instrument pointer area, and the midpoints centerP2, centerP3 of the two long sides. The rectangle formed by P0, P1, centerP2, centerP3 is denoted as the first end, and the rectangle formed by P2, P3, centerP2, centerP3 is denoted as the second end;

[0065] Step S33. Obtain the intersection points X1 of the straight line and the short side of the first end, the intersection points X2 of the straight line and the short side of the second end, the midpoint centerP0 of the short side of the first end, and the midpoint centerP1 of the short side of the second end; Count the number of white pixel points in the first end, denoted as whiteNum1, and count the number of white pixel points in the second end, denoted as whiteNum2;

[0066] Step S34. Calculate the distance from centerP0 to the image center centerP of the instrument pointer area, denoted as len1, and calculate the distance from centerP1 to the image center centerP of the instrument pointer area, denoted as len2;

[0067] If len1 > 2 * len2, calculate the distance D1 between X1 and centerP0, and calculate the distance D2 between X2 and centerP0; If D1 < D2, then take X1 as the front end of the pointer; If D1 > D2, then take X2 as the front end of the pointer;

[0068] If len2 > 2 * len1, calculate the distance D3 between X1 and centerP1, and calculate the distance D4 between X2 and centerP1; If D3 < D4, then take X1 as the front end of the pointer; If D3 > D4, then take X2 as the front end of the pointer;

[0069] If whiteNum1 < whiteNum2, then take X1 as the front end of the pointer;

[0070] If whiteNum1 > whiteNum2, then take X2 as the front end of the pointer;

[0071] Otherwise, the determination of the front and rear ends of the pointer fails.

[0072] Step S4 is specifically:

[0073] Calculate the angle α through the slope of the straight line;

[0074] If the horizontal coordinate of the front end of the pointer is less than the horizontal coordinate of the center point of the pointer and the slope of the straight line is less than 0, the pointer rotation angle = 90-α;

[0075] If the horizontal coordinate of the front end of the pointer is less than the horizontal coordinate of the center point of the pointer and the slope of the straight line is greater than or equal to 0, the pointer rotation angle = 90 + α;

[0076] If the horizontal coordinate of the front end of the pointer is greater than or equal to the horizontal coordinate of the center point of the pointer and the slope of the straight line is less than 0, the pointer rotation angle = 270-α;

[0077] Otherwise, the pointer rotation angle = 270 + α.

[0078] The above are only preferred embodiments of the present invention. It should be pointed out that for those skilled in the art, several modifications and improvements can be made without departing from the structure of the present invention, which will not affect the effect of the implementation of the present invention and the practicality of the patent.

Claims

1. An automatic instrument positioning and identification method applied to a wheeled inspection robot, characterized in that: include: Step S1. Collect images of instrument detection points, perform signless positioning through deep learning algorithms, and obtain instrument areas; Step S2. Preprocessing the instrument area using a threshold method and a morphological preprocessing method to obtain an instrument pointer area; Step S3. Determine the front and rear ends of the pointer by comparing the number of pixels at both ends of the instrument pointer area; Step S4. Determine the rotation angle by the slope of the straight line of the pointer and the front vertex of the pointer, and finally calculate the reading; The step S2 comprises: Step S21. Perform mean filtering and stretching on the instrument area, obtain the grayscale histogram of all pixels in the instrument area, obtain the trough grayscale value, the peak grayscale value, the minimum grayscale value of the number of non-zero pixels, and the maximum grayscale value of the number of non-zero pixels; Step S22. Using the trough grayscale value as the initial value of the binarization threshold of the instrument area; Step S23. Using the binary threshold as a threshold to threshold the instrument area, obtain a threshold image, eliminate noise interference through morphological processing, and count the actual number of pixels with a gray value of 255; Step S24. If the ratio of the actual number to the number of pixels in the instrument area is less than the set threshold, the binarization threshold is increased by (maximum grayscale value - minimum grayscale value) * set threshold + minimum grayscale value, and step S23 is executed; Step S25. Perform morphological calculation on the threshold image obtained in step S23, remove interference noise, calculate the outline of the threshold image and the minimum area enclosing rectangle, and determine whether the aspect ratio and length of the longest side rectangle meet the requirements; if yes, take the longest side rectangle as the instrument pointer area and execute step S3; otherwise, execute step S26; Step S26. Perform secondary calculation to determine whether the second longest side rectangle meets the requirements. If yes, execute step S3; otherwise, execute step S1; Determine whether the aspect ratio and length of the longest side rectangle meet the requirements: the aspect ratio of the longest side rectangle is greater than 4 and the length of the longest side rectangle is greater than the instrument area * 0.66; The secondary calculation is as follows: if at least one of the length-to-width ratios of the second longest side rectangle and the longest side rectangle is greater than 3.5, compare the length-to-width ratios of the two rectangles and select the rectangle with the larger length-to-width ratio as the instrument pointer area; otherwise; if at least one of the lengths of the second longest side rectangle and the longest side rectangle is greater than the instrument area * 0.5, compare the length-to-width ratios of the two rectangles and select the larger one as the instrument pointer area; The step S3 comprises: Step S31. fitting a straight line according to the pointer pixel points in the instrument pointer area; Step S32. Obtain four vertices P0, P1, P2, and P3 of the two short sides of the instrument pointer area, and the midpoints centerP2 and centerP3 of the two long sides, and record the rectangle formed by P0, P1, centerP2, and centerP3 as the first end, and record the rectangle formed by P2, P3, centerP2, and centerP3 as the second end; Step S33. Obtain the intersection point X1 of the straight line and the short side of the first end, the intersection point X2 of the straight line and the short side of the second end, the midpoint centerP0 of the short side of the first end, and the midpoint centerP1 of the short side of the second end; count the number of white pixel points at the first end, denoted as whiteNum1, and count the number of white pixel points at the second end, denoted as whiteNum2; Step S34. Calculate the distance from centerP0 to the image center centerP of the instrument pointer area, denoted as len1, and calculate the distance from centerP1 to the image center centerP of the instrument pointer area, denoted as len2; If len1 > 2 * len2, calculate the distance D1 between X1 and centerP0, and calculate the distance D2 between X2 and centerP0; if D1 < D2, then take X1 as the front end of the pointer; if D1 > D2, then take X2 as the front end of the pointer; If len2 > 2 * len1, calculate the distance D3 between X1 and centerP1, and calculate the distance D4 between X2 and centerP1; if D3 < D4, then take X1 as the front end of the pointer; if D3 > D4, then take X2 as the front end of the pointer; If whiteNum1 < whiteNum2, then take X1 as the front end of the pointer; If whiteNum1 > whiteNum2, then take X2 as the front end of the pointer; Otherwise, the determination of the front and rear ends of the pointer fails.

2. The automatic positioning and identification method for an instrument applied to a wheeled inspection robot according to claim 1 is characterized in that: The set threshold is 0.035 - 0.

05.

3. The automatic instrument positioning and identification method for a wheeled inspection robot according to claim 1 is characterized in that: The specific content of the said Step S4 is as follows: Calculate the angle α through the slope of the straight line; If the abscissa of the front end of the pointer is less than the abscissa of the center point of the pointer and the slope of the straight line is less than 0, then the pointer rotation angle = 90 - α; If the abscissa of the front end of the pointer is less than the abscissa of the center point of the pointer and the slope of the straight line is greater than or equal to 0, then the pointer rotation angle = 90 + α; If the abscissa of the front end of the pointer is greater than or equal to the abscissa of the center point of the pointer and the slope of the straight line is less than 0, then the pointer rotation angle = 270 - α; Otherwise, the pointer rotation angle = 270 + α.

Images