A pointer instrument identification and enhanced detection method

Abstract

This invention discloses a method for enhanced detection and identification of pointer instruments. The method includes a point image entry module, a parameter debugging and selection module, and a recognition module. The image entry module is used to obtain a template photo, the parameter debugging and selection module is used to identify various data of the pointer instrument based on the obtained template photo, and the recognition module is used to obtain image information of the target device captured by an inspection robot and perform feature matching between the target device image information and the template photo. The present invention's combined detection of instrument pointers and dial targets, based on a calibrated contour frame, breaks away from a single traditional algorithm, greatly improving detection accuracy and partially anti-interference capabilities.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a pointer instrument recognition enhancement detection method. Background Art

[0002] Current methods for detecting and recognizing instrument scale pointers include traditional image recognition techniques and deep learning-based recognition methods. Traditional image recognition methods include template matching, maximum connected component, silhouette, least squares fitting, line drawing, and region growing. Traditional methods are then applied within a rectangular frame to detect scale readings and pointer angles. Deep learning-based network model training methods offer advantages such as interference immunity and simplicity. However, they require a certain amount of differentiated data samples; a large number of differentiated data samples can improve recognition accuracy. In common industrial scenarios, some abnormal meter samples are difficult to obtain. End-to-end recognition methods based on deep learning have limitations in practical applications due to the black-box effect of deep networks, data collection issues in real-world production environments, and the long tail effect of data samples. Traditional methods manually extract key information by analyzing a small number of images. For example, the commonly used feature template matching localization method has poor robustness in practical applications and performs poorly for images of varying resolutions. Using a novel matching network can improve matching stability. Least squares fitting and line drawing methods, when used alone, are generally not robust in noisy images. Traditional methods require the use of a very general parameter extraction design method to reduce the impact of different noises by configuring a set of parameters. Summary of the Invention

[0003] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0004] The present invention provides a pointer instrument identification enhancement detection method,

[0005] The detection method includes a point image entry module, a parameter debugging and selection module, and an identification module. The image entry module is used to obtain a template photo, the parameter debugging and selection module is used to identify various data of the pointer instrument in the obtained template photo, and the identification module is used to obtain the target device image information taken by the inspection robot, and perform feature matching between the target device image information and the template photo.

[0006] Optionally, the various data of the template photo include instrument type, pointer length, and image corrosion parameters. The various data of the template photo are stored in the database storage module. The various data of the template photo are stored in the database storage module using ROS robot topic messages or GRPC remote calls.

[0007] Optionally, the parameter debugging and selection module is used to view the results of the captured template image and the binarization process image, and return the adjusted meter parameters based on the readings and the binarization image.

[0008] Optionally, when performing an inspection task, after the robot or camera reaches a preset point, it takes a picture of the target device after positioning, and sends three pieces of information, namely, the preset template picture of the point, the recognition parameter information, and the picture taken of the point, to the recognition module. The recognition module is used to perform feature matching between the target device picture information and the template photo.

[0009] Optionally, the identification module includes the following process:

[0010] S1: Identify and correct the position of the captured image, using feature key point matching models including traditional sift / lift / superpoint traditional feature points and deep matching network models to obtain the affine transformation matrix from the template image to the captured image;

[0011] S2: Analyze the parameter data set and obtain the calibration parameter information of the captured image from the affine transformation matrix;

[0012] S3: Select an optional recognition area in the captured recognition image, and find a binary image of the pointer area based on the calibrated center point and the recognition scale point information, with the center point as the center and the distance from the scale information to the center point as the radius.

[0013] Optionally, the parameter data includes template image meter position, straight line selection parameters, center point, scale point, reading selection, and corrosion parameter information.

[0014] Optionally, finding the binarized image of the area where the pointer is located includes the following steps:

[0015] The first step is to use the Hough line detection method to find all pointers that meet the line parameter length, and calculate the distance from the line to the center point, filtering out lines with too large a distance;

[0016] In the second step, if an incorrect pointer line is found in the image in the test result, the recognition module reads the calibration area, removes the influence of the interference area, and finds a straight line in the specified recognition area that meets the distance and length to the center of the circle. If no straight line is found, jump to the third step;

[0017] The third step is to divide the identification selection into equal distances according to the scale value in the specified identification area in the second step. In order to improve the accuracy, the number of identification area divisions is usually equal to twice the minimum scale number. The formula is as follows: the maximum reading scale, the unit scale value is 0.1, the reading accuracy is 0.05, num = scale / 0.1*2, and the segmentation area is iteratively selected in sequence with a scale value of 0.05 to obtain a segmentation area set. In the segmentation selection set, the contour area in each segmentation area is calculated respectively, and the contour area is fitted into an outer rectangular frame. If the aspect ratio of the outer rectangular frame is greater than the threshold, and the distance from the straight line of the contour area fitting to the center origin is less than the threshold, then the straight line is the pointer reading. If the conditions are not met, use image enhancement or restricted area image enhancement method to find the segmentation area with the largest contour area in the segmentation area set. The reading value corresponding to the segmentation area is the maximum probability pointer area.

[0018] Beneficial effects of the present invention

[0019] The invention achieves joint detection of instrument pointers and dial targets based on a calibrated contour frame, breaking away from a single traditional algorithm and greatly improving detection accuracy and partial anti-interference capability. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 Schematic diagram of the identification module steps of the present invention.

[0021] Figure 2 This is a schematic diagram of the intermediate result of binarization of the meter image of the present invention. DETAILED DESCRIPTION

[0022] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0023] Example

[0024] like Figure 1-Figure 2 As shown, the present invention provides a pointer instrument identification enhancement detection method,

[0025] The detection method includes a point image entry module, a parameter debugging and selection module, and an identification module. The image entry module is used to obtain a template photo. The parameter debugging and selection module is used to identify various data of the pointer instrument for the obtained template photo. The identification module is used to obtain the target device image information taken by the inspection robot and perform feature matching on the target device image information and the template photo. The various data of the template photo include instrument type, pointer length, and image corrosion parameters. The various data of the template photo are stored in the database storage module. The various data of the template photo are stored in the database storage module using ROS robot topic messages or GRPC remote calls. The parameter debugging and selection module is used to view the results of the template image taken and the binarization process image, and adjust the meter parameters according to the readings and the binarization image. When executing the inspection task, after the robot or camera reaches the preset point, it takes the target device image information after positioning, and sends the point preset template image, recognition parameter information, and point shot image to the identification module. The identification module is used to perform feature matching on the target device image information and the template photo.

[0026] The identification module includes the following processes:

[0027] S1: Identify and correct the position of the captured image, using feature key point matching models including traditional sift / lift / superpoint traditional feature points and deep matching network models to obtain the affine transformation matrix from the template image to the captured image;

[0028] S2: Analyze the parameter data set, including the template image meter position, line selection parameters, center point, scale point, reading selection, and corrosion parameter information, and obtain the calibration parameter information of the captured image from the affine transformation matrix;

[0029] S3: Select an optional recognition area in the captured recognition image, and find the binary image of the pointer area based on the calibrated center point and the recognition scale point information, with the center point as the center and the distance from the scale information to the center point as the radius.

[0030] The identification module also includes the following steps:

[0031] S101: Online shooting of pointer meter of substation pointer type;

[0032] S102: parsing the template calibration data and correcting the calibration data according to the template image;

[0033] S103: Step 1: Use straight line detection to find clearly visible straight line segments;

[0034] S104: Step 2: The recognition module reads the calibration area to remove the influence of the interference area;

[0035] S105: In the designated recognition area, select equidistant segmentation based on the scale value. The aspect ratio of the circumscribed rectangle is greater than the threshold, and the segmentation area with the largest contour area in the segmentation area set is found;

[0036] The method of finding the binary image of the area where the pointer is located comprises the following steps:

[0037] The first step is to use the Hough line detection method to find all pointers that meet the line parameter length, and calculate the distance from the line to the center point, filtering out lines with too large a distance;

[0038] In the second step, if an incorrect pointer line is found in the image in the test result, the recognition module reads the calibration area, removes the influence of the interference area, and finds a straight line in the specified recognition area that meets the distance and length to the center of the circle. If no straight line is found, jump to the third step;

[0039] The third step is to divide the identification selection into equal distances according to the scale value in the specified identification area in the second step. In order to improve the accuracy, the number of identification area segmentations is usually equal to twice the minimum scale number. The formula is as follows: the maximum reading scale, the unit scale value is 0.1, the reading accuracy is 0.05, num = scale / 0.1*2, and the segmentation area is iteratively selected in sequence with a scale value of 0.05 to obtain a segmentation area set. In the segmentation selection set, the contour area in each segmentation area is calculated respectively, and the contour area is fitted into an outer rectangular frame. If the aspect ratio of the outer rectangular frame is greater than the threshold, and the distance from the straight line of the contour area fitting to the center origin is less than the threshold, then the straight line is the pointer reading. If the conditions are not met, use image enhancement or restricted area image enhancement method to find the segmentation area with the largest contour area in the segmentation area set. The reading value corresponding to the segmentation area is the maximum probability pointer area.

[0040] The invention achieves joint detection of instrument pointers and dial targets based on a calibrated contour frame, breaking away from a single traditional algorithm and greatly improving detection accuracy and partial anti-interference capability.

[0041] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A pointer instrument identification enhancement detection method, characterized in that: The detection method includes a point image entry module, a parameter debugging and selection module, and an identification module. The point image entry module is used to obtain a template photo. The parameter debugging and selection module is used to identify various data of the pointer instrument based on the obtained template photo. The identification module is used to obtain the image information of the target device taken by the inspection robot and perform feature matching between the target device image information and the template photo. When the robot or camera reaches the preset point during the inspection, it will take a picture of the target device after positioning, and send the preset template photo of the point, recognition parameter information and the picture of the point to the recognition module. The recognition module is used to perform feature matching between the target device picture information and the template photo; The identification module includes the following processes: S1: Identify the position of the point-captured image and correct it. Use the feature key point matching model, including traditional sift / lift / superpoint traditional feature points and deep matching network model, to obtain the affine transformation matrix from the point preset template photo to the point-captured image. S2: Analyze the parameter data set and obtain the calibration parameter information of the point-captured image from the affine transformation matrix; S3: Select an optional recognition area in the point-captured image, and find the binary image of the pointer area based on the calibrated center point and the identification scale point information, with the center point as the center and the distance from the scale information to the center point as the radius; The method of finding the binary image of the area where the pointer is located comprises the following steps: The first step is to use the Hough line detection method to find all pointers that meet the line parameter length, and calculate the distance from the line to the center point, filtering out lines with too large a distance; In the second step, if an incorrect pointer line is found in the image in the test result, the recognition module reads the calibration area, removes the influence of the interference area, and finds a straight line in the specified recognition area that meets the distance and length to the center of the circle. If no straight line is found, jump to the third step; The third step is to divide the recognition area into equal distances according to the scale value in the specified recognition area in the second step. In order to improve the accuracy, the number of recognition area divisions is usually equal to twice the minimum scale number. The formula is as follows: the maximum reading scale, the unit scale value is 0.1, the reading accuracy is 0.05, num = scale / 0.1 * 2, and the segmented areas are iteratively selected in sequence with a scale value of 0.05 to obtain a segmented area set. In the segmented area set, the contour area in each segmented area is calculated respectively, and the contour area is fitted into an outer rectangular frame. If the aspect ratio of the outer rectangular frame is greater than the threshold and the distance from the straight line of the contour area fitting to the center of the circle is less than the threshold, then the straight line is the pointer reading. If the conditions are not met, the image enhancement method is used to find the segmented area of ​​the largest contour area in the segmented area set. The reading value corresponding to the segmented area is the pointer reading value of the pointer area with the maximum probability.

2. The pointer instrument recognition enhancement detection method according to claim 1, characterized in that: The various data of the template photo include instrument type, pointer length, and image corrosion parameters. The various data of the template photo are stored in the database storage module. The various data of the template photo are stored in the database storage module using ROS robot topic messages or GRPC remote calls.

3. The pointer instrument recognition enhancement detection method according to claim 1, characterized in that: The parameter debugging and selection module is used to view the results of the template photo taken and the binarization process picture, and return the adjustment meter parameters according to the readings and the binarization picture.

4. The pointer instrument identification enhancement detection method according to claim 1, characterized in that: The parameter data includes template photo meter position, straight line selection parameters, center point, scale point, reading selection, and corrosion parameter information.

Images