An instrument reading recognition method based on a quadruped robot

Abstract

The application discloses a kind of instrument reading identification method based on four-legged robot, comprising the following steps: step S1, point calibration and image acquisition;Step S2, image pre-processing;Step S3, multi-type instrument identification;Step S4, multidimensional verification;Step S5, result output and alarm.The present application solves the existing problems of manual and remote transmission, uses four-legged robot to replace manual meter reading, and automatically interfaces operation and maintenance management platform, realizes instrument table account automatic update and abnormal alarm, discovers remote transmission fault of instrument itself in time, improves the integrity and accuracy of sewage plant instrument data.

Description

Technical Field

[0001] This invention relates to intelligent inspection and machine vision fault detection technology for wastewater treatment equipment, and more specifically, to a method for recognizing instrument readings based on a quadruped robot. Background Technology

[0002] Commonly used instruments in wastewater treatment plants include water quality instruments, gas instruments, level gauges, and flow meters. These instruments play a crucial role in flow measurement, process control, and safety management in wastewater treatment plants. Therefore, instrument reading identification is a core part of the daily work of wastewater treatment plants.

[0003] Currently, industrial instrument readings mainly rely on manual inspection and identification (meter reading) and PLC data acquisition (meter remote transmission). Manual meter reading is inefficient, has poor timeliness, is prone to large subjective errors (human transcription errors), and meter reading at high / low installation locations poses safety risks such as falls from heights and confined space operations. On the other hand, meter remote transmission mainly relies on sensors and communication modules, which are susceptible to data drift due to environmental electromagnetic interference, have long fault detection times, and are prone to data acquisition and transmission failures. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the purpose of this invention is to provide a method for identifying meter readings based on a quadruped robot. This method solves the problems associated with existing manual and remote meter reading methods. It uses a quadruped robot to replace manual meter reading and automatically connects to the operation and maintenance management platform to achieve automatic updates of meter records and anomaly alarms. It also enables timely detection of remote transmission faults in the meters themselves, thereby improving the integrity and accuracy of meter data in wastewater treatment plants.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A method for identifying instrument readings based on a quadruped robot includes the following steps:

[0007] Step S1: Point calibration and image acquisition;

[0008] Step S2, image preprocessing;

[0009] Step S3, multi-type instrument identification;

[0010] Step S4, multi-dimensional verification;

[0011] Step S5: Output the results and issue alarms.

[0012] Preferably, step S1 specifically includes:

[0013] Information on all instrument locations must be entered in advance;

[0014] After the quadruped robot reaches the instrument location, it automatically adjusts its body posture and the angle of the camera, turns on adaptive lighting, and automatically adjusts the exposure parameters according to the ambient lighting conditions to take three instrument images with different exposure parameters.

[0015] Preferably, the information of the instrument location includes the instrument type, range, installation location, and shooting parameters;

[0016] The quadruped robot's body posture includes a pitch angle adjustment range of -30° to 60° and a horizontal rotation range of 0° to 360°.

[0017] The exposure parameters for the three instrument images are -2EV, 0EV, and +2EV.

[0018] Preferably, step S2 specifically includes:

[0019] The three instrument images with different exposure parameters are HDR fused, and then perspective correction, reflection suppression, and contrast enhancement are performed to obtain an instrument image viewed from the front.

[0020] Preferably, the perspective correction error is ≤0.5°;

[0021] The reflection suppression employs a polarization filtering algorithm, achieving a reflection suppression rate of ≥85%.

[0022] The time required to obtain the instrument image at a normal viewing angle is ≤200 milliseconds.

[0023] Preferably, step S3 specifically includes:

[0024] The instrument image viewed from the front is input into the dual-branch recognition model. The instrument type recognition branch of the dual-branch recognition model uses a deep learning classification network to classify and identify the instrument type in the instrument image and output the instrument type label.

[0025] The instrument type label is passed to the dial detection and segmentation branch in the dual-branch recognition model to locate and detect the dial area of ​​the instrument in the instrument image, output the bounding box coordinates of the dial area, and crop the dial area from the original instrument image accordingly; and perform geometric correction on the instrument image.

[0026] The instrument type adaptation branch in the dual-branch recognition model receives the instrument type label and selects the corresponding recognition strategy and algorithm parameters by the reading calculation branch in the dual-branch recognition model accordingly.

[0027] The multi-verification logic module in the dual-branch recognition model receives the original reading from the instrument type adaptation branch, as well as the correction parameters and segmentation results output by the dial detection segmentation branch. After performing multi-level verification, it outputs the final reading.

[0028] Preferably, the instrument type label includes pointer instruments and digital instruments;

[0029] The dial detection and segmentation branch uses an image segmentation network to perform semantic segmentation on the dial region, segmenting out the key elements in the dial:

[0030] For pointer-type instruments, the pointer mask area and the scale mask area are separated;

[0031] For digital instruments, the digital display area is divided;

[0032] Geometric correction of the instrument image specifically includes: ellipse fitting of the dial mask image, detection of the smallest circumcircle of the dial and calculation of the perspective transformation matrix; and preliminary correction of the instrument image using the perspective transformation matrix to eliminate tilt distortion caused by the shooting angle.

[0033] For pointer-type instruments, the reading calculation process is as follows: Based on the pointer mask area output by the segmented branches of the dial detection, the least squares method is used to perform straight line fitting on the pointer area to obtain the pointer fitting line and its slope; based on the corrected dial image, the scale mask area is extracted, and the distribution range of the scale and the angular interval between adjacent scales are calculated; the range digital information on the dial is identified to obtain the maximum and minimum range of the instrument; then the deflection angle between the pointer fitting line and the dial coordinate system is calculated, and the pointer deflection angle is converted into the instrument reading by combining the range information;

[0034] For digital instruments, the reading calculation process is as follows: Based on the digital display area output by the segmented branches of the dial detection, an OCR recognition network is used to recognize the digital characters in the display area;

[0035] The verification of the multi-verification logic module includes one or more of the following: angle consistency verification, OCR confidence verification, attitude distortion tolerance verification, time continuity verification, and multi-reading fusion verification.

[0036] Preferably, step S4 specifically includes:

[0037] Verify whether the reading is within the instrument's range; if it exceeds the instrument's range, the recognition is deemed to have failed.

[0038] The recognition results of the three frames are smoothed, and results with fluctuations exceeding max_fluctuation are discarded.

[0039] Preferably, step S5 specifically includes:

[0040] The recognition results are output in JSON format;

[0041] If the reading exceeds the process threshold or deviates too much from the remote data, an alarm will be triggered and pushed to the operation and maintenance platform simultaneously, along with an annotated instrument image, and the alarm will be displayed in a pop-up window on the platform.

[0042] The identified data is automatically stored in the instrument log for use in operational trend analysis and equipment life prediction.

[0043] Preferably, the JSON format includes the instrument's unique identifier, acquisition time, original image path, identification reading, unit, confidence level, and abnormal status.

[0044] This invention provides a method for meter reading recognition based on a quadruped robot. The quadruped robot replaces manual meter reading and can adapt to complex plant environments with water accumulation, debris, steps, etc. It has a climbing ability of ≥30° and an obstacle-crossing height of ≥15cm. It completes meter inspections throughout the plant at a preset frequency, with a time of ≤30 minutes for inspecting 100 meters in a single plant, improving meter reading efficiency by over 80% (currently, manual meter reading for 100 meters takes approximately 150 minutes, averaging 1.5 minutes per meter (walking and reading time)). This avoids the risks of personnel working at heights or in confined spaces, achieving unmanned inspection. It automatically connects to the operation and maintenance management platform, enabling automatic updates of meter records and anomaly alarms, timely detection of remote transmission faults in the meters, and improving the integrity and accuracy of wastewater treatment plant meter data. Using quadruped robots to identify instrument readings effectively solves the pain points of manual meter reading and remote instrument transmission in existing industrial systems. It can adapt to complex plant environments and various types of instruments, achieving high-accuracy reading identification, automatic data connection to the platform, automatic updating of instrument ledgers and anomaly alarms, significantly improving the integrity and accuracy of instrument data in wastewater treatment plants and the level of intelligence and automation in inspections. Attached Figure Description

[0045] Figure 1 This is a flowchart illustrating the instrument reading recognition method of the present invention;

[0046] Figure 2 This is a flowchart illustrating the dual-branch recognition model in the instrument reading recognition method of the present invention. Detailed Implementation

[0047] To better understand the above-mentioned technical solutions of the present invention, the technical solutions of the present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0048] Combination Figure 1 As shown, this invention provides a method for identifying instrument readings based on a quadruped robot, comprising the following steps:

[0049] Step S1, point calibration and image acquisition, specifically includes:

[0050] Information for all instrument locations must be entered in advance, including instrument type, range, installation location, and shooting parameters.

[0051] After the quadruped robot reaches the instrument panel location, it automatically adjusts its body posture and camera angle, activates adaptive lighting, and automatically adjusts the exposure parameters according to the ambient lighting conditions to take three instrument panel images with different exposure parameters, ensuring that clear instrument panel details can be obtained under different lighting conditions.

[0052] The quadruped robot's body posture includes a pitch angle adjustment range of -30° to 60° and a horizontal rotation range of 0° to 360°.

[0053] The exposure parameters for the three instrument images are -2EV, 0EV, and +2EV.

[0054] The quadruped robot is equipped with a 12x optical zoom industrial camera (resolution 3840×2160, frame rate 30fps) and an adaptive RGBW fill light module (0-1000lm stepless adjustment, color rendering index Ra≥90). It automatically reaches the installation points of each instrument along a preset inspection path. Based on the installation height of the instrument (supports adaptive adaptation within the range of 0.5m~5m) and distance (0.3m~2m autofocus), it adaptively adjusts the camera's focal length and fill light intensity to obtain clear images of the instrument's front. This solves the problem of shooting instruments under different installation positions and lighting conditions. It can work stably under ambient temperatures of -10℃~55℃ and relative humidity ≤95% without condensation.

[0055] Step S2, image preprocessing, specifically includes:

[0056] HDR fusion is performed on three instrument images with different exposure parameters, followed by perspective correction, reflection suppression, and contrast enhancement to eliminate the effects of shooting angle deviation, ambient reflection, low light, and other factors, thus obtaining an instrument image from the front viewing angle.

[0057] The error of perspective correction is ≤0.5°.

[0058] The reflection suppression uses a polarization filtering algorithm, with a reflection suppression rate of ≥85%.

[0059] The time required to obtain an instrument image at a normal viewing angle is ≤200 milliseconds.

[0060] Step S3, multi-type instrument identification, combined with Figure 2 As shown, it specifically includes:

[0061] The instrument image at a frontal view angle is input into the dual-branch recognition model. The instrument type recognition branch of the dual-branch recognition model uses a deep learning classification network to classify and identify the instrument type in the instrument image and output the instrument type label.

[0062] The instrument type label includes pointer instruments and digital instruments; pointer instruments are further divided into subtypes such as circular pointer instruments and sector pointer instruments. The instrument type label will be passed to the instrument type adaptation branch for subsequent invocation of the corresponding identification strategy.

[0063] The instrument type label is passed to the dial detection and segmentation branch in the dual-branch recognition model. The target detection network (such as YOLOv8 or YOLOv12 network model) is used to locate and detect the dial area of ​​the instrument in the instrument image, output the bounding box coordinates of the dial area, and crop the dial area from the original instrument image accordingly.

[0064] The dial detection and segmentation branch uses an image segmentation network (such as U-Net, improved U-Net, or OCRNet) to perform semantic segmentation on the dial region, extracting the key elements within the dial:

[0065] For pointer-type instruments, the pointer mask area and the scale mask area are separated;

[0066] For digital instruments, the digital display area is divided;

[0067] The dial color band area is segmented to help determine the device's operating status.

[0068] After segmentation, the instrument image undergoes geometric correction. This correction includes: ellipse fitting to the dial mask image, detecting the smallest circumcircle of the dial, and calculating the perspective transformation matrix; using the perspective transformation matrix to perform preliminary correction of the instrument image, eliminating tilt distortion caused by the shooting angle; a secondary correction strategy can also be employed: first, perspective transformation tilt correction is performed, and then rotation correction is performed based on the symmetry features in the dial. The corrected instrument image serves as input for subsequent reading calculations and also provides correction parameters for use by the multi-verification logic module to ensure the accuracy of readings under tilted shooting conditions.

[0069] In the dual-branch recognition model, the instrument type adaptation branch receives the instrument type label and selects the corresponding recognition strategy and algorithm parameters based on this label from the reading calculation branch in the dual-branch recognition model.

[0070] For pointer-type instruments, the reading calculation process is as follows: Based on the pointer mask area output by the segmented branches of the dial detection, the least squares method is used to perform straight line fitting on the pointer area to obtain the pointer fitting line and its slope; based on the corrected dial image, the scale mask area is extracted, and the distribution range of the scale and the angular interval between adjacent scales are calculated; the range digital information on the dial is identified to obtain the maximum and minimum range of the instrument; then the deflection angle between the pointer fitting line and the dial coordinate system is calculated, and the pointer deflection angle is converted into the instrument reading by combining the range information; for multi-pointer instruments (including dual-pointer and triple-pointer instruments), different pointers are distinguished by the instance segmentation results of the pointer mask area (e.g., according to the difference in pointer color or length), the reading of each pointer is calculated separately, and the final reading is output according to the preset combination rules (e.g., the combination of long and short pointer readings into tens and units digits).

[0071] For digital instruments, the reading calculation process is as follows: Based on the digital display area output by the segmented branch of the dial detection, an OCR recognition network (such as PP-OCRv3 or CRNN network) is used to recognize the digital characters in the display area; if the numbers in the digital area have specific digit meanings according to their positional relationship (such as the tens digit and the units digit being located in different areas), the recognized numbers are recombined according to their positional relationship to obtain the complete instrument reading.

[0072] The multi-verification logic module in the dual-branch recognition model receives the raw reading from the instrument type adaptation branch, as well as the correction parameters and segmentation results output by the dial detection segmentation branch. After performing multi-level verification, it outputs the final reading.

[0073] The verification of the multi-verification logic module includes one or more of the following: angle consistency verification, OCR confidence verification, attitude distortion tolerance verification, time continuity verification, and multi-reading fusion verification.

[0074] (1) Angle consistency verification. When the instrument type is a pointer instrument, based on the scale mask area of ​​the output of the segmented branch of the dial detection, the range value and corresponding angle of the adjacent scale on both sides of the pointer are extracted, and the pointer deflection angle is verified to be between the minimum scale and the maximum scale. If the pointer angle exceeds the range, it is judged as an abnormal reading and recalculation is triggered.

[0075] (2) OCR Confidence Verification. When the instrument type is a digital instrument, the OCR recognition network outputs a confidence score for each recognized character. The verification module determines whether all confidence scores are higher than a preset threshold. If the confidence score of any character is lower than the threshold, the recognition result is deemed unreliable, triggering re-recognition or reporting an anomaly.

[0076] (3) Attitude distortion tolerance verification. Based on the correction parameters (including the distortion degree of the perspective transformation matrix) output by the dial detection segmentation branch, the tilt angle of the instrument image is calculated. If the tilt angle exceeds the preset tolerance threshold (e.g., 15°), the re-correction and recognition process is triggered to ensure the accuracy of the reading under tilt shooting conditions.

[0077] (4) Time continuity verification. Compare the current recognition result with the historical recognition results of the previous frame or multiple frames, and calculate the rate of change of the reading. If the rate of change exceeds the preset physical reasonable range (e.g., the pointer cannot make a large angle jump between adjacent frames), the current recognition result is determined to be abnormal, triggering re-recognition or reporting the abnormality.

[0078] (5) Multi-reading fusion verification. When the instrument is a multi-pointer instrument, the combined result of each pointer reading is verified for rationality, and it is determined whether the combined result is within the preset range and whether the relationship between the pointers conforms to physical constraints.

[0079] After passing the verification by the multi-verification logic module, the final reading is output. If any level of verification fails, one of the following operations is performed according to preset rules: discard the current recognition result and trigger a re-recognition process; mark the current recognition result as low confidence and attach verification failure information; or report the abnormal status.

[0080] The digital / LCD meter obtains readings through character segmentation and CRNN network recognition, supporting full character recognition including 0-9, A-Z, decimal point, and plus / minus sign. For situations where the meter dial is temporarily obstructed (e.g., ≤30% obstruction by rags or tools), a contextual information completion algorithm is employed, combining historical reading trends and process logic to calculate a reasonable reading range. In obstructed scenarios, the recognition accuracy can reach over 88%.

[0081] A dual-branch recognition model that integrates OCR and target detection is adopted and optimized for three common industrial instruments: digital meters, LCD screen meters, etc. For LCD screen meters, the improved CRNN network is used to recognize character content, which can adapt to complex scenarios such as dirt, reflection, and low light (illuminance ≤ 5 lux). Even in scenarios where the dial is covered with water vapor and dust thickness ≤ 2 mm, the recognition accuracy can still be maintained at over 85%.

[0082] A feature library for instrument types has been built, supporting custom configuration of instrument type, range, unit, and threshold parameters. Currently, feature templates for more than 120 common industrial instruments have been pre-configured, and the configuration time for new instrument templates is ≤5 minutes. During the recognition process, the system automatically verifies whether the reading exceeds the reasonable range of the instrument and filters out obviously abnormal recognition results.

[0083] Step S4, multi-dimensional verification, specifically includes:

[0084] Verify whether the reading is within the instrument's range; if it exceeds the instrument's range, the recognition is deemed to have failed.

[0085] The recognition results of 3 frames of images are smoothed (clear images from the front view, 3 frames are sufficient in normal configuration), and results with fluctuations exceeding max_fluctuation (maximum allowable fluctuation ratio, default 5%, which can be customized according to the instrument type) are discarded.

[0086] By introducing reading fluctuation verification logic, the recognition results of multiple consecutive frames of the same instrument are smoothed. The parameters read_smooth_switch (timing smoothing switch, enabled by default), smooth_frame_num (smoothing frame number, 5 frames by default, configurable range 3-10 frames), and max_fluctuation (maximum allowable fluctuation ratio, 5% by default, customizable according to instrument type) are used to filter out erroneous readings that jump instantaneously, thereby improving recognition stability. After smoothing, the reading fluctuation error is ≤2%.

[0087] Step S5, result output and alarm, specifically includes:

[0088] The recognition results are output in JSON format;

[0089] If the reading exceeds the process threshold or deviates too much from the remote data, an alarm will be triggered and pushed to the operation and maintenance platform simultaneously, along with an annotated instrument image, and the alarm will be displayed in a pop-up window on the platform.

[0090] The identified data is automatically stored in the instrument log for use in operational trend analysis and equipment life prediction.

[0091] The JSON format includes the instrument's unique identifier, acquisition time, original image path, recognition reading, unit, confidence level, and abnormal status.

[0092] Those skilled in the art should recognize that the above embodiments are merely illustrative of the present invention and are not intended to limit the present invention. Any variations or modifications to the above embodiments that are within the spirit and essence of the present invention will fall within the scope of the claims of the present invention.

Claims

1. A method for recognizing instrument readings based on a quadruped robot, characterized in that, Includes the following steps: Step S1: Point calibration and image acquisition; Step S2, image preprocessing; Step S3, multi-type instrument identification; Step S4, multi-dimensional verification; Step S5: Output the results and issue alarms.

2. The instrument reading recognition method based on a quadruped robot according to claim 1, characterized in that, Step S1 specifically includes: Information on all instrument locations must be entered in advance; After the quadruped robot reaches the instrument location, it automatically adjusts its body posture and the angle of the camera, turns on adaptive lighting, and automatically adjusts the exposure parameters according to the ambient lighting conditions to take three instrument images with different exposure parameters.

3. The instrument reading recognition method based on a quadruped robot according to claim 2, characterized in that: The information of the instrument location includes the instrument type, range, installation location, and shooting parameters; The quadruped robot's body posture includes a pitch angle adjustment range of -30° to 60° and a horizontal rotation range of 0° to 360°; The exposure parameters for the three instrument images are -2EV, 0EV, and +2EV.

4. The instrument reading recognition method based on a quadruped robot according to claim 2, characterized in that, Step S2 specifically includes: The three instrument images with different exposure parameters are HDR fused, and then perspective correction, reflection suppression, and contrast enhancement are performed to obtain an instrument image viewed from the front.

5. The instrument reading recognition method based on a quadruped robot according to claim 4, characterized in that: The error of the perspective correction is ≤0.5°; The reflection suppression employs a polarization filtering algorithm, achieving a reflection suppression rate of ≥85%. The time required to obtain the instrument image at a normal viewing angle is ≤200 milliseconds.

6. The instrument reading recognition method based on a quadruped robot according to claim 4, characterized in that, Step S3 specifically includes: The instrument image viewed from the front is input into the dual-branch recognition model. The instrument type recognition branch of the dual-branch recognition model uses a deep learning classification network to classify and identify the instrument type in the instrument image and output the instrument type label. The instrument type label is passed to the dial detection and segmentation branch in the dual-branch recognition model to locate and detect the dial area of ​​the instrument in the instrument image, output the bounding box coordinates of the dial area, and crop the dial area from the original instrument image accordingly; and perform geometric correction on the instrument image. The instrument type adaptation branch in the dual-branch recognition model receives the instrument type label and selects the corresponding recognition strategy and algorithm parameters by the reading calculation branch in the dual-branch recognition model accordingly. The multi-verification logic module in the dual-branch recognition model receives the original reading from the instrument type adaptation branch, as well as the correction parameters and segmentation results output by the dial detection segmentation branch. After performing multi-level verification, it outputs the final reading.

7. The instrument reading recognition method based on a quadruped robot according to claim 6, characterized in that: The instrument type label includes pointer instruments and digital instruments; The dial detection and segmentation branch uses an image segmentation network to perform semantic segmentation on the dial region, segmenting out the key elements in the dial: For pointer-type instruments, the pointer mask area and the scale mask area are separated; For digital instruments, the digital display area is divided; Geometric correction of the instrument image specifically includes: ellipse fitting of the dial mask image, detection of the smallest circumcircle of the dial and calculation of the perspective transformation matrix; and preliminary correction of the instrument image using the perspective transformation matrix to eliminate tilt distortion caused by the shooting angle. For pointer-type instruments, the reading calculation process is as follows: Based on the pointer mask area output by the segmented branches of the dial detection, the least squares method is used to perform a straight line fitting on the pointer area to obtain the pointer fitting line and its slope; based on the corrected dial image, the scale mask area is extracted, and the distribution range of the scale and the angular interval between adjacent scales are calculated; the range digital information on the dial is identified to obtain the maximum and minimum range of the instrument; then the deflection angle between the pointer fitting line and the dial coordinate system is calculated, and the pointer deflection angle is converted into the instrument reading by combining the range information; For digital instruments, the reading calculation process is as follows: Based on the digital display area output by the segmented branches of the dial detection, an OCR recognition network is used to recognize the digital characters in the display area; The verification of the multi-verification logic module includes one or more of the following: angle consistency verification, OCR confidence verification, attitude distortion tolerance verification, time continuity verification, and multi-reading fusion verification.

8. The instrument reading recognition method based on a quadruped robot according to claim 6, characterized in that, Step S4 specifically includes: Verify whether the reading is within the instrument's range; if it exceeds the instrument's range, the recognition is deemed to have failed. The recognition results of the three frames are smoothed, and results with fluctuations exceeding max_fluctuation are discarded.

9. The instrument reading recognition method based on a quadruped robot according to claim 8, characterized in that, Step S5 specifically includes: The recognition results are output in JSON format; If the reading exceeds the process threshold or deviates too much from the remote data, an alarm will be triggered and pushed to the operation and maintenance platform simultaneously, along with an annotated instrument image, and the alarm will be displayed in a pop-up window on the platform. The identified data is automatically stored in the instrument log for use in operational trend analysis and equipment life prediction.

10. The instrument reading recognition method based on a quadruped robot according to claim 9, characterized in that: The JSON format includes the instrument's unique identifier, acquisition time, original image path, recognition reading, unit, confidence level, and abnormal status.

Images