Instrument state monitoring method based on image recognition and storage medium

Abstract

The invention discloses an instrument state monitoring method based on image recognition and a storage medium, and the method comprises the following steps: S1, image collection and image preprocessing: collecting and preprocessing an instrument image; s2, canny edge detection: processing the preprocessed instrument image by adopting canny edge point detection to obtain an edge contour; s3, through Hough straight line detection, obtaining a Hough straight line in the image and end point coordinates of the Hough straight line; s4, screening the obtained Hough straight lines to obtain Hough straight lines corresponding to effective pointers; and S5, performing angle calculation based on the Hough straight line corresponding to the effective pointer, calculating the included angle between the Hough straight line and the horizontal plane, and judging the indication state of the instrument based on the calculated angle value. The method has the advantages that through self-adaptive edge detection and high-precision straight line recognition, accurate pointer recognition is achieved, and instrument state monitoring and result output are achieved based on the pointer straight line.

Description

Technical Field

[0001] This invention relates to the field of image recognition, and in particular to a method and storage medium for monitoring the status of instruments based on image recognition. Background Technology

[0002] In scenarios such as industrial smart grids and facility monitoring, especially for traditional instruments like pointer meters, accurately and in real-time acquiring their status (readings, pointer angles, switching status, fault status, etc.) is fundamental to achieving safe operation, energy efficiency management, and fault early warning. Therefore, how to continuously monitor a large number of distributed instruments in a low-cost and highly reliable manner has become a key issue for enterprises in achieving refined management and stable system operation.

[0003] The current mainstream method is still manual inspection and reading, but this method has the following common drawbacks:

[0004] (1) Cost and efficiency: The large number of instruments, their dispersed locations, and harsh environments (high temperature / high humidity / high altitude / toxic) result in high labor costs and low efficiency for inspections;

[0005] (2) Accuracy and reliability: Factors such as lighting, viewing angle, differences in operator experience and fatigue make manual readings prone to errors and omissions;

[0006] (3) Insufficient real-time performance: Fixed-cycle inspections are unable to capture instantaneous anomalies or sudden failures in a timely manner;

[0007] (4) Safety risks: There are potential safety hazards in manual entry into dangerous areas;

[0008] (5) Inefficient data management: Paper or simple digital entry methods result in low efficiency in data storage, traceability, analysis and linkage alarm.

[0009] To address the issue of manual inspection, existing technologies attempt to use wired sensors (such as RS485, ModBus, etc.) to directly access the instrument output signals for data acquisition. However, this approach has significant limitations: (1) High difficulty and cost in deployment and modification: For a large number of instruments already in use without pre-installed communication interfaces (especially old pointer instruments), wiring construction is complex, modification costs are high, and it may even be infeasible in some scenarios; (2) Poor instrument compatibility: The interface standards of various instruments are inconsistent, making it difficult to unify access; (3) Limited coverage: The limitations of physical wiring make it difficult to cover all the instrument points that need to be monitored.

[0010] For pointer-type instruments, determining the current pointer state quickly and accurately is a technical problem that needs to be solved. Although traditional solutions also utilize image recognition, the accuracy and reliability of the recognition cannot meet the requirements. Summary of the Invention

[0011] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method and storage medium for monitoring the status of instruments based on image recognition. Through adaptive edge detection and high-precision line recognition, accurate pointer recognition is achieved, and the monitoring of instrument status and result output are realized based on the pointer line.

[0012] To achieve the above objectives, the technical solution adopted by the present invention is: a method for monitoring the status of instruments based on image recognition, comprising the following steps:

[0013] S1. Image acquisition and image preprocessing: Acquire instrument images and perform preprocessing.

[0014] S2, Canny edge detection: Canny edge point detection is used to process the preprocessed instrument image to obtain the edge contour;

[0015] S3. Obtain the Hough line and its endpoint coordinates in the image through Hough line detection;

[0016] S4. Filter the obtained Hough lines to obtain the Hough lines corresponding to the valid pointers.

[0017] S5. Calculate the angle based on the Hough line corresponding to the valid pointer, calculate the angle between the Hough line and the horizontal plane, and determine the indication status of the instrument based on the calculated angle value.

[0018] In step S1, sample images of the instrument are obtained by frame-sampling at a set frequency through the monitoring video of the instrument; then the ROI area of ​​the instrument is obtained by cropping.

[0019] In step S2, the threshold for the Canny edge detection algorithm is obtained through dynamic calculation. The dynamic threshold calculation for Canny edge detection includes: converting the cropped image to a grayscale image, calculating the grayscale mean in the grayscale image, and calculating the Canny dual threshold based on the grayscale mean.

[0020] The dynamic threshold setting for Canny edge detection includes: reading the instrument image and converting it to grayscale; using the Canny edge detection operator; setting the initial values ​​of the low threshold and high threshold to 50 and 250 respectively; and recording the number L of edge points detected based on these parameter values; when L < the set threshold, dynamically adjusting the low threshold and high threshold using adaptive parameters for different images captured by different cameras; and iteratively calculating until L detected by Canny > the set threshold; where the low threshold is used for edge connection, and pixels with gradient values ​​greater than the low threshold are considered potential edge points; the high threshold is used to determine strong edges, and all pixels with gradient values ​​greater than the high threshold are considered determined edge points.

[0021] Step S3 includes: determining the five parameter combinations for Hough line detection using a grid search method, and then detecting the Hough line using the parameter-determined Hough line detection method.

[0022] The method for determining the combination of five parameters for Hough line detection includes: the five parameters for Hough line detection are rho, theta, threshold, minLineLength, and maxLineGap.

[0023] Define the candidate value ranges for rho, theta, threshold, minLineLength, and maxLineGap, and randomly select values ​​within the range to form multiple parameter combinations;

[0024] For each set of parameters, the Hough probability line transformation function HoughLinesP is called to obtain the candidate line set lines;

[0025] For a non-empty set of lines, calculate the length of each line segment, count the length of the longest line segment and the number of line segments, and combine this with the ROI dimensions to calculate the comprehensive evaluation metric, score.

[0026] score = (longest length / actual dial side length) × exp(-0.5 × (number of line segments - 1));

[0027] The parameter combination with the highest score is selected as the optimal Hough detection parameters for the current instrument scene and stored for use in the detection of straight line segments in the current and subsequent images.

[0028] In step S4, the coordinates of the endpoints of the detected straight lines are obtained, the angle α between each straight line and the horizontal line is calculated, and a filtering range is set according to the working characteristics of the instrument. Hough lines whose angle is within the filtering range are selected as the lines corresponding to the valid pointer.

[0029] In step S5, the Hough line angle α_current corresponding to the valid pointer identified in the current frame image is smoothed to obtain the smoothed angle α_filtered. The current state of the instrument is determined by comparing the angle α_filtered with a set threshold. Wherein α_filtered = K1 × α_current + K2 × α_previous; α_current is the Hough line angle corresponding to the valid pointer identified in the current frame image; α_previous is the angle value obtained after smoothing the Hough line angle corresponding to the valid pointer identified in the previous frame image; where K1 and K2 are proportional coefficients, and K1 + K2 = 1.

[0030] The current instrument state is determined by comparing the angle α_filtered with a set threshold. This includes setting a hysteresis interval based on the set threshold. When the angle α_filtered is detected to be in the hysteresis interval, no instrument state switch is performed, and the instrument state judgment result of the previous frame is maintained.

[0031] A computer-readable storage medium storing a computer program, which, when executed by a processor, implements the image recognition-based instrument status monitoring method.

[0032] The advantages of this invention are: through adaptive edge detection and high-precision line recognition, accurate pointer recognition is achieved, and the instrument status is monitored and the results are output based on the pointer line.

[0033] This invention solves the problems of insufficient recognition accuracy and reliability caused by defects such as light interference, angle misjudgment and background interference in existing solutions. It can effectively, accurately and reliably realize image recognition of instrument status and achieve the purpose of automated instrument monitoring. Attached Figure Description

[0034] The following is a brief explanation of the contents of each of the accompanying drawings and the markings in the drawings:

[0035] Figure 1 This is a schematic diagram illustrating the principle and flow of the monitoring method of the present invention. Detailed Implementation

[0036] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and the description of the preferred embodiments.

[0037] This embodiment primarily addresses the problem of monitoring and identifying instrument status. Instruments typically use pointers to indicate their current state. By recognizing the instrument pointer from an image and calculating its angle, the current state of the instrument can be determined based on its characteristics and the angle, and even the pointer reading can be judged solely based on the angle. This solution, based on image recognition for monitoring instrument status, includes the following steps:

[0038] S1. Image acquisition and image preprocessing: Acquire instrument images and perform preprocessing.

[0039] S2, Canny edge detection: Canny edge point detection is used to process the preprocessed instrument image to obtain the edge contour;

[0040] S3. Obtain the Hough line and its endpoint coordinates in the image through Hough line detection;

[0041] S4. Filter the obtained Hough lines to obtain the Hough lines corresponding to the valid pointers.

[0042] S5. Calculate the angle based on the Hough line corresponding to the valid pointer, calculate the angle between the Hough line and the horizontal plane, and determine the indication status of the instrument based on the calculated angle value.

[0043] In step S1, sample images of the instrument are acquired using frame-by-frame extraction at a set frequency through the monitoring video of the instrument. Then, the ROI region of the instrument is obtained through cropping. Specifically, images are acquired via video stream access, using the RTSP protocol to obtain video streams from deployed cameras, and sample images are obtained by frame extraction at a set frequency (e.g., 5 frames / second). Then, ROI region extraction is performed, drawing multi-level bounding boxes (indices increasing from -1) in the image, and cropping out the dial area (ROI). Index -1 is the outermost region, and indices 0, 1, etc., correspond to specific instrument locations. Finally, the extracted ROI region is format-converted, converting the ROI image to numpy.ndarray format, generating a mask and filling the polygonal region, calculating the bounding box coordinates, and then cropping the target region.

[0044] In step S2, the threshold for the Canny edge detection algorithm is obtained through dynamic calculation. The dynamic threshold calculation for Canny edge detection includes: converting the cropped image to a grayscale image, calculating the grayscale mean in the grayscale image, and calculating the Canny dual threshold based on the grayscale mean. The setting of the dynamic threshold for Canny edge detection includes: reading the instrument image and converting it to a grayscale image, using the Canny edge detection operator, setting the initial values ​​of the low threshold and high threshold to 50 and 250 respectively, and recording the number L of edge points detected based on these parameter values; when L < the set threshold, the low threshold and high threshold are dynamically adjusted using adaptive parameters for different images captured by different cameras, and the calculation is repeated until L obtained by Canny detection is greater than the set threshold; the low threshold is used for edge connection, and pixels with gradient values ​​greater than the low threshold are considered potential edge points; the high threshold is used to determine strong edges, and all pixels with gradient values ​​greater than the high threshold are considered determined edge points. This step involves dynamic threshold calculation. The threshold can also be calculated quickly using the following method: convert the cropped image to grayscale, dynamically calculate the Canny double threshold based on the OTSU algorithm: T_low = 0.7 × grayscale mean, T_high = 1.3 × grayscale mean, and call the OpenCV Canny operator to extract edge contours and Hough line detection to solve the problem of fixed threshold being affected by illumination.

[0045] Step S3 includes: determining the five parameter combinations for Hough line detection using a grid search method, and then detecting Hough lines using a parameter-determined Hough line detection method. The method for determining the five parameter combinations for Hough line detection includes: the five parameters for Hough line detection are rho, theta, threshold, minLineLength, and maxLineGap; setting candidate value ranges for rho, theta, threshold, minLineLength, and maxLineGap, and randomly selecting values ​​within these ranges to form multiple parameter combinations; for each parameter combination, calling the Hough probability line transformation function HoughLinesP to obtain a candidate line set `lines`.

[0046] For a non-empty set of lines, calculate the length of each line segment, count the length of the longest line segment and the number of line segments, and combine this with the ROI dimensions to calculate the comprehensive evaluation metric, score.

[0047] score = (longest length / actual dial side length) × exp(-0.5 × (number of line segments - 1)); Select the parameter combination with the largest score as the optimal Hough detection parameters for the current instrument scene and store them for use in the detection of straight line segments in the current and subsequent images. Select the parameter combination with the largest score as the optimal parameter combination, and use the parameter combination as the parameters for the Hough line detection method to perform line detection.

[0048] In step S4, the endpoint coordinates of the detected straight lines are obtained, the angle α between each straight line and the horizontal line is calculated, and a filtering range is set according to the working characteristics of the instrument. Hough lines with angles within the filtering range are selected as the lines corresponding to valid pointers. Because different instruments have different working range characteristics, the filtering range can be set according to their characteristic angles. Pointer states outside the filtering range are impossible for the instrument and can therefore be directly filtered out. In this way, detection errors can be avoided or reduced, and the accuracy of pointer recognition can be improved.

[0049] In step S5, the Hough line angle α_current corresponding to the valid pointer identified in the current frame image is smoothed to obtain the smoothed angle α_filtered. The current state of the instrument is determined by comparing the angle α_filtered with a set threshold. Here, α_filtered = K1 × α_current + K2 × α_previous; α_current is the Hough line angle corresponding to the valid pointer identified in the current frame image; α_previous is the smoothed Hough line angle corresponding to the valid pointer identified in the previous frame image; K1 and K2 are proportional coefficients, and K1 + K2 = 1. Because different instrument angles represent different states, they can be compared and judged. In this embodiment, the smoothed angle is used to determine the instrument state. A preset angle threshold for instrument state switching is set. The current monitored instrument state is determined by angle comparison. When the angle threshold is crossed, the state change is determined, and alarms and warnings are output based on the changed state.

[0050] In a preferred embodiment of this application, determining the current state of the instrument based on the comparison between the angle α_filtered and a set threshold includes setting a hysteresis interval based on the set threshold. When the angle α_filtered is detected to be in the hysteresis interval, the instrument state is not switched, and the output of the instrument state judgment result of the previous frame is maintained. This avoids frequent fluctuations in the instrument state switching caused by the angle fluctuating in the hysteresis interval, which affects the accuracy and reliability of the output result and reduces false alarms.

[0051] This embodiment also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the image recognition-based instrument status monitoring method described in the above embodiment.

[0052] This application uses the image monitoring of a main power supply indicator as an example for illustration. The main power supply indicator is an instrument used in industrial production to monitor the power supply status of a production line or equipment. The current power supply status is indicated by the instrument's pointer, and monitoring this indicator reveals the current power supply status. The following describes the monitoring of the main power supply indicator according to this scheme, based on its characteristics:

[0053] The monitoring method includes the following steps:

[0054] 1. Image acquisition and preprocessing

[0055] Video stream access: Video streams are acquired from deployed cameras via the RTSP protocol, and sample images are obtained by extracting frames at a set frequency (e.g., 5 frames / second). A specific FPS value is set, and sample images are obtained by extracting frames from the video feed of the recorder. Images with similar features before and after are deleted, while images with clearly distinguishable features are retained and saved.

[0056] ROI extraction: Draw multi-level bounding boxes in the image (indexing starts from -1 and increments), and crop out the dial area (ROI). Index -1 is the outermost region, and indices 0, 1, etc. correspond to specific instrument locations.

[0057] Format conversion: Convert the ROI image to numpy.ndarray format, generate a mask and fill the polygonal region, calculate the bounding box coordinates and then crop the target region.

[0058] 2. Adaptive Canny edge detection

[0059] Adaptive dynamic method to determine threshold 1 (low threshold) and threshold 2: OpenCV reads the... Figure 1 The .png image is converted to grayscale, and the Canny edge detection operator is applied. Threshold 1 (low threshold) and threshold 2 (high threshold) are initialized to 50 and 250 respectively, and the number L of edge points detected based on these parameter values ​​is recorded. When L < 20, threshold 1 and threshold 2 are dynamically adjusted using adaptive parameters for different images captured by different cameras, and the calculation is repeated until L > 20 as determined by Canny detection. Threshold 1 is used for edge connection; pixels with gradient values ​​greater than threshold 1 are considered potential edge points. Threshold 2 is used to determine strong edges; all pixels with gradient values ​​greater than threshold 2 are considered confirmed edge points.

[0060] This step involves dynamic threshold calculation, which is as follows: the cropped image is converted to grayscale, and the Canny double threshold is dynamically calculated based on the OTSU algorithm: T_low = 0.7 × grayscale mean, T_high = 1.3 × grayscale mean. The OpenCV Canny operator is called to extract edge contours and Hough line detection to solve the problem of fixed threshold being affected by illumination.

[0061] 3. Determining the Hough Line using the Grid Search Method

[0062] The search method determines the optimal combination of the five parameters and obtains the endpoint coordinates of the Hough line. The grid search method determines the five parameters for Hough line detection: rho, theta, threshold, minLineLength, and maxLineGap.

[0063] A. Preprocessing: The binary edge image has been obtained through Canny edge detection in step 2.

[0064] B. Determine the parameter search space (set based on experience): rho: resolution of the polar radius in the Hough space (unit: pixels), with values ​​such as [0.5, 1, 1.5, 2], representing the radial step size for each accumulation;

[0065] theta: The resolution of polar angles in Hough space (unit: radians), such as [π / 180×1, π / 180×0.5, π / 180×0.3], which correspond to angular resolutions of 1°, 0.5°, and 0.3° respectively;

[0066] threshold: Hough accumulator threshold (unit: count), used to limit the minimum number of votes required for a line to be judged as a "valid line", with values ​​such as [50, 100, 150, 200];

[0067] minLineLength: Minimum line segment length (unit: pixels), such as [40 (0.5 times the dial radius), 48 (0.6 times), 56 (0.7 times)], used to remove excessively short noise line segments;

[0068] `maxLineGap`: The maximum allowed discontinuity interval (in pixels) between points on the same straight line, such as [5, 10, 15], used to determine whether several short line segments belong to the same pointer or edge. C. For each set of parameters, call the Hough probability line transformation function: `lines=cv2.HoughLinesP(edges,rho=rho_val,theta=theta_val, threshold=threshold_val,minLineLength=minLineLength_val, maxLineGap=maxLineGap_val)`.

[0069] D. Calculate evaluation indicators:

[0070] If `lines` is empty, the return value is None, and the index is 0. Otherwise, the lengths of all line segments are calculated, `lengths = [l1, l2, ..., ln]`. The longest length is `max(lengths)`, and the number of lines is `len(lines)`. The actual dial side length is `min(roi_width, roi_height)`, where `roi_width` and `roi_height` are the width and height (in pixels) of the Region of Interest (ROI) of the instrument. Since the dial is within the ROI, the dial diameter will not exceed the shorter side of the ROI. Let the comprehensive evaluation index `score` be `(longest length / actual dial side length) * math.exp(-0.5 * (len(lines)-1))`, and select the parameter combination with the largest index as the optimal parameter combination.

[0071] 4. Multi-stage filtering optimization

[0072] Noise suppression: Gaussian filtering (σ=1.5) is used to eliminate image noise. Edge restoration: Morphological closing operations (kernel size 5×5) are used to fill edge breaks and eliminate spurious interference.

[0073] 5. High-precision pointer recognition

[0074] First, set the polar coordinate parameters (rho, theta) to improve the angle resolution to 0.5°. Set a line segment length threshold to retain only continuous line segments with a length greater than 70% of the dial radius. Adaptively optimize the parameters minLineLength (minimum line segment length) and maxLineGap (maximum line segment gap) using a grid search algorithm. For the detected lines (endpoint coordinates (x1, y1), (x2, y2)), calculate the angle:

[0075] α = |arctan(Δy / Δx)| (unit: degrees)

[0076] Filtering strategy: Remove dial edge interference lines near 0° or 90° from the five longest straight lines, retaining valid pointer angles in the 50°~60° range. For the detected straight lines (endpoint coordinates (x1, y1), (x2, y2)), calculate the angle:

[0077] α = |arctan(Δy / Δx)| (unit: degrees) Filtering strategy: Remove dial edge interference lines near 0° or 90° from the longest 5 straight lines, and retain the effective pointer angle in the range of 50°~60°.

[0078] 6. Intelligent state decision-making.

[0079] An angle smoothing filter algorithm is introduced, with a hysteresis interval set (28°-32° no state switching): alpha_filtered = 0.7 × alpha_current + 0.3 × alpha_previous. If alpha < 30° → no power is supplied, triggering an alarm (total decrement state value is set to -1, and an alarm is broadcast); if alpha ≥ 30° → power is supplied (total decrement state value is set to 1). During testing, the angle value and state flag are written to the log and synchronized to the AI ​​management platform in real time.

[0080] Since the effective range of the total drop indicator is 50-60 degrees, interference can be removed by filtering. At the same time, the angle threshold after smoothing and filtering is set to 30 degrees for judgment. No judgment is made on the switching of the total drop indicator state within the hysteresis range of 28-32 degrees.

[0081] The above-mentioned technical solutions are not only applicable to the monitoring of the switching or safety status of traditional instruments such as pointer-type voltmeters, ammeters, wattmeters, pressure gauges, thermometers, and flow meters, but also applicable to instrument panels with "obvious geometric edges or indicating structures" such as DIP switches, status indicator lights, and mechanical handle positions. They have broad application prospects in various scenarios such as power systems, petrochemical and chemical plants, metallurgical mines, rail transportation, data centers, building automation, and energy management in industrial parks.

[0082] This solution has the following technical features:

[0083] 1. Enhanced environmental adaptability

[0084] By employing an adaptive Canny thresholding algorithm (T_low = 0.7 × mean gray level, T_high = 1.3 × mean gray level), it dynamically responds to changes from strong backlight (>100,000 lux) to dim environments (≥200 lux), overcoming the failure risk of traditional fixed threshold schemes and improving illumination robustness. Multi-level filtering (Gaussian filtering σ = 1.5 + morphological closing operation 5×5 kernels) effectively eliminates false edges caused by reflections from the dial glass, lens dust contamination, and mechanical vibration interference (image blur when amplitude <0.1mm).

[0085] 2. Breakthroughs in recognition accuracy and robustness

[0086] By employing grid search to optimize the Hough transform parameters (rho, theta, threshold, etc.), the angular resolution is improved to 0.5°. Combined with length threshold filtering (>70% of dial length), the pointer angle detection error is reduced from ±5° in the traditional scheme to ±0.8°. An angle hysteresis interval (28°-32° without state switching) and smoothing filtering (α_filtered=0.7×current value+0.3×historical value) are designed to avoid frequent state jumps, reducing the false alarm rate when power is not supplied from 8% to below 0.5%.

[0087] 3. Resource efficiency optimization

[0088] Based on the RTSP protocol, dynamic frame extraction (retaining frames with obvious feature differences) reduces redundant image processing by 80%, and the time for a single detection is less than 3 seconds (compared to 15 minutes per cabinet for traditional manual inspection). The adaptive mechanism for grid search parameters reduces the cost of manual optimization and shortens deployment and debugging time by 70%, meeting the real-time requirements of edge devices (resolution ≥1280×960).

[0089] 4. Security and Intelligent Upgrades

[0090] Non-contact monitoring completely avoids the risk of electric shock in high-voltage scenarios and meets the requirements for unmanned inspection. The real-time early warning and linkage control platform has a broadcast response time of less than 2 seconds for abnormal status (total degradation = -1) and supports log persistence and remote synchronization.

[0091] 5. Maintenance costs are significantly reduced.

[0092] Calibration-free deployment: The multi-level ROI constraint design (index-1 to N) simplifies the installation and calibration process, eliminating the need for professional personnel to perform precise calibration on-site, and improving deployment efficiency by 60%.

[0093] Self-diagnostic mechanism: When the number of edge points L < 20, parameter recalibration is automatically triggered.

[0094] Explanation of variables and terms in the embodiments of this application:

[0095] RTSP: Real-Time Streaming Protocol, used to acquire video streams from network cameras or hard disk recorders;

[0096] FPS: Frames Per Second, which represents the frequency at which image frames are extracted from a video stream, measured in frames per second;

[0097] ROI: Region of Interest, refers to the area of ​​the instrument that is cropped out from the original image for subsequent analysis, and has a width roi_width and a height roi_height (unit: pixels);

[0098] L: The number of points detected as edge pixels under the current Canny threshold combination, in units of pixels;

[0099] Threshold 1 and Threshold 2: These correspond to the low and high thresholds in the Canny operator, respectively, and are used to distinguish between "potential edges" and "strong edges".

[0100] T_low, T_high: Canny low and high thresholds obtained by dynamic calculation based on image grayscale statistics;

[0101] rho (ρ): Polar radius resolution in the Hough transform, in pixels;

[0102] theta (θ): Polar resolution in the Hough transform, in radians;

[0103] threshold: Hough accumulator threshold, representing the minimum number of votes required for a line to be considered valid;

[0104] minLineLength: The minimum line segment length in the Hough probability linear transform, in pixels;

[0105] maxLineGap: The maximum allowed gap between lines on a straight line, in pixels;

[0106] alpha: The angle between the straight line (i.e., the pointer) obtained by Hough detection and the horizontal direction, in degrees (°);

[0107] alpha_current: The pointer angle calculated in the current frame;

[0108] alpha_previous: The pointer angle after smoothing the previous frame;

[0109] alpha_filtered: The pointer angle obtained after smoothing the angles of the current frame and the previous frame;

[0110] Hysteresis range: [28°, 32°], an angle range used to avoid frequent state switching near the critical angle;

[0111] Total Degradation Status Value: A flag used to reflect the "total degradation / power supply" status. A value of -1 indicates no power supply (total degradation), and a value of 1 indicates power supply.

[0112] Image acquisition and preprocessing

[0113] Video stream access: The video stream is obtained from the deployed camera via the RTSP protocol, and sample images are obtained by extracting frames at a set frequency (e.g., 5 frames / second).

[0114] ROI extraction: Draw multi-level bounding boxes in the image (indexing starts from -1 and increments), and crop out the dial area (ROI). Index -1 is the outermost region, and indices 0, 1, etc. correspond to specific instrument locations.

[0115] Format conversion: Convert the ROI image to numpy.ndarray format, generate a mask and fill the polygonal region, calculate the bounding box coordinates and then crop the target region.

[0116] 3. Adaptive Canny edge detection

[0117] This step involves dynamic threshold calculation, which is as follows: the cropped image is converted to grayscale, and the Canny double threshold is dynamically calculated based on the OTSU algorithm: T_low = 0.7 × grayscale mean, T_high = 1.3 × grayscale mean. The OpenCV Canny operator is called to extract edge contours and Hough line detection to solve the problem of fixed threshold being affected by illumination.

[0118] 4. Determining the Hough Line using the Grid Search Method

[0119] After obtaining the edge image, a grid search method is used to determine the optimal combination of the five parameters rho, theta, threshold, minLineLength, and maxLineGap for Hough line detection.

[0120] (1) Define the candidate value ranges for rho, theta, threshold, minLineLength, and maxLineGap, as described above:

[0121] rho ∈ [0.5, 1, 1.5, 2];

[0122] theta ∈ [π / 180×1, π / 180×0.5, π / 180×0.3];

[0123] threshold ∈ [50, 100, 150, 200];

[0124] minLineLength ∈ [0.5R, 0.6R, 0.7R] (where R is the pixel length corresponding to the dial radius);

[0125] maxLineGap ∈ [5, 10, 15].

[0126] (2) For each set of parameters, call the Hough probability line transformation function HoughLinesP to obtain the candidate line set lines.

[0127] (3) Calculate the length of each line segment for a non-empty set of lines, count the length of the longest line segment and the number of line segments, and calculate the comprehensive evaluation index score in combination with the ROI size:

[0128] score = (longest length / actual dial side length) × exp(-0.5 × (number of line segments - 1)),

[0129] The actual dial side length is min(roi_width, roi_height).

[0130] (4) Select the parameter combination with the largest score as the optimal Hough detection parameters for the current instrument scenario. This parameter combination can be reused in multiple subsequent frames and the grid search is only triggered again when the environment changes significantly, thereby reducing computational overhead.

[0131] 5. Multi-stage filtering optimization

[0132] Noise suppression: Gaussian filtering (σ=1.5) is used to eliminate image noise. Edge restoration: Morphological closing operations (kernel size 5×5) are used to fill edge breaks and eliminate spurious interference.

[0133] 6. High-precision pointer recognition

[0134] First, set the polar coordinate parameters (rho, theta) to improve the angle resolution to 0.5°. Set a line segment length threshold to retain only continuous line segments with a length greater than 70% of the dial radius. Adaptively optimize the parameters minLineLength (minimum line segment length) and maxLineGap (maximum line segment gap) using a grid search algorithm. For the detected lines (endpoint coordinates (x1, y1), (x2, y2)), calculate the angle:

[0135] alpha = |arctan(Δy / Δx)| (unit: degrees)

[0136] Screening strategy: Remove interference lines at the dial edge near 0° or 90° from the five longest straight lines, and retain the effective pointer angle in the range of 50° to 60°.

[0137] 7. Intelligent State Decision Making

[0138] An angle smoothing filter algorithm is introduced, with a hysteresis interval set (28°-32° no state switching): alpha_filtered = 0.7 × alpha_current + 0.3 × alpha_previous. If alpha < 30° → no power is supplied, triggering an alarm (total decrement state value is set to -1, and an alarm is broadcast); if alpha ≥ 30° → power is supplied (total decrement state value is set to 1). During testing, the angle value and state flag are written to the log and synchronized to the AI ​​management platform in real time.

[0139] The step in this application of "removing lines near 0° and 90° from the five longest lines and retaining lines in the 50°–60° range" is essentially a coarse screening step for candidate lines. Its purpose is to quickly eliminate lines that are clearly irrelevant to the pointer in complex backgrounds, such as dial edges and scale lines. In actual engineering images, due to the width of the pointer, the presence of high-light reflection, and edge breaks, the same pointer often corresponds to two approximately parallel line segments or is divided into multiple segments in the Hough detection results. Therefore, after screening, there may still be two or more candidate lines, which is normal in visual inspection and does not directly equate to a recognition error. At the implementation level, the system further selects a pointer line for angle calculation from these candidates according to rules such as "maximum line segment length" and "minimum difference in smooth angle from the previous frame." If multiple candidate angles differ significantly or their lengths are all clearly insufficient, the frame is marked as a low-confidence frame, inheriting the state of the previous frame without immediate switching, thus ensuring overall recognition robustness.

[0140] In this embodiment, "retaining the effective pointer angle in the 50°–60° range" is an engineering experience setting based on the fact that, in a specific embodiment, the pointer of the total droop indicator roughly clusters within this angle range when powered on. This setting is used for more refined geometric filtering and fitting of the pointer under powered-on posture, and does not mean that only straight lines within this range are "true pointers." In reality, it can be adjusted according to the actual operating state of the instrument. When the instrument is not started and the reading is 0, the pointer is usually close to horizontal, and the straight line angle obtained by Hough detection will also be close to 0°. In this case, no straight lines are usually selected within the 50°–60° angle window, and the algorithm will naturally determine the current frame as "not in the powered-on area," and, combined with the subsequent comparison result of α_filtered and the threshold, classify it as "not powered-on state." In more general scenarios, this angle window can be configured to any [θ_min, θ_max], determined by the scale arrangement and calibration results of the specific instrument. 50°–60° is only an example embodiment for a certain type of total droop indicator and does not affect the correct identification of the true pointer near 0° and the determination of the not-powered state.

[0141] In this application, 50–60° and 30° serve different functions: 50°–60° is a typical angle window under the power-on attitude, used for geometric filtering and robust fitting; 30° is the logical boundary threshold in state decision-making, used to distinguish between power-on and non-power-on intervals. The algorithm first obtains the instantaneous angle α_current of the current frame through Hough transform and geometric filtering, and then obtains α_filtered after time smoothing. The final state determination criteria are: α_filtered < 30° indicates no power-on, and α_filtered ≥ 30° indicates power-on. From the perspective of the numerical axis, the 50°–60° interval itself is a subset of the (30°, 90°) interval, that is, as long as the pointer is in the 50°–60° range, it inherently satisfies the power-on criterion of "≥ 30°". The angle window is used to improve detection accuracy and robustness within the power-on interval, while 30° is the state threshold for coarse segmentation of the entire angle domain.

[0142] In this embodiment, the "no switching in the hysteresis interval 28°–32°" adopts a dual-threshold hysteresis control approach to suppress frequent state flips caused by noise and jitter near the threshold. Specifically, two thresholds are introduced: θ_off = 28° (off threshold) and θ_on = 32° (on threshold), and the instantaneous angle α_current is smoothed to obtain α_filtered. The state update rule is as follows: when α_filtered < θ_off, it is forcibly determined to be in a power-off state (total state drop value = -1); when α_filtered > θ_on, it is forcibly determined to be in a power-on state (total state drop value = 1); when α_filtered falls between θ_off and θ_on, i.e., within the hysteresis interval of 28°–32°, the current frame does not re-determine based on the angle, but maintains the state of the previous frame unchanged, which is the meaning of "no switching". This design requires a 32° transition from "not powered" to "powered," and a drop to below 28° when switching back from "powered" to "not powered," thus creating a wide hysteresis band. This effectively eliminates status jitter and false alarms caused by small fluctuations around 30°. Both 28° and 32° are configurable parameters; the manual provides a set of typical values ​​that have proven effective through field testing, and adjustments can be made as needed in actual applications.

[0143] Obviously, the specific implementation of this invention is not limited to the above-described methods. Any non-substantial improvements made using the inventive concept and technical solution of this invention are within the protection scope of this invention.

Claims

1. A method of monitoring the state of an instrument based on image recognition, characterized in that: Includes the following steps: S1. Image acquisition and image preprocessing: Acquire instrument images and perform preprocessing. S2, Canny edge detection: Canny edge point detection is used to process the preprocessed instrument image to obtain the edge contour; S3. Obtain the Hough line and its endpoint coordinates in the image through Hough line detection; S4. Filter the obtained Hough lines to obtain the Hough lines corresponding to the valid pointers. S5. Calculate the angle based on the Hough line corresponding to the valid pointer, calculate the angle between the Hough line and the horizontal plane, and determine the indication status of the instrument based on the calculated angle value.

2. The image recognition based monitoring of meter status method as claimed in claim 1, wherein: In step S1, sample images of the instrument are obtained by frame-sampling at a set frequency through the monitoring video of the instrument; then the ROI area of ​​the instrument is obtained by cropping.

3. The image recognition based monitoring of meter status method as claimed in claim 1, wherein: In step S2, the threshold for the Canny edge detection algorithm is obtained through dynamic calculation. The dynamic threshold calculation for Canny edge detection includes: converting the cropped image to a grayscale image, calculating the grayscale mean in the grayscale image, and calculating the Canny dual threshold based on the grayscale mean.

4. The method for monitoring instrument status based on image recognition as described in claim 3, characterized in that: The dynamic threshold setting for Canny edge detection includes: reading the instrument image and converting it to grayscale; using the Canny edge detection operator; setting the initial values ​​of the low threshold and high threshold to 50 and 250 respectively; and recording the number L of edge points detected based on these parameter values; when L < the set threshold, dynamically adjusting the low threshold and high threshold using adaptive parameters for different images captured by different cameras; and iteratively calculating until L detected by Canny > the set threshold; where the low threshold is used for edge connection, and pixels with gradient values ​​greater than the low threshold are considered potential edge points; the high threshold is used to determine strong edges, and all pixels with gradient values ​​greater than the high threshold are considered determined edge points.

5. The method for monitoring instrument status based on image recognition as described in claim 1, characterized in that: Step S3 includes: determining the five parameter combinations for Hough line detection using a grid search method, and then detecting the Hough line using the parameter-determined Hough line detection method.

6. The method for monitoring instrument status based on image recognition as described in claim 5, characterized in that: The method for determining the combination of five parameters for Hough line detection includes: the five parameters for Hough line detection are rho, theta, threshold, minLineLength, and maxLineGap. Define the candidate value ranges for rho, theta, threshold, minLineLength, and maxLineGap, and randomly select values ​​within the range to form multiple parameter combinations; For each set of parameters, the Hough probability line transformation function HoughLinesP is called to obtain the candidate line set lines; For a non-empty set of lines, calculate the length of each line segment, count the length of the longest line segment and the number of line segments, and combine this with the ROI dimensions to calculate the comprehensive evaluation metric, score. score = (longest length / actual dial side length) × exp(-0.5 × (number of line segments - 1)); The parameter combination with the highest score is selected as the optimal Hough detection parameters for the current instrument scene and stored for use in the detection of straight line segments in the current and subsequent images.

7. The method for monitoring instrument status based on image recognition as described in claim 6, characterized in that: In step S4, the coordinates of the endpoints of the detected straight lines are obtained, the angle α between each straight line and the horizontal line is calculated, and a filtering range is set according to the working characteristics of the instrument. Hough lines whose angle is within the filtering range are selected as the lines corresponding to the valid pointer.

8. The method for monitoring instrument status based on image recognition as described in claim 7, characterized in that: In step S5, the Hough line angle α_current corresponding to the valid pointer identified in the current frame image is smoothed to obtain the smoothed angle α_filtered. The current state of the instrument is determined by comparing the angle α_filtered with a set threshold. Where α_filtered = K1 × α_current + K2 × α_previous; α_current is the Hough line angle corresponding to the valid pointer identified in the current frame image; α_previous is the angle value obtained after smoothing the Hough line angle corresponding to the valid pointer identified in the previous frame image. Where K1 and K2 are proportionality coefficients, and K1+K2=1.

9. The method for monitoring instrument status based on image recognition as described in claim 8, characterized in that: The current instrument state is determined by comparing the angle α_filtered with a set threshold. This includes setting a hysteresis interval based on the set threshold. When the angle α_filtered is detected to be in the hysteresis interval, no instrument state switch is performed, and the instrument state judgment result of the previous frame is maintained.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the image recognition-based instrument status monitoring method as described in any one of claims 1-9.

Images