A mechanical instrument recognition method and system based on multi-view fusion

Abstract

The application provides a mechanical instrument recognition method and system based on multi-view fusion, comprising the following steps: S1, collecting multiple frames of original images of a mechanical instrument to be recognized; S2, preprocessing each frame of original image to obtain a preprocessed image, performing feature matching and spatial transformation on the highest definition frame as a reference image to obtain multiple frames of registration images; S3, obtaining an enhanced image through weighted fusion and image enhancement processing; S4, inputting the enhanced image into a target detection model to recognize a dial area, and obtaining an effective pointer contour through edge detection and contour extraction based on a pointer feature condition; and S5, obtaining running state data of the mechanical instrument to be recognized, and obtaining an instrument reading according to the running state data and the effective pointer contour. The application can improve the quality of low-quality images, suppress interference, and reuse existing hardware to stably recognize the reading of a mechanical instrument in combination with the running law of the instrument.

Description

Technical Field

[0001] This invention relates to the technical field of automatic identification of mechanical instruments, and more specifically, to a method and system for identifying mechanical instruments based on multi-view fusion. Background Technology

[0002] Mechanical instruments are core condition monitoring components in power operation and maintenance, industrial production, and rail transportation, and their reading accuracy directly affects the safe operation of equipment and production stability. In actual inspection scenarios such as power distribution rooms, tunnels, and large unmanned equipment, due to limitations in on-site deployment conditions and costs, most sites still use early-deployed low-resolution monitoring equipment to collect instrument images. Such equipment has low imaging resolution, and the on-site environment is complex, with common interference factors such as uneven lighting, dust, fog, equipment obstruction, and shadows and reflections caused by light projection, resulting in low contrast, loss of detail, and severe interference in the acquired instrument images.

[0003] Existing mechanical instrument identification technologies are mostly based on monocular vision image acquisition, relying on a single image for feature extraction and reading calculation. When there are partial occlusions, reflections, or shadows in the image, a single-view image cannot provide complete dial information, and it is difficult to suppress interference through multi-view comparison. This causes traditional detection and segmentation algorithms to easily misjudge shadows and reflective edges as pointers, resulting in reading errors and false alarms. Furthermore, for images acquired from multiple views, existing technologies lack effective registration, fusion, and enhancement methods. It is difficult to eliminate viewpoint and positional deviations between different cameras, and it is impossible to integrate the effective features of multiple frames. The fused image often suffers from ghosting and loss of detail, making it difficult to truly improve the recognition accuracy of low-quality images.

[0004] Furthermore, existing technologies largely focus on improving the accuracy of the algorithms themselves, without fully considering the engineering needs of reusing existing monitoring equipment on-site. This often requires the addition of new high-definition cameras or dedicated acquisition equipment, resulting in high system modification costs, poor adaptability, and difficulty in large-scale application in complex industrial scenarios. Although mainstream object detection algorithms such as YOLOv10 and OpenCV edge detection technology provide technical support for instrument recognition, existing solutions have not been specifically optimized for low-resolution and highly interfering images, nor have they combined the complementary advantages of multi-view images to address the shortcomings of single images, resulting in insufficient recognition accuracy and robustness.

[0005] Therefore, there is an urgent need for a stable recognition scheme that can improve the quality of low-quality images and suppress interference by relying on multi-view fusion, and can reuse existing hardware and combine the operating rules of instruments, so as to solve the problems of low recognition accuracy, high misjudgment rate, high transformation cost and poor adaptability of existing technologies. Summary of the Invention

[0006] The technical problem to be solved by the present invention is how to achieve stable identification of mechanical instrument readings by relying on multi-view fusion to improve the quality of low-quality images, suppress interference, reuse existing hardware, and combine the instrument operation rules. In order to overcome the defects of the above-mentioned prior art (or related technology), the present invention provides a mechanical instrument identification method and system based on multi-view fusion.

[0007] This invention provides a method for identifying mechanical instruments based on multi-view fusion, comprising the following steps: Step S1: Using at least two monitoring devices deployed in different locations, simultaneously acquire multiple frames of original images of the mechanical instrument to be identified, wherein the multiple frames of original images cover all dial areas of the mechanical instrument to be identified; Step S2: Preprocess each frame of the original image to obtain a preprocessed image. Use the frame with the highest clarity among the preprocessed images as the reference image and perform feature matching and spatial transformation with the other preprocessed images to obtain a multi-frame registration image. Step S3: According to the preset fusion weight, the registered images of each frame are weighted and fused to obtain a fused image, and the fused image is then subjected to image enhancement processing to obtain an enhanced image; Step S4: Input the enhanced image into the target detection model to identify and locate the dial area, and obtain the effective pointer contour through edge detection and contour extraction based on the preset pointer feature conditions; Step S5: Obtain the operating status data of the mechanical instrument to be identified, and obtain the corresponding matching result based on the operating status data and the valid pointer profile to determine the instrument reading.

[0008] Compared with existing technologies, the mechanical instrument identification method based on multi-view fusion proposed in this invention has the following advantages: This invention establishes an end-to-end mechanical instrument identification scheme through multi-view acquisition, preprocessing registration, fusion enhancement, dial positioning and pointer segmentation, and reading determination. It utilizes multi-view image complementarity to compensate for the poor imaging quality, occlusion, or local interference inherent in single monitoring devices. By using the frame with the highest clarity as the reference image for registration, it eliminates viewpoint and positional biases between multi-source images, laying the foundation for subsequent fusion. Weighted fusion and image enhancement further improve the feature representation capabilities of low-quality images. Finally, it combines operational status data to determine the instrument reading, ensuring the accuracy and reliability of the results. Overall, this scheme significantly improves the identification accuracy and robustness of mechanical instruments in complex environments without adding new hardware.

[0009] In one possible implementation, the preprocessing method used in step S2 is interference suppression processing, including: The preprocessed image is obtained by using histogram equalization combined with an adaptive threshold segmentation algorithm to separate the shadow and / or reflective areas from the effective area of ​​the dial in each frame of the original image.

[0010] Compared with existing technologies, the above-mentioned technical solution can effectively separate the interference area from the target area, enhance the contrast between the pointer and the background, and reduce the interference of false contours from the source, thus providing a higher quality input image for subsequent feature matching and pointer segmentation, and significantly reducing the misjudgment rate caused by environmental interference.

[0011] In one possible implementation, in step S2, the feature matching uses the SIFT feature matching algorithm, and the spatial transformation uses perspective transformation to correct the visual and positional deviations between the preprocessed images, so that the dial areas in the preprocessed images are spatially aligned.

[0012] Compared with existing technologies, the above technical solution can accurately match key points in multi-view images through the SIFT algorithm; perspective transformation can effectively correct affine distortion caused by different camera installation positions, ensuring that the dial area in multiple frames of images is aligned at the pixel level, thereby providing a spatially consistent image sequence for subsequent weighted fusion and avoiding fusion ghosting or loss of details due to misalignment.

[0013] In one possible implementation, in step S3, the preset fusion weights are dynamically allocated based on the sharpness score and / or environmental interference level of each frame of the registered image, wherein the fusion weights corresponding to the registered images whose sharpness score is greater than a first preset threshold and whose environmental interference level is less than a second preset threshold are less than 40%.

[0014] Compared with existing technologies, the above-mentioned technical solution can avoid the problem of excessive concentration of weight on a single image, which would cause the fusion result to be affected by the local defects of that image, through dynamic weight allocation, while ensuring that the effective details of the high-quality image are fully preserved. Controlling the weight of the high-quality image to below 40% reflects the core concept of "complementarity rather than dependence" in multi-view fusion, enabling the fused image to combine the advantages of each viewpoint. At the same time, by weakening interference such as shadows and reflections through multi-view comparison, the robustness of the fused image is improved.

[0015] In one possible implementation, the target detection model used in step S4 is the YOLOv10 model, and an ECA attention mechanism is introduced into the network structure of the YOLOv10 model. The loss function used by the target detection model is the CIoU loss function, and the anchor frame size of the target detection model is adjusted according to the dial specifications of the mechanical instrument to be identified.

[0016] Compared with existing technologies, the above technical solution can enhance the feature extraction capability of the target detection model for dials and pointers in low-resolution, highly interfering images through the ECA attention mechanism, and suppress false responses in shadow and reflective areas; the CIoU loss function considers the overlap area between the predicted box and the ground truth box, the distance between the center point and the aspect ratio, which can reduce the dial positioning error; adjusting the anchor frame size enables the target detection model to better adapt to the dials of different sizes of mechanical instruments.

[0017] In one possible implementation, in step S4, multiple candidate contours are obtained by contour detection and contour extraction within the dial area, and then the multiple candidate contours are filtered based on the preset pointer feature conditions to obtain the effective pointer contour.

[0018] Compared with existing technologies, the above technical solution can clearly separate extraction and screening into two steps, which reflects the refined processing of traditional edge detection results. By first obtaining all candidate contours and then screening them in combination with preset pointer feature conditions, false contours formed by scale lines, stains, text, and shadow reflective edges can be effectively filtered out, ensuring that the final retained valid pointer contours are real pointers, thereby greatly improving the accuracy of pointer segmentation and reducing the risk of missegmentation.

[0019] In one possible implementation, the preset pointer feature conditions in step S4 include shape filtering conditions, grayscale filtering conditions, and position filtering conditions. The shape filtering condition is that the candidate contour is a thin, elongated frame. The grayscale filtering condition is that the grayscale value of the candidate contour is within a preset pointer grayscale range. The position filtering condition is that the candidate contour is located within a region defined by a preset range position.

[0020] Compared with existing technologies, the above-mentioned technical solution can constrain candidate contours from different dimensions based on three screening conditions: shape, grayscale, and position. The shape screening condition utilizes the geometric feature of the pointer being a thin strip, the grayscale screening condition utilizes the grayscale difference between the pointer and the background, and the position screening condition combines the physical law of the fixed range of the instrument. This multi-dimensional screening mechanism can form complementary verification and effectively eliminate various interferences. In particular, for shadow edges or scale lines with similar shapes to the pointer, they can be distinguished by position and grayscale conditions, which significantly improves the robustness of pointer segmentation in complex environments.

[0021] In one possible implementation, after performing step S5, the method further includes: If the valid pointer profile is not extracted in step S4 or the deviation of the instrument reading from the corresponding fixed normal operating reading in step S5 exceeds a preset abnormal threshold, then the instrument is determined to be abnormal.

[0022] Compared with existing technologies, the above technical solution can include both "no pointer" and "reading over the limit" in the anomaly judgment category, reflecting a comprehensive coverage of the instrument's abnormal state. The absence of a pointer may correspond to pointer detachment, obstruction, or image acquisition failure, while the instrument reading over the limit corresponds to abnormal instrument indication or equipment operation. By combining fixed range position with actual reading for dual verification, rapid and accurate anomaly warning can be achieved, providing timely support for equipment operation and maintenance and reducing manual inspection costs.

[0023] This invention provides a mechanical instrument identification system based on multi-view fusion, which applies the above-mentioned mechanical instrument identification method and includes: The multi-view image acquisition module is used to simultaneously acquire multiple frames of raw images of the mechanical instrument to be identified by at least two monitoring devices deployed in different locations. The preprocessing and registration module is connected to the multi-view image acquisition module. It is used to preprocess each frame of the original image and use the preprocessed image with the highest resolution as the reference image to perform feature matching and spatial transformation to obtain multi-frame registered images. The image enhancement and dial recognition module is connected to the preprocessing and registration module. It is used to perform weighted fusion and image enhancement on multiple registered images to obtain an enhanced image, and to identify and locate the dial area based on the target detection model. The pointer segmentation and anomaly determination module, connected to the image enhancement and dial recognition module, is used to extract and filter valid pointer outlines within the dial area, and then combine them with the operating status data of the mechanical instrument to be identified to perform reading verification and anomaly determination to obtain the instrument reading.

[0024] Compared with existing technologies, the mechanical instrument identification system based on multi-view fusion of this invention has the following advantages: This invention integrates the entire process of multi-view acquisition, preprocessing registration, fusion enhancement, detection and recognition, segmentation and judgment in a modular manner. The data flow between modules is clear and the coupling is low, which facilitates deployment and maintenance. By reusing existing monitoring equipment on site, a low-cost and highly adaptable automated inspection system is realized, which can operate stably in complex industrial scenarios such as power distribution rooms, tunnels, and large unmanned equipment, and meet the needs of actual engineering applications. Attached Figure Description

[0025] Figure 1 This is a flowchart of the method steps of the present invention; Figure 2 This is a schematic diagram of the system structure of the present invention. Detailed Implementation

[0026] First, those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention. Those skilled in the art can make adjustments as needed to adapt to specific application scenarios.

[0027] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0028] See Figure 1 This invention discloses a method for identifying mechanical instruments based on multi-view fusion, comprising the following steps: Step S1: Using at least two monitoring devices deployed in different locations, simultaneously acquire multiple frames of original images of the mechanical instrument to be identified, with the multiple frames of original images covering all dial areas of the mechanical instrument to be identified. Step S2: Preprocess each frame of the original image to obtain the preprocessed image. Use the frame with the highest clarity among the preprocessed images as the reference image and perform feature matching and spatial transformation with the other preprocessed images to obtain multi-frame registration images. Step S3: According to the preset fusion weights, the registered images of each frame are weighted and fused to obtain a fused image, and the fused image is then enhanced to obtain an enhanced image. Step S4: Input the enhanced image into the target detection model to identify and locate the dial area, and obtain the effective pointer contour through edge detection and contour extraction based on the preset pointer feature conditions; Step S5: Obtain the operating status data of the mechanical instrument to be identified, and obtain the corresponding matching result based on the operating status data and the valid pointer profile to determine the instrument reading.

[0029] In this embodiment of the invention, in step S1, the monitoring equipment can directly reuse the low-definition monitoring equipment already deployed on site, such as a 480P resolution camera, without adding any new hardware. During the acquisition process, the ambient light parameters (such as 50-200 lux), shadow and reflection distribution of each monitoring device can be recorded simultaneously to provide data support for subsequent interference suppression processing. The acquisition frequency can be set according to the instrument reading update frequency, for example, set to 1 frame / second, to ensure that the acquired original image can reflect the changes in the instrument status in a timely manner. The deployment position of each monitoring device only needs to meet the requirements of being unobstructed and able to fully observe the instrument panel, without fixed position restrictions. The original image is transmitted to the processing end in real time through the existing wireless or wired transmission equipment on site.

[0030] In this embodiment of the invention, the preprocessing method used in step S2 is interference suppression processing, specifically including: using histogram equalization combined with an adaptive threshold segmentation algorithm to separate the shadow and / or reflective areas in each frame of the original image from the effective area of ​​the dial to obtain the preprocessed image. Histogram equalization is used to stretch the image contrast, making the grayscale difference between the pointer and the background more obvious; the adaptive threshold segmentation algorithm dynamically determines the segmentation threshold according to the grayscale distribution of the local area of ​​the image, which can effectively separate the shadow, reflective areas and the effective area of ​​the dial, and enhance the contrast between the pointer and the background. In addition, the preprocessing method may also include median filtering for noise reduction, grayscale conversion, Hough transform tilt correction and other operations. For example, a median filtering algorithm with a kernel size of 3×3 can be used to remove image noise, convert the color image to a grayscale image to enhance the contrast between the pointer and the background, and use Hough transform to detect image straight lines to correct the image tilt caused by poor shooting angle, ensuring that the dial is in a horizontal viewing angle.

[0031] In this embodiment of the invention, in step S2, the feature matching adopts the SIFT (Scale Invariant Feature Transform) feature matching algorithm, and the spatial transformation adopts perspective transformation to correct the viewpoint deviation and position deviation between each preprocessed image, so that the dial area in each preprocessed image is spatially aligned. The SIFT feature matching algorithm is robust to changes in image scale, rotation and illumination, and can accurately extract and match key points in each frame of preprocessed image. The perspective transformation calculates the homography matrix and projects each frame of preprocessed image onto the coordinate system of the reference image to achieve pixel-level spatial alignment, providing a spatially consistent image sequence for subsequent image fusion.

[0032] In this embodiment of the invention, in step S3, the preset fusion weights are dynamically allocated based on the sharpness score and / or environmental interference level of each registered image frame. The sharpness score can be obtained by calculating the image gradient, Laplacian variance, or a sharpness evaluation function based on the reference image. The environmental interference level can be evaluated based on the proportion of shadow and reflection areas. Among them, the fusion weight corresponding to the registered image with a sharpness score greater than a first preset threshold and an environmental interference level less than a second preset threshold is less than 40%. This weight allocation strategy can avoid over-concentrating the weight on a single image, ensuring that the fused image fully retains the effective details of each registered image frame, and weakening interference such as shadows and reflections through multi-view comparison. For example, in a specific application, the fusion weight of the registered image with less interference and higher sharpness can be set to 35%, and the fusion weight of another set of registered images can be set to 65%. After fusion, the Retinex enhancement algorithm can be used to sharpen the fused image, suppress residual interference, and further improve the recognizability of the dial scale and pointer.

[0033] In this embodiment of the invention, the target detection model used in step S4 is the YOLOv10 model, and the ECA (Efficient Channel Attention) attention mechanism is introduced into the network structure of the YOLOv10 model. The loss function used by the target detection model is the CIoU (Complete Intersection over Union) loss function. The anchor frame size of the target detection model is adjusted according to the dial specifications of the mechanical instrument to be identified. The ECA attention mechanism can enhance the target detection model's ability to extract features of the dial and pointer in low-resolution, highly interfering images and suppress false responses in shadow and reflective areas. The CIoU loss function considers the overlap area between the predicted box and the real box, the distance between the center point and the aspect ratio, which can reduce the dial positioning error. The anchor frame size can be preset according to the actual dial specifications, for example, set to the range of 200×200 to 300×300 pixels to adapt to different specifications of mechanical instrument dials.

[0034] In this embodiment of the invention, in step S4, multiple candidate contours are obtained within the dial area through contour detection and contour extraction. Then, based on preset pointer feature conditions, the multiple candidate contours are filtered to obtain valid pointer contours. In specific implementation, an independent dial image can be cropped using the OpenCV library according to the dial coordinates, and the dial image is binarized (e.g., the threshold is optimized to 127) to enhance the contrast between the pointer and the reflective and shadow areas. The Canny edge detection algorithm combined with the interference filtering function is used to extract the inner edge contours of the dial, and all contours are extracted using the cv2.findContours function to obtain multiple candidate contours.

[0035] In this embodiment of the invention, the preset pointer feature conditions in step S4 include shape filtering conditions, grayscale filtering conditions, and position filtering conditions. The shape filtering condition is that the candidate contour is a thin strip (e.g., 50-80 pixels in length and 5-10 pixels in width). The grayscale filtering condition is that the grayscale value of the candidate contour is within the preset pointer grayscale range (e.g., grayscale value range is 100-150). The position filtering condition is that the candidate contour is located within the area defined by the preset range position. Through triple filtering, false contours formed by scale lines, stains, text, and shadow reflective edges can be effectively filtered out, ensuring that the final retained valid pointer contour is the real pointer.

[0036] In this embodiment of the invention, the operating status data obtained in step S5 includes, but is not limited to: fixed normal operating readings and the preset range position, abnormal threshold, scale distribution rules, and pointer morphology parameters corresponding to the fixed normal operating readings. In specific implementation, the operating status data can be automatically obtained by connecting to the on-site operation and maintenance management system, or manual supplementation and entry can be supported. The instrument reading determination process can combine the Hough line detection algorithm to extract the pointer line, calculate the initial reading based on the relationship between the pointer line and the preset range position, and then use the scale matching method to match and correct the initial reading with the fixed normal operating reading at the corresponding fixed range position (for example, if the error is ≤2%, it is corrected to the corresponding fixed normal operating reading). Combine the pointer grayscale features and morphology features for verification again to confirm the final instrument reading.

[0037] In this embodiment of the invention, after executing step S5, the method further includes: if no valid pointer profile is extracted in step S4 or the deviation of the instrument reading in step S5 from the corresponding fixed normal operating reading exceeds a preset abnormal threshold, then the instrument is determined to be abnormal. The preset abnormal threshold can be set to ±5% of the deviation from the fixed normal operating reading. When the instrument reading deviation exceeds the preset abnormal threshold or no valid pointer profile is detected, the system automatically marks the abnormality and triggers an early warning to provide timely support for equipment operation and maintenance.

[0038] In this embodiment of the invention, the method further includes an iterative optimization step: collecting multi-view images, enhanced images, recognition results, pointer segmentation results, and verification results for each contour recognition process, with a focus on collecting misjudged samples caused by shadows and reflections. If the recognition confidence is below 95%, or if there are cases of pointer omission, missegmentation, interference, or misrecognition, the set of samples and the corresponding instrument operating status data are added to the training set. The parameters of the YOLOv10 model, pointer segmentation parameters, and image fusion weight allocation logic are iteratively fine-tuned, and the target detection model is retrained to form a closed-loop mechanism of "collection-enhancement-recognition-segmentation-verification-optimization" to continuously improve the robustness of the system's recognition.

[0039] See Figure 2 This invention also discloses a mechanical instrument identification system based on multi-view fusion, comprising: The multi-view image acquisition module is used to simultaneously acquire multiple frames of raw images of the mechanical instrument to be identified by at least two monitoring devices deployed in different locations. The preprocessing and registration module is connected to the multi-view image acquisition module. It is used to preprocess each frame of the original image and use the preprocessed image with the highest resolution as the reference image to perform feature matching and spatial transformation to obtain multi-frame registered images. The image enhancement and dial recognition module is connected to the preprocessing and registration module. It is used to perform weighted fusion and image enhancement on multiple registered images to obtain an enhanced image, and to identify and locate the dial area based on the target detection model. The pointer segmentation and anomaly detection module, connected to the image enhancement and dial recognition module, is used to extract and filter valid pointer outlines within the dial area. Subsequently, it combines the operating status data of the mechanical instrument to be identified to perform reading verification and anomaly detection to obtain the instrument reading.

[0040] In this embodiment of the invention, the following specific application example further illustrates the invention to enable those skilled in the art to better understand the technical solution of the invention. This embodiment takes the automatic identification of voltmeters in a State Grid distribution room as an example. The voltmeter only operates at three fixed conventional readings (220V, 380V, 0V), corresponding to three fixed range positions. Two existing low-definition monitoring devices (480P resolution) are deployed on-site, with the deployment position ensuring no obstruction, so that the voltmeter dial can be completely observed. The specific execution process is as follows: Multi-view image acquisition: The acquisition frequency is set to 1 frame / second, matching the voltmeter reading update frequency. During the acquisition process, the ambient light parameters (50-200 lux), shadow and reflection distribution of the two monitoring devices are recorded. The images are transmitted to the preprocessing and registration module in real time through the existing wired transmission line. Image preprocessing and registration: The acquired images are denoised using a median filter algorithm with a kernel size of 3×3. The color images are converted to grayscale images. The image tilt is corrected by Hough transform. Histogram equalization combined with an adaptive threshold segmentation algorithm is used to separate the reflective and shadow areas from the effective area of ​​the dial. The SIFT feature matching algorithm is used to align the voltmeter dial area of ​​the two sets of images with one set of images with less interference as a reference, eliminating position and angle deviations, and outputting the registered image. Image Fusion and Enhancement: A weighted fusion method is used to fuse two sets of registered images. The image with less interference and relatively higher clarity is weighted at 35%, while the other set of images is weighted at 65%. The fused image is then sharpened using the Retinex enhancement algorithm to suppress residual interference and output a high-quality instrument image. Dial detection and pointer segmentation: An optimized YOLOv10 model is used, introducing the ECA attention mechanism. The anchor box size is adjusted from 200×200 to 300×300 pixels, and the loss function is optimized to CIoU loss function. The voltmeter dial in the image is accurately identified, and the dial area coordinates are output. Based on the dial area coordinates, an independent voltmeter dial image is cropped using the OpenCV library. The voltmeter dial image is binarized (optimized threshold is 127). The Canny edge detection algorithm combined with interference filtering function is used to extract the inner edge contour of the dial. All contours are extracted using the cv2.findContours function. The contours that are thin and long (50-80 pixels in length, 5-10 pixels in width), with a grayscale range of 100-150 and located within 3 preset fixed range positions are selected as valid pointer contours. Reading Verification and Anomaly Judgment: By connecting to the power distribution room operation and maintenance management system, the operating status data of the voltmeter is automatically obtained, including fixed normal operating readings (220V, 380V, 0V), anomaly thresholds (deviation from fixed normal operating reading ±5%), coordinates of three fixed range positions, scale distribution rules, and pointer morphology parameters. The Hough line detection algorithm is used to extract the pointer line, and the initial reading is calculated in combination with the scale distribution rules. The initial reading is matched and corrected with the fixed normal operating reading at the corresponding fixed range position using the scale matching method (if the error is ≤2%, it is corrected to the corresponding fixed normal operating reading). The reading is then verified again by combining the pointer grayscale characteristics and morphological characteristics to confirm the final instrument reading. If no valid pointer outline is detected, or the reading deviates from the fixed normal operating reading ±5%, or the pointer position exceeds the preset fixed range, the voltmeter is judged to be abnormal, the abnormality is marked, and an alarm is triggered. Iterative optimization: Collect two sets of images, enhanced images, dial recognition results, pointer segmentation results, calculated readings, and verification results for each recognition process. Focus on collecting misjudged samples caused by reflections and shadows. If the recognition confidence is below 95%, or if there are cases of pointer omissions, missegments, or misjudgments of reflections / shadows as pointers, add the misjudged samples and the corresponding voltmeter's operating status data to the training set. Fine-tune the attention mechanism weights, anchor box size, and binarization threshold of the OpenCV pointer segmentation in the YOLOv10 model, retrain the object detection model, and continuously improve recognition accuracy and robustness.

[0041] In the description of this invention, the references to "one embodiment," "some embodiments," "in this embodiment," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0042] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for identifying mechanical instruments based on multi-view fusion, characterized in that, Includes the following steps: Step S1: Using at least two monitoring devices deployed in different locations, simultaneously acquire multiple frames of original images of the mechanical instrument to be identified, wherein the multiple frames of original images cover all dial areas of the mechanical instrument to be identified; Step S2: Preprocess each frame of the original image to obtain a preprocessed image. Use the frame with the highest clarity among the preprocessed images as the reference image and perform feature matching and spatial transformation with the other preprocessed images to obtain a multi-frame registration image. Step S3: According to the preset fusion weight, the registered images of each frame are weighted and fused to obtain a fused image, and the fused image is then subjected to image enhancement processing to obtain an enhanced image; Step S4: Input the enhanced image into the target detection model to identify and locate the dial area, and obtain the effective pointer contour through edge detection and contour extraction based on the preset pointer feature conditions; Step S5: Obtain the operating status data of the mechanical instrument to be identified, and obtain the corresponding matching result based on the operating status data and the valid pointer profile to determine the instrument reading.

2. The mechanical instrument identification method according to claim 1, characterized in that, The preprocessing method used in step S2 is interference suppression processing, which includes: The preprocessed image is obtained by using histogram equalization combined with an adaptive threshold segmentation algorithm to separate the shadow and / or reflective areas from the effective area of ​​the dial in each frame of the original image.

3. The mechanical instrument identification method according to claim 1, characterized in that, In step S2, the SIFT feature matching algorithm is used for feature matching, and the perspective transformation is used for spatial transformation to correct the visual and positional deviations between the preprocessed images, so that the dial areas in the preprocessed images are spatially aligned.

4. The mechanical instrument identification method according to claim 1, characterized in that, In step S3, the preset fusion weights are dynamically allocated based on the sharpness score and / or environmental interference level of each frame of the registered image. The fusion weights corresponding to the registered images whose sharpness score is greater than a first preset threshold and whose environmental interference level is less than a second preset threshold are less than 40%.

5. The mechanical instrument identification method according to claim 1, characterized in that, The target detection model used in step S4 is the YOLOv10 model, and the ECA attention mechanism is introduced into the network structure of the YOLOv10 model. The loss function used by the target detection model is the CIoU loss function, and the anchor frame size of the target detection model is adjusted according to the dial specifications of the mechanical instrument to be identified.

6. The mechanical instrument identification method according to claim 1, characterized in that, In step S4, multiple candidate contours are obtained by contour detection and contour extraction within the dial area. Then, the multiple candidate contours are filtered based on the preset pointer feature conditions to obtain the effective pointer contour.

7. The mechanical instrument identification method according to claim 6, characterized in that, The preset pointer feature conditions in step S4 include shape filtering conditions, grayscale filtering conditions, and position filtering conditions. The shape filtering condition is that the candidate contour is a thin, elongated frame. The grayscale filtering condition is that the grayscale value of the candidate contour is within a preset pointer grayscale range. The position filtering condition is that the candidate contour is located within a region defined by a preset range position.

8. The mechanical instrument identification method according to claim 1, characterized in that, After performing step S5, the method further includes: If the valid pointer profile is not extracted in step S4 or the deviation of the instrument reading from the corresponding fixed normal operating reading in step S5 exceeds a preset abnormal threshold, then the instrument is determined to be abnormal.

9. A mechanical instrument identification system based on multi-view fusion, characterized in that, The mechanical instrument identification method according to any one of claims 1-8 includes: The multi-view image acquisition module is used to simultaneously acquire multiple frames of raw images of the mechanical instrument to be identified by at least two monitoring devices deployed in different locations. The preprocessing and registration module is connected to the multi-view image acquisition module. It is used to preprocess each frame of the original image and use the preprocessed image with the highest resolution as the reference image to perform feature matching and spatial transformation to obtain multi-frame registered images. The image enhancement and dial recognition module is connected to the preprocessing and registration module. It is used to perform weighted fusion and image enhancement on multiple registered images to obtain an enhanced image, and to identify and locate the dial area based on the target detection model. The pointer segmentation and anomaly determination module, connected to the image enhancement and dial recognition module, is used to extract and filter valid pointer outlines within the dial area, and then combine them with the operating status data of the mechanical instrument to be identified to perform reading verification and anomaly determination to obtain the instrument reading.

Images