Pointer instrument positioning and identification method and system based on target detection
By applying a computer vision method based on object detection in chemical enterprises, combined with YOLO and Faster-RCNN models, the problems of low reading efficiency and safety risks of pointer instruments in chemical enterprises are solved, and efficient and accurate instrument positioning and identification are achieved, reducing the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number of times the number Resource consumption.
Patent Information
- Application Number
- CN202411083318.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-08
- Publication Date
- 2024-11-22
AI Technical Summary
Traditional pointer instrument reading methods in chemical enterprises are inefficient, poor real-time and personal safety risks. The existing intelligent technology consumes a lot of resources when running on site and the recognition accuracy is affected by the environment.
Using a computer vision method based on object detection, a pointer instrument positioning and recognition system is constructed through the combination of YOLO and Faster-RCNN models, the pointer instrument positioning and recognition system is constructed, and the CSPDarknet53 and Resnet50 networks are used, and the dial positioning and pointer recognition are carried out through data enhancement and The labeling tool forms a sample library to achieve fast and accurate identification of instrument readings.
It realizes efficient and accurate pointer instrument positioning and recognition under different lighting and slight occlusion, reduces resource consumption, improves identification accuracy, and solves the scope limitations, low efficiency and security risks of traditional methods.
Smart Images

Figure CN119007172A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer vision, and in particular to a pointer instrument positioning and recognition method and system based on target detection. Background Art
[0002] In chemical enterprises, data collection of instruments plays an important role in process control and safe production. Pointer instruments are deployed in large quantities in chemical enterprise parks due to their mechanical structure, no need for power supply, and low cost. Therefore, most instruments are pointer instruments, and their status will be irreplaceable for some time to come. However, the traditional reading method is that patrol personnel complete instrument reading records along a predetermined route. This method has problems such as limited scope, low efficiency, poor real-time performance, and personal safety risks. With the development of intelligent technology, some technologies have been applied to pointer instruments, but most technologies are operated in experimental environments, which will consume a lot of resources when operated on site, and the recognition accuracy is affected by the environment. Therefore, a method and system for positioning and identifying pointer instruments in chemical enterprises based on target detection is needed to promote the automation and intelligent construction of chemical enterprise patrol inspections, so as to achieve the goals of reducing resources, reducing safety risks, and improving recognition accuracy. Summary of the invention
[0003] Purpose of the invention: The present invention aims to provide a pointer instrument positioning and identification method with high reading accuracy and efficiency based on target detection; another purpose of the present invention is to provide a pointer instrument positioning and identification system based on target detection.
[0004] Technical solution: The pointer instrument positioning and identification method based on target detection described in the present invention comprises the following steps:
[0005] (1) Collect the original instrument image, pre-process it, and then establish a sample library;
[0006] (2) Build and train the instrument positioning and pointer recognition model;
[0007] (3) Collect real-time instrument images, clean the data, and use the trained instrument positioning and pointer recognition model to perform recognition operations;
[0008] (4) Correcting the instrument image identified in step (3) and calculating the angle between the pointer and the horizontal direction;
[0009] (5) Determine the instrument reading corresponding to the pointer based on the instrument range, the full range coverage angle, and the angle obtained in step (4).
[0010] Furthermore, step (1) is specifically as follows:
[0011] Execute the capture operation through freely set inspection tasks to obtain original high-resolution images of instruments at different time periods and preset positions;
[0012] The original high-resolution images of the above instruments are screened to obtain clear images as data to be annotated;
[0013] Use data annotation tools to annotate the instrument positions and pointer positions in the clear image and record the annotation information;
[0014] Use data enhancement tools to perform expansion operations, increase the sample size, and form a sample library.
[0015] Furthermore, in step (2), based on the YOLO model and the Faster-RCNN model, an instrument positioning and pointer recognition model is constructed; wherein the YOLO model is used to locate the dial, and the Faster-RCNN model is used to recognize the pointer.
[0016] Furthermore, the backbone network of the YOLO model is CSPDarknet53, the SPP module is used as an additional module of Neck, PANet is used as the feature fusion module of Neck, and the prediction box screening algorithm is changed from MNS to DIOU_NMS;
[0017] The backbone network of the Faster-RCNN model is Resnet50. The feature map is input into the RPN network after convolution. The coordinates of the candidate box are extracted and it is determined whether the foreground image is included. Linear regression is performed on the filtered generated box to obtain the regression coefficient for correcting the position of the generated box. After executing RPN, a mask layer is added to enhance the image center feature. The ROIPooling layer is used to perform geometric transformation on the image, and the box-selected image is converted into a fixed-ratio feature map. The fully connected layer is classified to finally obtain the pointer coordinate position.
[0018] Furthermore, the annotation information includes rectangle coordinates, label name, image length and width, and image name.
[0019] Furthermore, the expansion operation is as follows: using an open source data enhancement tool, the expansion operation is performed by any method of flipping, random radiation transformation, combination and random selection, and cropping, the expanded data is added to the original image data, and combined with the annotation file to form a sample library.
[0020] Furthermore, in step (3), data cleaning is specifically performed as follows: the blur degree is calculated using the Laplacian algorithm, and if it is less than a set threshold, the collected real-time instrument image is judged to be a blurred image and re-sampled.
[0021] Furthermore, in step (4), the perspective transformation method is used to implement instrument calibration, and the angle between the pointer and the horizontal direction is calculated through the pointer position coordinates.
[0022] The system of the present invention according to the above-mentioned method for positioning and identifying a pointer instrument based on target detection comprises:
[0023] An image acquisition unit, used for acquiring instrument images captured by a camera device;
[0024] A data preprocessing unit, used for preprocessing the collected instrument images;
[0025] A data annotation unit is used to annotate the instrument position and pointer position in the clear image and record the annotation information;
[0026] The model training unit divides the sample library into training set, validation set and test set in proportion, and trains the selected model;
[0027] A model deployment unit is configured to deploy the trained model to a needle instrument positioning and recognition application.
[0028] Furthermore, the model deployment unit includes
[0029] Data cleaning unit to remove blurry and poor-quality images;
[0030] A correction unit, used for correcting the angle between the calculation pointer and the horizontal direction;
[0031] The calculation unit is used to determine the instrument reading corresponding to the pointer according to the instrument range, the full range coverage angle and the angle obtained by calibration calculation.
[0032] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: 1. The present invention uses the target detection technology in computer vision to realize the positioning and recognition of pointer instruments under different lighting conditions and slight occlusion; while considering the recognition speed and accuracy, the YOLO model and Faster-RCNN model are used to realize the rapid positioning and accurate recognition of pointer instruments; 2. The YOLO model of the present invention locates the dial position, uses the SPP network to increase the network's receptive field, and PANet retains spatial information to enhance the instance segmentation process, and changes the prediction box screening algorithm from MNS to DIOU_NMS, so as to accurately extract the located dial from the image according to the coordinates; 3. The pointer is identified by the Faster-RCNN model, and a layer of mask is added after the RPN is executed to enhance the image center feature, so that the pointer recognition result is more sensitive to the center of the instrument and the obtained recognition result is more accurate; 4. The present invention adopts an architecture mode that separates model training and recognition deployment, and the trained model can be deployed separately on the application server, saving computing resources and disk space, and avoiding unnecessary storage and computing resources during on-site deployment; 5. The present invention solves the problems of traditional instrument reading inspection range limitations, low efficiency, poor real-time performance, and personal safety risks in chemical enterprises. By continuously enriching the sample library, the recognition accuracy can be continuously improved, which greatly improves the practicality. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1 is a flow chart of the present invention;
[0034] Figure 2 Flow chart for instrument positioning and pointer identification;
[0035] Figure 3 It is a schematic diagram of the instrument calibration result in an embodiment of the present invention;
[0036] Figure 4 A schematic diagram of a pointer position recognition result in an embodiment of the present invention;
[0037] Figure 5 It is a schematic diagram of the dial positioning result in an embodiment of the present invention. DETAILED DESCRIPTION
[0038] The present invention will be further described below in conjunction with the accompanying drawings.
[0039] The pointer instrument positioning and recognition method based on target detection of the present invention comprises the following steps: model training and recognition deployment;
[0040] (1) Model training includes the following steps:
[0041] (11) Collect historical instrument image data
[0042] First, a polling mechanism is used to set the "preset position" for the camera equipment to shoot the pointer instrument on an inspection route, and then the inspection plan start time is set to collect image samples 24 hours a day, and collect pointer instrument image data samples under different lighting and different environments over a period of time.
[0043] (12) Instrument image data annotation
[0044] Label the qualified samples collected in step (11). Since the "preset position" of the camera device rotates in the same position under different lighting and environments, these images have the same labeling box and label. Therefore, it is only necessary to use open source tools to label different "preset positions" at a certain time point. The obtained rectangular coordinates, label name, image length and width, image name and other information are stored in a standard json format file.
[0045] (13) Establishing an instrument image data sample library
[0046] Use open source data enhancement tools to perform operations such as flipping, random radial transformation, combination and random selection, and cropping on images to enrich the number of samples. At the same time, eliminate negative samples generated during the transformation process, and store the image data and the corresponding annotation files in a non-relational database to form a sample library.
[0047] (14) Divide the sample library into training set, validation set and test set
[0048] Before model training, the allocation ratio of the training set, validation set, and test set is determined based on experience according to the sample size of pointer instrument images in the sample library.
[0049] (15) Establish and train instrument positioning and pointer recognition models
[0050] Combination Figure 2 The YOLO model in the target detection algorithm is used to locate the position of the dial, and the located dial is extracted from the image according to the coordinates as the input of the improved Faster-RCNN model, so that the pointer position can be located more accurately. At the same time, by continuously increasing the sample size and updating the model parameters, the increase in adaptation scenarios can be achieved.
[0051] Among them, the YOLO model includes:
[0052] CSPDarknet53 is used as the backbone network, where the input is a 608*608*3 tensor, which contains five large residual blocks and the corresponding small residual units are 1, 2, 8, 8, and 4 respectively. This network can enhance the learning ability of CNN and maintain lightweight while having good accuracy.
[0053] SPP (Spatial Pyramid Pooling) is used as an additional module of Neck, and PANet is used as the feature fusion module of Neck. The SPP network is used to increase the receptive field of the network, and PANet mainly enhances the instance segmentation process by retaining spatial information.
[0054] In the YOLO model, the main improvement is made to the regression box position loss function CIOU_Loss during training, and the prediction box screening algorithm is changed from MNS to DIOU_NMS.
[0055] Among them, the improved Faster-RCNN includes:
[0056] Using Resnet50 as the backbone network to extract image data features will reduce the pixels of the input original image by a fixed ratio.
[0057] After convolution, the feature map is input into the RPN network to extract the coordinates of the candidate box and determine whether it contains the foreground image. In order to make the generated box closer to the real box, it is necessary to perform linear regression on the filtered generated box to obtain the regression coefficient for correcting the position of the generated box. After executing RPN, a layer of mask is added to enhance the image center feature.
[0058] Then, the ROIPooling layer is used to perform geometric transformation on the image, convert the framed image into a fixed-scale feature map, and perform full-connection layer classification to finally obtain the pointer coordinate position.
[0059] (2) Identification deployment includes the following steps:
[0060] (21) Collect real-time instrument image data
[0061] Use the streaming media server to be compatible with the camera equipment of mainstream manufacturers, and obtain the current pointer instrument equipment image through the "preset position" setting of the camera equipment.
[0062] (22) Instrument image data cleaning
[0063] Using the image captured in step (21), the blur degree is calculated using the Laplacian algorithm. If it is less than the set threshold, the original image is judged to be a blurred image, discarded and resampled.
[0064] (23) Deploy the trained recognition model
[0065] Using container technology to deploy the model training step (15), the trained model is input into the cleaned image in the previous step, and the position of the pointer instrument and its pointer position are obtained. Figure 3 and Figure 4 shown.
[0066] (24) Instrument calibration and calculation of pointer angle
[0067] The instrument calibration result is achieved by using the perspective transformation method. Figure 5 As shown, the angle between the pointer and the horizontal direction is calculated using the pointer position coordinates.
[0068] (25) Output instrument reading recognition results
[0069] Calculate the instrument reading corresponding to the pointer based on the instrument range, full range coverage angle and the angle obtained in the previous step.
[0070] The system of the present invention according to the above-mentioned method for positioning and identifying a pointer instrument based on target detection comprises:
[0071] An image acquisition unit, used for acquiring instrument images captured by a camera device;
[0072] A data preprocessing unit, used for preprocessing the collected instrument images;
[0073] A data annotation unit is used to annotate the instrument position and pointer position in the clear image and record the annotation information;
[0074] The model training unit divides the sample library into training set, validation set and test set in proportion, and trains the selected model;
[0075] A model deployment unit is configured to deploy the trained model to a needle instrument positioning and recognition application.
[0076] Wherein, the model deployment unit includes
[0077] Data cleaning unit to remove blurry and poor-quality images;
[0078] A correction unit, used for correcting the angle between the calculation pointer and the horizontal direction;
[0079] The calculation unit is used to determine the instrument reading corresponding to the pointer according to the instrument range, the full range coverage angle and the angle obtained by calibration calculation.
Claims
1. A pointer instrument positioning and recognition method based on target detection, characterized in that: The following steps are involved: (1) Collect the original instrument image, pre-process it, and then establish a sample library; (2) Build and train the instrument positioning and pointer recognition model; (3) Collect real-time instrument images, clean the data, and use the trained instrument positioning and pointer recognition model to perform recognition operations; (4) Correcting the instrument image identified in step (3) and calculating the angle between the pointer and the horizontal direction; (5) Determine the instrument reading corresponding to the pointer based on the instrument range, the full range coverage angle, and the angle obtained in step (4).
2. According to claim 1, the method for positioning and identifying a pointer instrument based on target detection is characterized in that: Step (1) is as follows: Execute the capture operation through freely set inspection tasks to obtain the original high-resolution images of the instrument at different time periods and different preset positions; The original high-resolution images of the above instruments are screened to obtain clear images as data to be annotated; Use data annotation tools to annotate the instrument positions and pointer positions in the clear image and record the annotation information; Use data enhancement tools to perform expansion operations, increase the sample size, and form a sample library.
3. The method for positioning and identifying a pointer instrument based on target detection according to claim 2 is characterized in that: In step (2), based on the YOLO model and the Faster-RCNN model, an instrument positioning and pointer recognition model is constructed; wherein the YOLO model is used to locate the dial, and the Faster-RCNN model is used to recognize the pointer.
4. The method for positioning and identifying a pointer instrument based on target detection according to claim 3 is characterized in that: The backbone network of the YOLO model is CSPDarknet53, the SPP module is used as the additional module of Neck, PANet is used as the feature fusion module of Neck, and the prediction box screening algorithm is changed from MNS to DIOU_NMS; The backbone network of the Faster-RCNN model is Resnet50. The feature map is input into the RPN network after convolution. The coordinates of the candidate boxes are extracted and it is determined whether the foreground image is included. A linear regression is performed on the filtered generated boxes to obtain the regression coefficients for correcting the position of the generated boxes. After executing RPN, a layer of mask is added to enhance the image center features. The ROIPooling layer is used to perform geometric transformation on the image, convert the framed image into a fixed-scale feature map, and perform full-connection layer classification to finally obtain the pointer coordinate position.
5. The method for positioning and identifying a pointer instrument based on target detection according to claim 2, characterized in that: The annotation information includes rectangle coordinates, label name, image length and width, and image name.
6. The method for positioning and identifying a pointer instrument based on target detection according to claim 2, characterized in that: The specific expansion operation is as follows: using open source data enhancement tools, the expansion operation is performed through any of the methods of flipping, random radiation transformation, combination and random selection, and cropping, the expanded data is added to the original image data, and combined with the annotation file to form a sample library.
7. The method for positioning and identifying a pointer instrument based on target detection according to claim 4, characterized in that: In step (3), data cleaning is specifically as follows: the blur degree is calculated using the Laplacian algorithm. If it is less than a set threshold, the collected real-time instrument image is judged to be a blurred image and re-sampled.
8. The method for positioning and identifying a pointer instrument based on target detection according to claim 4, characterized in that: In step (4), the perspective transformation method is used to implement instrument calibration, and the angle between the pointer and the horizontal direction is calculated using the pointer position coordinates.
9. A system for the pointer instrument positioning and identification method based on target detection according to any one of claims 1 to 8, characterized in that: include An image acquisition unit, used for acquiring instrument images captured by a camera device; A data preprocessing unit, used for preprocessing the collected instrument images; A data annotation unit is used to annotate the instrument position and pointer position in the clear image and record the annotation information; The model training unit divides the sample library into training set, validation set and test set in proportion, and trains the selected model; A model deployment unit is configured to deploy the trained model to a needle instrument positioning and recognition application.
10. The system according to claim 9, characterized in that: The model deployment unit includes Data cleaning unit to remove blurry and poor-quality images; A correction unit, used for correcting the angle between the calculation pointer and the horizontal direction; The calculation unit is used to determine the instrument reading corresponding to the pointer according to the instrument range, the full range coverage angle and the angle obtained by calibration calculation.
