Pointer type instrument reading method based on image processing and target detection
By using image processing and target detection technologies, and leveraging YOLO-Pose and YOLOv8S models, the reading of pointer-type instruments is automated, solving the problem of low efficiency in manual reading and enabling fast and accurate instrument reading in complex scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-03
- Publication Date
- 2026-03-10
AI Technical Summary
In industrial production sites, existing technologies rely on manual inspections for reading pointer-type instruments, which is inefficient and cannot meet the needs of accurate detection in complex scenarios.
An image processing and target detection-based approach is adopted, using YOLO-Pose and YOLOv8S models to detect key points of the instrument, combined with OpenCV contour detection and MINST handwritten dataset for digit recognition, and automated reading is achieved by calculating the angle between the instrument pointer and the scale and the range value.
It enables rapid and accurate identification of various types of pointer instruments in complex scenarios. The model is lightweight and suitable for industrial production monitoring.
Smart Images
Figure CN121640435A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the fields of image processing and deep learning technologies, and in particular to a pointer-type instrument reading method based on image processing and object detection. Background Technology
[0002] With the rapid development of instrumentation technology, pointer-type instruments are widely used in industrial production processes such as petroleum and chemical industries. Effective monitoring of industrial production processes requires timely and accurate instrument readings. Currently, pointer instrument readings in industrial production sites still largely rely on manual inspection and reading. However, due to the complex environment of industrial production sites and the low efficiency of manual reading, manual inspection and reading alone cannot meet the current needs of industrial production monitoring.
[0003] With the development of technologies such as industrial automation, image processing, and deep learning, methods have been proposed to replace manual inspection with technologies such as robot inspection and to read instrument readings based on image processing and deep learning algorithms. However, traditional image processing methods have a large computational load and low accuracy, making it difficult to achieve accurate detection of pointer instruments in complex scenarios. Summary of the Invention
[0004] To address the aforementioned issues, this disclosure provides a pointer-type instrument reading method based on image processing and target detection.
[0005] The technical solution adopted in this disclosure is:
[0006] A pointer-type instrument reading method based on image processing and target detection, the specific implementation steps of which are as follows:
[0007] Acquire the visible light image of the pointer-type instrument to be read as the raw image;
[0008] Cropping the original image yields a pointer-style instrument sub-image;
[0009] The pointer instrument sub-images are labeled and a pointer instrument key point detection dataset is created to train the YOLO-Pose model; the YOLO-Pose model is used to detect the pointer instrument sub-images to obtain key point information of the instrument pointer and scale, and position information of the instrument range.
[0010] Based on the position information of the instrument range, the instrument range image is obtained by cropping the pointer-type instrument sub-image;
[0011] Contour detection and segmentation are performed on the instrument range image to obtain a segmented image for digit recognition;
[0012] The numbers in the segmented image are read and stitched together to obtain the instrument range reading;
[0013] Correct the instrument range reading to obtain an accurate instrument range value;
[0014] Based on the key point information of the instrument pointer and scale, the positioning of the instrument pointer and scale is determined, thereby obtaining the angle θ between the instrument pointer positioning and the instrument minimum value scale positioning. p The angle θ between the maximum and minimum scale settings of the instrument; determine the maximum and minimum range values (_max and _min) based on the instrument's range values; based on the angle θ... p The included angle θ, the maximum range value _max, and the minimum range value _min are used to calculate the final instrument reading L. p .
[0015] Furthermore, the visible light image is captured at a fixed monitoring point.
[0016] Furthermore, the cropping of the original image to obtain the pointer-type instrument sub-image includes:
[0017] The original images were labeled and a pointer-type meter detection dataset was created for training the YOLOv8S model;
[0018] The YOLOv8S model was used to detect the original image and obtain the position information of the pointer instrument image;
[0019] Based on the position information of the pointer instrument image, the original image is cropped to obtain the pointer instrument sub-image.
[0020] Furthermore, the step of annotating the original images and creating a pointer-type instrument detection dataset includes:
[0021] The original image was annotated using labeling software, and the annotated object was a pointer-type instrument;
[0022] Data augmentation is performed on the labeled original images, and the augmented original images are used to construct a pointer instrument detection dataset.
[0023] The pointer instrument detection dataset is divided into training, validation, and test sets for training and testing of the YOLOv8S deep learning network model.
[0024] Furthermore, the step of annotating the pointer instrument sub-image and creating a pointer instrument key point detection dataset includes:
[0025] Labeling software was used to label the sub-images of pointer-type instruments, with the labeling objects being the instrument pointer, instrument scale, and instrument range;
[0026] Data augmentation is performed on the labeled pointer instrument sub-images, and the pointer instrument key point detection dataset is constructed using the data augmented pointer instrument sub-images;
[0027] The key point detection dataset for pointer instruments is divided into training, validation, and test sets for training and testing the YOLO-Pose deep learning network model.
[0028] Furthermore, the contour detection and segmentation of the instrument range image specifically includes:
[0029] Using contour detection technology in OpenCV, contour detection and segmentation are performed on the instrument range image.
[0030] Furthermore, the reading and splicing of numbers in the segmented image specifically includes:
[0031] A digit recognition model was trained using the MINST handwritten dataset, and then used to read and stitch together digits from segmented images.
[0032] Furthermore, the correction of the instrument range reading specifically includes:
[0033] The instrument range reading is corrected based on the principle of arithmetic sequence.
[0034] Furthermore, the digit recognition model is trained using the MINST handwritten dataset, and the loss function selected during the training process is Cross-Entropy Loss.
[0035] Furthermore, the formula for the final reading of the computing instrument is as follows:
[0036]
[0037] Among them, L p For the final instrument reading, θ p θ is the angle between the instrument pointer positioning and the instrument minimum scale positioning, θ is the angle between the instrument maximum scale positioning and the instrument minimum scale positioning, and _max and _min are the maximum and minimum values of the range, respectively.
[0038] Compared with the prior art, this disclosure has the following advantages:
[0039] This disclosure provides a pointer instrument reading method based on image processing and target detection. This invention can identify various models of pointer instruments in complex scenarios, and the model is lighter, faster, and more accurate, making it widely applicable to industrial production monitoring. Attached Figure Description
[0040] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0041] Figure 1 A schematic diagram of a pointer-type instrument reading method based on image processing and target detection according to an embodiment of the present disclosure is shown;
[0042] Figure 2 A flowchart of a pointer-type instrument reading method based on image processing and target detection according to an embodiment of the present disclosure is shown;
[0043] Figure 3 A schematic diagram of the target detection result of a pointer-type instrument reading method based on image processing and target detection according to an embodiment of the present disclosure is shown;
[0044] Figure 4 A schematic diagram of the pointer angle is shown for a pointer-type instrument reading method based on image processing and target detection according to an embodiment of the present disclosure;
[0045] Figure 5 A schematic diagram of instrument range segmentation is shown for a pointer-type instrument reading method based on image processing and target detection according to an embodiment of the present disclosure.
[0046] Figure 6 The diagram illustrates the segmented image reading, stitching, and correction results of a pointer-type instrument reading method based on image processing and target detection according to an embodiment of the present disclosure.
[0047] Figure 7 The diagram illustrates the minimum and maximum values of the instrument scale in a pointer-type instrument reading method based on image processing and target detection according to an embodiment of the present disclosure. Detailed Implementation
[0048] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.
[0049] In one embodiment of the present invention, a pointer-type instrument reading method based on image processing and target detection is proposed, such as... Figures 1 to 7 As shown, it includes the following steps:
[0050] The visible light image of the pointer instrument to be read is collected as the raw image. The visible light image is captured at a fixed monitoring point.
[0051] The original images are labeled using labeling software. The labeled objects are pointer meters, and the labeling format is a region selection box. Data augmentation is then performed on the labeled original images, and the pointer meter detection dataset is constructed using the data-augmented original images. Data augmentation methods can include flipping, mirroring, and adding noise.
[0052] The pointer meter detection dataset is divided into training, validation, and test sets for training and testing the YOLOv8S deep learning network model. The YOLOv8S model is used to detect the original images to obtain the position information of the pointer meter images.
[0053] Based on the position information of the pointer instrument image, the original image is cropped to obtain pointer instrument sub-images. The cropping information for the pointer instrument sub-images is as follows:
[0054]
[0055] Among them, (x min ,y min ) and (x max y max (W, H) represent the top left and bottom right corner positions of the bounding box of the instrument sub-image in the original image, respectively, and (W, H) represent the width and height of the original image, respectively.
[0056] Labeling software is used to label the sub-images of pointer instruments. The labeling objects are the instrument pointer, instrument scale, and instrument range. The labeling form of the instrument pointer, instrument scale, and instrument range is a region selection box, and the labeling form of key points is points. Data augmentation is performed on the labeled pointer instrument sub-images, and the pointer instrument key point detection dataset is constructed using the data augmented pointer instrument sub-images. Among them, the data augmentation methods can be flipping, mirroring, and adding noise.
[0057] The key point detection dataset for pointer instruments is divided into training, validation, and test sets for training and testing the YOLO-Pose deep learning network model. The YOLO-Pose model is used to detect key point information of pointer instruments and scales, as well as the position information of the instrument range.
[0058] Based on the instrument range position information, the pointer-type instrument sub-image is cropped to obtain the instrument range image. The instrument range image cropping information is as follows:
[0059]
[0060] Among them, (x n，min y n，min ) and (x max y max (W) represents the top-left and bottom-right corner positions of the bounding box of the instrument range image in the pointer-type instrument sub-image. i H i ) represent the width and height of the pointer-type instrument sub-image, respectively.
[0061] Using contour detection technology in OpenCV, contour detection and segmentation are performed on the instrument range image to obtain a segmented image for digit recognition.
[0062] A digit recognition model was trained using the MINST handwritten dataset. Cross-Entropy Loss was chosen as the loss function during the training process. The digit recognition model was then used to read and stitch together the digits in the segmented images to obtain the instrument range reading.
[0063] Based on the principle of arithmetic sequences, the differences between adjacent and intermittent readings are calculated, and the instrument range readings are corrected after comparative analysis to obtain an accurate instrument range value. In this embodiment, the accurate instrument range value can be between 0 and 16.
[0064] Based on the key point information of the instrument pointer and scale, the positioning of the instrument pointer and scale is determined, thereby obtaining the angle θ between the instrument pointer positioning and the instrument minimum value scale positioning. p The angle θ between the maximum and minimum scale settings of the instrument; determine the maximum and minimum range values (_max and _min) based on the instrument's range values; based on the angle θ... p The included angle θ, the maximum range value _max, and the minimum range value _min are used to calculate the final instrument reading L. p The formula for calculating the final reading of the instrument is as follows:
[0065]
[0066] Among them, L p For the final instrument reading, θ p θ is the angle between the instrument pointer positioning and the instrument minimum scale positioning, θ is the angle between the instrument maximum scale positioning and the instrument minimum scale positioning, and _max and _min are the maximum and minimum values of the range, respectively.
[0067] From the above formula, we can see that:
[0068]
[0069] Among them, (x p ,y p ) represents the pointer's endpoint position information, (x o yo ) represents the pointer's origin position information, (x min y min (x) represents the position information of the minimum scale point of the instrument. max y max ) represents the position information of the instrument's maximum scale point. When θ p When θ = 156.875, θ = 268.21, _max = 16, and _min = 0, the reading calculation formula of this invention can be used to obtain L. p =9.3584.
[0070] Although the present disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure.
Claims
1. A pointer-type instrument reading method based on image processing and object detection, characterized by, The method comprises the following steps: Collecting a visible light image of a pointer instrument to be read as an original image; Cutting the original image to obtain a pointer instrument sub-image; Labeling the pointer instrument sub-image and making a pointer instrument key point detection dataset for training a YOLO-Pose model; using the YOLO-Pose model to detect the pointer instrument sub-image to obtain key point information of an instrument pointer and an instrument scale and position information of an instrument range; According to the position information of the instrument range, the pointer instrument sub-image is cut to obtain an instrument range image; Performing contour detection and segmentation on the instrument range image to obtain a segmented image for digital recognition; Reading and splicing the numbers in the segmented image to obtain an instrument range reading; Correcting the instrument range reading to obtain an accurate instrument range value; According to the instrument pointer and the instrument scale key point information, the positioning of the instrument pointer and the instrument scale is determined, so as to obtain the included angle θ of the instrument pointer positioning and the instrument minimum scale positioning p , the included angle θ of the instrument maximum scale positioning and the instrument minimum scale positioning; according to the instrument range value, the maximum value_max and the minimum value_min of the range are determined; According to the included angle θ p , the included angle θ, the maximum and minimum values of the range _max and _min, so as to calculate the final reading L of the instrument p .
2. A pointer type instrument reading method based on image processing and object detection according to claim 1, characterized in that, The visible light image is taken at a fixed monitoring point.
3. The pointer type instrument reading method based on image processing and object detection according to claim 1, characterized in that, The cutting of the original image to obtain the pointer instrument sub-image comprises: Labeling the original image and making a pointer instrument detection dataset for training a YOLOv8S model; Using the YOLOv8S model to detect the original image to obtain position information of the pointer instrument image: According to the position information of the pointer instrument image, the original image is cut to obtain the pointer instrument sub-image.
4. The pointer type instrument reading method based on image processing and object detection according to claim 3, characterized in that, The labeling of the original image and the making of the pointer instrument detection dataset comprise: Labeling the original image using a label labeling software, and the labeling object is a pointer instrument; Performing data enhancement on the labeled original image, and using the data-enhanced original image to construct a pointer instrument detection dataset; Dividing the pointer instrument detection dataset into a training set, a validation set and a test set for model training and testing of a deep learning network YOLOv8S.
5. The pointer type instrument reading method based on image processing and object detection according to claim 1, characterized in that, The labeling of the pointer instrument sub-image and the making of the pointer instrument key point detection dataset comprise: Labeling the pointer instrument sub-image using a label labeling software, and the labeling object is an instrument pointer, an instrument scale and an instrument range; Performing data enhancement on the labeled pointer instrument sub-image, and using the data-enhanced pointer instrument sub-image to construct a pointer instrument key point detection dataset; Dividing the pointer instrument key point detection dataset into a training set, a validation set and a test set for model training and testing of a deep learning network YOLO-Pose.
6. The pointer type instrument reading method based on image processing and object detection according to claim 1, characterized in that, The contour detection and segmentation of the instrument range image specifically comprise: Using a contour detection technology in Opencv to perform contour detection and segmentation on the instrument range image.
7. The pointer type instrument reading method based on image processing and object detection according to claim 1, characterized in that, The reading and splicing of the numbers in the segmented image specifically comprise: Training a digital recognition model using a MINST handwritten dataset, and using the digital recognition model to read and splice the numbers in the segmented image.
8. The pointer type instrument reading method based on image processing and object detection according to claim 1, characterized in that, The correction of the instrument range reading specifically comprises: Correcting the instrument range reading based on the principle of an arithmetic sequence.
9. The pointer type instrument reading method based on image processing and object detection according to claim 7, characterized in that, The loss function selection of the training process of the digital recognition model is Cross-Entropy Loss.
10. The pointer type instrument reading method based on image processing and object detection according to claim 1, characterized in that, The formula for calculating the final reading of the instrument is as follows: where L p is the final meter reading, θ p is the angle between the meter pointer positioning and the meter minimum scale positioning, θ is the angle between the meter maximum scale positioning and the meter minimum scale positioning, _max, _min are the maximum and minimum values of the range, respectively.