Improved YOLOv8n-based pointer instrument detection identification reading method and system

Abstract

The invention discloses a pointer instrument detection and identification reading method based on improved YOLOv8n, and the method comprises the steps: obtaining the image data of a pointer instrument, and sequentially carrying out the data expansion and dial marking, and obtaining a first marking data set; performing optimization adjustment based on partial convolution modules in a backbone network and a neck network of the YOLOv8n network structure and a convolution module in a head network to obtain an improved YOLOv8n network model; training the improved YOLOv8n network model based on the first annotation data set to obtain a dial region detection model; inputting the first annotation data set into a dial region detection model to obtain a dial data set; performing secondary annotation based on the dial data set to obtain a second annotation data set, and training the dial area detection model to obtain a target frame detection model; inputting the to-be-detected instrument image into the target frame detection model to obtain a target frame; and obtaining the pointer reading of the to-be-measured instrument based on the target frame. And the reading identification efficiency and accuracy of the pointer type instrument are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of instrument reading recognition, and more specifically to a pointer instrument detection and recognition reading method and system based on improved YOLOv8n. Background Art

[0002] Pointer instruments are usually round or square, with a simple and beautiful appearance. The rotating pointer points to the numbers on the dial to reflect the final reading. The visual effect is vivid and vivid. It is widely used in power systems, oil systems, etc. The traditional method of manually checking the status of instruments and reading data not only consumes a lot of manpower and financial resources, but also has low inspection efficiency, large errors in recorded results, and cannot work normally in bad weather conditions. With the continuous development and innovation of artificial intelligence technology, more and more industries have begun to use inspection robots for automatic readings. However, due to the complexity of the environment during the inspection process, the photos taken by the robots are easily affected by dust, shooting angles, weather, brightness, etc., resulting in inaccurate final readings.

[0003] Deep learning algorithms have been widely used in recent years. The most popular deep learning algorithms can be divided into two categories: single-stage and dual-stage. The more popular single-stage algorithms are the YOLO (you only look once) series of algorithms and SSD (singleshot multibox detector), and the more popular dual-stage algorithms are Mask R-CNN and Faster R-CNN. Liu Yan et al. used the YOLOv5s network and introduced GhostNetV2 as the backbone network for lightweight design, then intercepted the dial area, collected pointers and other images through DeepLabV3+ semantic segmentation, and finally calculated the instrument readings. Although this method makes the model relatively lightweight, its accuracy needs to be improved. Li et al. proposed an algorithm design for indicator recognition. The algorithm mainly includes scale area extraction, scale line extraction, dial center determination, zero scale line determination and indicator recognition. This method has a large number of model parameters and a slow recognition speed. Peng Daogang et al. used YOLOv4 to extract the dial area, and then used the improved Hough transform to detect the pointer and finally completed the reading of the pointer instrument. Although this method can accurately read the instrument reading, the process is relatively cumbersome and the amount of calculation is large. In the detection of pointer instruments, although great progress has been made in reducing errors and improving practicality, the accuracy and environmental adaptability need to be improved.

[0004] Therefore, how to improve the reading recognition efficiency and accuracy of pointer-type instruments is an urgent problem to be solved by those skilled in the art. Summary of the invention

[0005] In view of this, the present invention provides a pointer instrument detection and recognition reading method and system based on improved YOLOv8n, which improves the recognition efficiency and accuracy of pointer instrument readings.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A method for detecting and identifying readings of a pointer instrument based on improved YOLOv8n, comprising:

[0008] Acquire pointer instrument image data and perform data expansion and dial annotation in sequence to obtain a first annotated data set;

[0009] Based on the YOLOv8n network structure, the backbone network, some convolution modules in the neck network, and the convolution modules in the head network are optimized and adjusted to obtain an improved YOLOv8n network model;

[0010] The improved YOLOv8n network model is trained based on the first labeled data set to obtain a dial area detection model;

[0011] Based on the first labeled data set, input into the dial area detection model to obtain a dial data set;

[0012] Perform secondary annotation based on the dial data set to obtain a second annotated data set and train the dial area detection model to obtain a target frame detection model;

[0013] Inputting the image of the instrument to be tested into the target frame detection model to obtain the target frame;

[0014] The pointer reading of the instrument to be tested is obtained based on the target frame.

[0015] Preferably, the data expansion specifically includes:

[0016] Divide the pointer instrument image data into A group of different data subsets;

[0017] Randomly select the data subsets in group A-1 and perform different image enhancement operations to obtain different expanded data subsets in group A-1;

[0018] An expanded data set is formed based on all the expanded data subsets and the data subset that has not been subjected to the image enhancement operation.

[0019] Preferably, the optimization adjustment specifically includes:

[0020] The second convolution module, the third convolution module, the fourth convolution module and the fifth convolution module in the backbone network based on the YOLOv8n network structure are replaced by the first Double-Conv module, the second Double-Conv module, the third Double-Conv module and the fourth Double-Conv module respectively;

[0021] The sixth convolution module and the seventh convolution module in the neck network based on the YOLOv8n network structure are replaced by the fifth Double-Conv module and the sixth Double-Conv module respectively;

[0022] All standard convolution modules in all detection heads in the head network based on the YOLOv8n network structure are replaced with GhostConv convolution modules.

[0023] Preferably, all Double-Conv modules include G groups of convolution filters, and the number of input channels is M;

[0024] The convolution filter includes two identical branches, each of which includes: a first convolution part and a second convolution part, the first convolution part includes an M / G layer and a first convolution layer, and the first convolution layer includes a 3×3 convolution layer and a first 1×1 convolution layer connected in sequence;

[0025] The second convolution part includes a second 1×1 convolution layer of the MM / G layer, and the first 1×1 convolution layer is located above the second 1×1 convolution layer;

[0026] From the first group of convolution filters to the Gth group of convolution filters, the position of the first convolution part is shifted downward from the top by M / G layers in sequence until the first convolution part of the Gth group of convolution filters is located at the bottom.

[0027] Preferably, the computational load of the Double-Conv module is FL DC for:

[0028]

[0029] Among them, Do represents the width and height dimensions of the output feature map, M represents the number of input feature map channels, and K 2 Represents the convolution kernel size, N represents the number of convolution filters, and G represents the number of convolution groups.

[0030] Preferably, the GhostConv convolution module includes: a standard convolution layer, a linear operation layer and a splicing layer;

[0031] Input the feature map to the standard convolution layer to obtain a first feature map;

[0032] The first feature map is input into a linear operation layer for linear operation to obtain a plurality of second feature maps;

[0033] All the second feature maps are input to the splicing layer and spliced ​​by channel to obtain a fused feature map.

[0034] Preferably, the secondary annotation includes:

[0035] Based on each dial image data in the dial data set, the starting scale, the end scale, the center position and the pointer position are marked, and are marked with numbers: C1, C2, C3 and C4.

[0036] Preferably, obtaining the pointer reading of the instrument to be tested based on the target frame specifically includes:

[0037] Based on the target frame, the starting scale coordinates, the end scale coordinates, the center position coordinates and the pointer position coordinates are obtained;

[0038] Obtaining a vector V1 based on the starting scale coordinates and the circle center position coordinates;

[0039] Obtaining a vector V2 based on the pointer position coordinates and the circle center position coordinates;

[0040] Obtaining vector V3 based on the end point scale coordinates and the circle center position coordinates;

[0041] A first angle θ is calculated based on the vector V1 and the vector V2;

[0042] The second angle θ is calculated based on the vector V1 and the vector V3 max ;

[0043] Based on the first angle θ and the second angle θ max And the system range r to obtain the pointer reading of the instrument under test:

[0044]

[0045] Among them, α represents the pointer reading of the instrument to be tested.

[0046] Preferably, it also includes: using the recall rate, precision rate and average precision mean as preset evaluation indicators;

[0047] The dial area detection model and the target frame detection model are evaluated respectively based on the evaluation index.

[0048] A pointer instrument detection, recognition and reading system based on improved YOLOv8n, comprising: a first annotation module, a model improvement module, a first training module, a second training module and a result output module;

[0049] The first annotation module is used to obtain the pointer instrument image data and perform data expansion and dial annotation in sequence to obtain a first annotation data set;

[0050] The model improvement module is used to optimize and adjust the backbone network and some convolution modules in the neck network and the convolution modules in the head network based on the YOLOv8n network structure to obtain an improved YOLOv8n network model;

[0051] The first training module is used to train the improved YOLOv8n network model based on the first labeled data set to obtain a dial area detection model;

[0052] The second training module is used to input the first annotated data set into the dial area detection model to obtain a dial data set; perform secondary annotation based on the dial data set to obtain a second annotated data set and train the dial area detection model to obtain a target frame detection model;

[0053] The result output module is used to input the image of the instrument to be tested into the target frame detection model to obtain the target frame; and obtain the pointer reading of the instrument to be tested based on the target frame.

[0054] It can be seen from the above technical solutions that compared with the prior art, the present invention discloses a method and system for detecting and identifying readings of pointer-type instruments based on improved YOLOv8n, which uses the improved YOLOv8n model to locate the pointer dial, and then uses the improved YOLOv8n model to identify the start scale, center of circle, pointer, and end scale of the identified dial, which greatly improves the accuracy. It has the following beneficial effects:

[0055] 1. The present invention improves the YOLOv8n network model and uses a lightweight convolution Double-Conv to replace some standard convolution modules in the backbone network and the neck network, thereby reducing the complexity of the model and improving the detection speed of the target object; at the same time, the Ghostconv convolution module is used to replace the standard convolution module in the detection head in the head network, while ensuring the model feature extraction capability, more feature maps are generated from the existing feature maps by using cheap linear transformations, greatly reducing the required calculation amount. Based on the above improvements, the computational complexity and parameter amount of the model are greatly reduced, and the accuracy and speed are improved to a certain extent.

[0056] 2. The data augmentation technology is used to increase the diversity of training samples, so that the model can better learn the performance of pointer instruments under different lighting conditions, angle changes and other factors, thereby enhancing the robustness and generalization ability of the model in actual application scenarios.

[0057] 3. The two-stage labeling strategy (first generating the first labeled data set for preliminary training to obtain the dial area detection model; then producing a more accurate second labeled data set based on the output results of this model for further training) can effectively reduce the false detection rate and has a better capture effect on the pointer position of small targets or occluded conditions.

[0058] 4. The present invention is not only suitable for single image analysis, but also can be easily extended to real-time monitoring tasks of continuous frames in video streams, providing a powerful support tool for equipment status monitoring in industrial production environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0060] Figure 1 A flow chart of a pointer instrument detection and identification reading method based on improved YOLOv8n provided by the present invention.

[0061] Figure 2 A schematic diagram of the improved YOLOv8n network structure provided by the present invention.

[0062] Figure 3 A schematic diagram of the structure of the improved YOLOv8n network detection head provided by the present invention.

[0063] Figure 4 Schematic diagram of the original YOLOv8n network structure provided by the present invention.

[0064] Figure 5 A schematic diagram of the general structure of the Double-Conv module provided by the present invention.

[0065] Figure 6 A schematic diagram of the Double-Conv module structure including two groups of convolution filters provided by the present invention.

[0066] Figure 7 Schematic diagram of the sliding feature extraction process of the Double-Conv module provided by the present invention.

[0067] Figure 8 This is a schematic diagram of the GhostConv convolution module structure provided by the present invention.

[0068] Fig. 9 Schematic diagram of the data processing process of the GhostConv convolution module provided by the present invention.

[0069] Fig.10 This is a schematic diagram of the comparison of detection effects of different models provided by the present invention.

[0070] Fig.11 A schematic diagram of the structure of a pointer instrument detection, identification and reading system based on improved YOLOv8n provided by the present invention. DETAILED DESCRIPTION

[0071] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] YOLOv8 is the latest target detection and image segmentation algorithm released by Ultralytics on January 10, 2023. YOLOv8 uses YOLOv5 as the basic model. After continuous innovation and improvement, it not only improves the detection accuracy and has more powerful performance, but also becomes lightweight.

[0073] YOLOv8 is the latest and most advanced YOLO model, mainly used for target detection and instance segmentation tasks. Compared with the currently popular YOLOv5, YOLOv7 and other networks, YOLOv8 has improvements in structural algorithms. The YOLOv8 network is divided into five models: n, s, l, m, and x according to the model application scenario and size, among which n is a lightweight model.

[0074] The backbone of YOLOv8 uses a new C2f module to replace the previous C3 module. The C2f module parallelizes more gradient flow branches in order to obtain richer gradient flow information while ensuring lightweight, and also adds an additional Split operation. The Head part has the biggest change, from the original coupling head to a decoupling head, and from YOLOv5's Anchor-Based to Anchor-Free. The difference between Anchor-based and Anchor-free lies in whether the prior box is set during the training process. The Anchor-based strategy is very dependent on a good prior box design. For different tasks and targets with large deformations (large changes in aspect ratio), it may be necessary to manually design and adjust the prior box hyperparameters based on experience. Anchor-free is much simpler, but the simple operation may also bring problems such as poor recall performance and average detection effect on overlapping targets.

[0075] The official provides five basic models, including YOLOv8n, YOLOv8s, YOLOv8m, YOLOv8l, and YOLOv8x. However, the basic models have problems such as large amount of calculation and long calculation time in the process of instrument detection, that is, low detection efficiency. Based on this, the present invention is improved based on the YOLOv8n network model to reduce the complexity of the model, reduce the amount of calculation, and thus improve the detection speed of the target.

[0076] Example 1

[0077] like Figure 1 As shown, the embodiment of the present invention discloses a method for detecting and identifying readings of a pointer instrument based on an improved YOLOv8n, comprising:

[0078] Acquire pointer instrument image data and perform data expansion and dial annotation in sequence to obtain a first annotated data set;

[0079] Based on the YOLOv8n network structure, the backbone network, some convolution modules in the neck network, and the convolution modules in the head network are optimized and adjusted to obtain an improved YOLOv8n network model;

[0080] The improved YOLOv8n network model is trained based on the first labeled data set to obtain a dial area detection model;

[0081] Based on the first labeled data set, input into the dial area detection model to obtain a dial data set;

[0082] Perform secondary annotation based on the dial data set to obtain a second annotation data set and train the dial area detection model to obtain a target box detection model;

[0083] Input the image of the instrument to be tested into the target frame detection model to obtain the target frame;

[0084] The pointer reading of the instrument to be tested is obtained based on the target frame.

[0085] Example 2

[0086] The embodiment of the present invention discloses a method for detecting and identifying readings of a pointer instrument based on an improved YOLOv8n, comprising:

[0087] Obtain the pointer instrument image data and perform data expansion and dial annotation in turn to obtain the first annotated data set:

[0088] Preferably, in this embodiment, pointer instrument images are captured and collected in a natural environment, and a total of 900 pointer instrument images are obtained as an initial data set.

[0089] Preferably, data expansion specifically includes:

[0090] Divide the pointer instrument image data into A group of different data subsets;

[0091] Randomly select a subset of the data in group A-1 and perform different image enhancement operations to obtain different expanded data subsets in group A-1;

[0092] The augmented data set is composed based on all augmented data subsets and data subsets that have not undergone image augmentation operations.

[0093] Preferably, in this embodiment, according to different scenarios existing on site, in order to further improve the generalization ability of the model, this embodiment adopts four forms of data enhancement:

[0094] The initial data set composed of pointer instrument image data is divided into five data subsets containing different image data. Four data subsets are randomly selected: 150 images in the first subset, 165 images in the second subset, 179 images in the third subset and 177 images in the fourth subset. Gaussian blur, brightness adjustment, rotation and scaling operations are performed on the first, second, third and fourth subsets respectively, and finally four different extended data subsets are obtained, which are combined with the fifth subset without image enhancement operation to form an extended data set.

[0095] Preferably, in this embodiment, labelimg is used to label the dial area of ​​the instrument data in the extended data set to obtain a first labeled data set.

[0096] Based on the YOLOv8n network structure, the backbone network, some convolution modules in the neck network, and the convolution modules in the head network are optimized and adjusted to obtain an improved YOLOv8n network model;

[0097] Preferably, the optimization and adjustment specifically include:

[0098] like Figure 2-Figure 4 As shown, the second convolution module, the third convolution module, the fourth convolution module and the fifth convolution module in the backbone network based on the YOLOv8n network structure are replaced by the first Double-Conv module Double-Conv1, the second Double-Conv module Double-Conv2, the third Double-Conv module Double-Conv3 and the fourth Double-Conv module Double-Conv4 respectively;

[0099] The sixth convolution module and the seventh convolution module in the neck network based on the YOLOv8n network structure are replaced by the fifth Double-Conv module Double-Conv5 and the sixth Double-Conv module Double-Conv6 respectively;

[0100] All standard convolution modules in all detection heads in the head network based on the YOLOv8n network structure are replaced by GhostConv convolution modules.

[0101] Preferably, with the improvement of the accuracy of the YOLOv8 model, the number of network layers has also increased, which brings challenges to the situation of limited computing resources, and it is particularly important to reduce the amount of calculation and the amount of parameters of the network model. In order to solve the problem of large amount of calculation of the YOLOv8 network model, the present invention adopts a Double-Conv module to replace some convolution modules of the backbone network and the neck network in the YOLOv8n network structure; the double convolution (Double-Conv) module is a new type of convolution filter, including M input channels, the convolution filter is divided into G groups, and 1×1 point-by-point convolution and 3×3 group convolution are integrated together, some convolution kernels perform 3×3 convolution operations and 1×1 convolution operations at the same time, and the remaining convolution kernels only perform 1×1 convolution operations, this operation is traversed downward in the channel in turn, and 3×3 convolution kernels are designed with parallel 1×1 convolution kernels, so as to retain the original information of the input feature map to help deeper convolution layers extract more effective features.

[0102] Preferably, Figure 5 As shown in the figure, all Double-Conv modules include G groups of convolution filters, the number of input channels is M, N is the total number of convolution filters, N / G is the number of each group, and the output of the previous layer is used as the input of the next layer;

[0103] The convolution filter includes two identical branches, each of which includes: a first convolution part and a second convolution part, the first convolution part includes an M / G layer and a first convolution layer, and the first convolution layer includes a 3×3 convolution layer and a first 1×1 convolution layer connected in sequence;

[0104] The second convolution part includes the MM / G layer and the second 1×1 convolution layer, and the first 1×1 convolution layer is located above the second 1×1 convolution layer;

[0105] From the first group of convolution filters to the G-th group of convolution filters, the position of the first convolution part is shifted downward from the top by M / G layers until the first convolution part of the G-th group of convolution filters is at the bottom.

[0106] Preferably, the overall structure of the first Double-Conv module Double-Conv1, the second Double-Conv module Double-Conv2, the third Double-Conv module Double-Conv3, the fourth Double-Conv module Double-Conv4, the fifth Double-Conv module Double-Conv5 and the sixth Double-Conv module Double-Conv6 are all the same, only the input channels are different. In this embodiment, the number of input channels M of the first Double-Conv module to the fourth Double-Conv module are 128, 256, 512 and 1024 respectively.

[0107] Preferably, the number of input channels M of the fifth Double-Conv module and the sixth Double-Conv module are 256 and 512 respectively.

[0108] like Figure 6 As shown, taking the Double-Conv module including 2 groups of convolution filters as an example, the input channel M is 128, the 3×3 convolution layer includes 128 / 2=64 layers, and the output channel number N is assumed to be 256, then there are 128 columns.

[0109] Preferably, Figure 7 As shown in the figure, the black part is the first convolution part, and the blue part is the second convolution part. In simple terms, the first input can be an image or other form of data. Then define the convolution kernel (filter), which is a smaller matrix used to extract features. Subsequently, the convolution kernel starts from the upper left corner of the input matrix and slides to the lower right corner according to the set step size. Each position is element-wise multiplied and summed to form the eigenvalues ​​in the output matrix.

[0110] Preferably, the Double-Conv module reduces the parameters of the backbone network model and the neck network model of the original model through the group convolution strategy, thereby making the optimized and adjusted model lighter.

[0111] Preferably, the Double-Conv module uses 1×1 convolution to retain the original information of the input feature map, and the 3×3 convolution filter of the deeper convolution layer can still learn from the original information of the input feature map. Compared with the model compression method, the Double-Conv module of the present invention is more practical and efficient and can be applied to all CNN architectures.

[0112] Preferably, some convolution modules of the backbone network and the neck network in the YOLOv8n network structure are replaced with Double-Conv modules, which inherits the feature extraction capability of the original convolution module and reduces excessive redundant information. This improvement makes the model lightweight and improves the detection speed and efficiency of the model.

[0113] Preferably, the computational effort of the Double-Conv module is FL DC Both are:

[0114]

[0115] Among them, Do represents the width and height dimensions of the output feature map, M represents the number of input feature map channels, and K 2 Represents the convolution kernel size, N represents the number of convolution filters, and G represents the number of convolution groups.

[0116] Preferably, Figure 8 As shown, the GhostConv convolution module includes: standard convolution layer, linear operation layer and splicing layer;

[0117] like Fig. 9 As shown, the input feature map input is input to the standard convolution layer Conv to obtain the first feature map;

[0118] The first feature map is input to the linear operation layer for linear operation to obtain multiple second feature maps Φ1, Φ2...Φ k ;

[0119] All second feature maps Φ1, Φ2, ...Φ k The input is concatenated by channel at the concatenation layer to obtain the fused feature map Output.

[0120] Preferably, the GhostConv convolution module is a convolution module in the GhostNet network, and the main idea is to generate more feature maps with less computational effort, and to generate more ghost feature maps through a series of linear transformations.

[0121] Preferably, the present invention replaces all standard convolution modules in all detection heads (Detect1, Detect2 and Detect3) with GhostConv convolution modules GhostConv.

[0122] Preferably, the GhostConv convolution module of the present invention can reduce the computational cost of general convolutional layers while maintaining the same recognition performance, and GhostNets can quickly reason about various tasks on mobile devices, with efficiency surpassing the most advanced efficient deep models.

[0123] Preferably, the GhostConv convolution module of the present invention generates more feature maps from existing feature maps by using linear transformation, which greatly reduces the required amount of calculation compared to relying on additional convolutional layers, maximizes the use of available computing and memory resources, and is suitable for embedded systems with limited resources; at the same time, due to its modular design, it can be seamlessly integrated into the existing CNN architecture, making it a general solution to improve efficiency.

[0124] Preferably, the present invention improves the YOLOv8n network model, and uses a lightweight convolution Double-Conv to replace some standard convolution modules in the backbone network and the neck network, thereby reducing the complexity of the model and improving the detection speed of the target object; at the same time, the Ghostconv convolution module is used to replace the standard convolution module in the detection head in the head network, while ensuring the model feature extraction capability, more feature maps are generated from the existing feature maps by using cheap linear transformations, greatly reducing the required calculation amount. Based on the above improvements, the computational complexity and parameter amount of the model are greatly reduced, and the accuracy and speed are improved to a certain extent.

[0125] Preferably, the improved YOLOv8n network model data processing process is:

[0126] The image to be processed is sequentially input into the convolution module Conv, Double-Conv1, C2f module, Double-Conv2 and C2f module for processing to obtain the first scale feature map FC1;

[0127] The first scale feature map FC1 is sequentially input into the Double-Conv3 and C2f modules for processing to obtain the second scale feature map FC2;

[0128] The second scale feature map FC2 is sequentially input into the Double-Conv4, C2f module and SPPF module for processing to obtain the third scale feature map FC3;

[0129] The first scale feature map FC1, the second scale feature map FC2, and the third scale feature map FC3 are all input into the improved neck network:

[0130] The third scale feature map FC3 is sequentially input into the UP module and the Concat module for processing and then fused with the second scale feature map FC2 to obtain the first fused feature map FR1;

[0131] The first fused feature map FR1 is input to the C2f module for processing to obtain the first processed feature map FZ1;

[0132] The first processed feature map FZ1 is sequentially input into the Up module and the Concat module for processing, and then fused with the first scale feature map FC1 to obtain the second fused feature map FR2;

[0133] The second fused feature map FR2 is input to the C2f module for processing to obtain the second processed feature map FZ2;

[0134] The second processed feature map FZ2 is sequentially input into the Double-Conv5 and Concat modules for processing and then fused with the first processed feature map FZ1 to obtain the third fused feature map FR3;

[0135] The third fused feature map FR3 is input to the C2f module for processing to obtain a third processed feature map FZ3;

[0136] The third processed feature map FZ3 is sequentially input into the Double-Conv6 and Concat modules for processing and then fused with the third scale feature map FC3 to obtain the fourth fused feature map FR4;

[0137] The fourth fused feature map FR4 is input to the C2f module for processing to obtain a fourth processed feature map FZ4;

[0138] The fourth processing feature map FZ4, the third processing feature map FZ3 and the second processing feature map FZ2 are correspondingly input to the first detection head Detect1, the second detection head Detect2 and the third detection head Detect3 in the head network for target detection, and the final bounding box and category information are output.

[0139] The improved YOLOv8n network model is trained based on the first labeled data set to obtain a dial area detection model.

[0140] Based on the first labeled data set, the dial area detection model is input to obtain the dial data set:

[0141] Preferably, the best model obtained based on training is used as the dial area detection model, the first labeled data set is detected to obtain the dial target area, and the dial target area is cropped to obtain the cropped dial data to form a dial data set.

[0142] Based on the dial dataset, secondary annotation is performed to obtain the second annotation dataset and train the dial area detection model to obtain the target box detection model:

[0143] Preferably, the secondary annotation includes:

[0144] Based on the image data of each dial in the dial dataset, the starting scale, end scale, center position and pointer position are marked, and the corresponding numbers are marked: C1, C2, C3 and C4.

[0145] Input the instrument image to be tested into the target frame detection model to obtain the target frame.

[0146] Get the pointer reading of the instrument under test based on the target frame:

[0147] Preferably, obtaining the pointer reading of the instrument to be tested based on the target frame specifically includes:

[0148] Based on the target frame, the starting scale coordinates, the end scale coordinates, the center position coordinates and the pointer position coordinates are obtained;

[0149] Get vector V1 based on the starting scale coordinates and the center position coordinates;

[0150] Get vector V2 based on the pointer position coordinates and the circle center position coordinates;

[0151] Get vector V3 based on the end point scale coordinates and the circle center position coordinates;

[0152] A first angle θ is calculated based on the vector V1 and the vector V2;

[0153] The second angle θ is calculated based on vector V1 and vector V3 max ;

[0154] Based on the first angle θ and the second angle θ max And the system range r to get the pointer reading of the instrument to be tested:

[0155]

[0156] Among them, α represents the pointer reading of the instrument to be tested.

[0157] Preferably, it also includes: using recall (R), precision (P) and mean average precision (mAP) as preset evaluation indicators;

[0158] The dial area detection model and the target box detection model are evaluated based on the evaluation indicators.

[0159] Preferably, the calculation formulas for recall rate P and precision rate R are as follows:

[0160]

[0161] Among them, T P Indicates the number of true positive samples, F N represents the number of positive samples that were incorrectly not detected, F P Indicates the number of negative samples that are incorrectly labeled as positive samples.

[0162] Preferably, the recall rate P measures how many of all samples detected as positive samples are truly positive samples. The higher the recall rate P, the more accurate the algorithm is in determining positive samples; the higher the precision rate R, the better the algorithm can detect the real target.

[0163] Preferably, the average precision mean formula is as follows:

[0164]

[0165] Among them, P Ai Represents the accuracy value of the i-th category, and N represents the total number of categories.

[0166] Example 3

[0167] The present invention improves the network model performance verification:

[0168] The hardware configuration to achieve the operation is shown in Table 1:

[0169] Table 1 Hardware configuration

[0170] The operating system Ubuntu 20.04 CPU I7-12700KF GPU RTX-3090 Memory 64G Language Python

[0171] The ablation experiment verifies the effect of the improved model of the present invention on improving the model performance. On the basis of the YOLOv8n network, the Double-Conv module is used to replace part of the Conv modules in the backbone network and the neck network, and the GhostConv convolution module is used to replace the Conv module in the detection head. Each improvement is accumulated in turn. The ablation experiment results of the dial detection dataset (the first labeled dataset) are shown in Table 2:

[0172] Table 2 Ablation experiment results of dial detection dataset

[0173]

[0174] From the ablation experiment results in Table 2, we can see that after using the improved Double-Conv lightweight module alone, the number of parameters decreased by 0.359×10 while the Precision and Recall did not change much. 6 , the computational workload decreased by 1.4GLOPS while the model accuracy only decreased by 0.5%.

[0175] After using the improved GhostConv lightweight module alone, the number of parameters decreased by 0.225×10 without much change in Precision and Recall. 6 , the computational workload decreased by 0.5GLOPS while the model accuracy only decreased by 0.2%.

[0176] By using two lightweight modules at the same time, the average detection accuracy is only reduced by 0.8% without much change in Precision and Recall, while the number of parameters is reduced by 0.584×10 6 The amount of computation dropped by 1.9 GLOPS, making the model very lightweight.

[0177] The ablation experiment results of the pointer detection dataset (the second annotated dataset) are shown in Table 3:

[0178] Table 3 Ablation experiment results of pointer detection dataset

[0179]

[0180] From the ablation experiment results in Table 2, we can see that after using the improved Double-Conv lightweight module alone, the Precision and Recall increased by 0.6% and 2.1% respectively, the average detection accuracy C1 increased by 1.2%, C2 increased by 0.3%, C3 increased by 0.2%, and C4 increased by 1.2%, and the number of parameters decreased by 0.358×10 6 , the computing workload decreased by 1.4GLOPS.

[0181] After using the improved GhostConv lightweight module alone, Precision decreased by 1.3%, Recall increased by 1.1%, the average detection accuracy C1 increased by 0.6%, C2 accuracy remained unchanged, C3 increased by 0.1%, C4 increased by 0.6%, and the number of parameters decreased by 0.225×10 6 , the computing workload decreased by 0.5GLOPS.

[0182] Using two lightweight modules at the same time for improvement, Precision is reduced by 0.2%, Recall remains unchanged, the average detection accuracy C1 is reduced by 1.4%, C2 accuracy remains unchanged, C3 increases by 0.1%, C4 increases by 0.8%, and the number of parameters is reduced by 0.225×10 6 , the amount of calculation decreased by 1.9GLOPS. It can be seen that, considering the processing effects of the two data sets, the YOLOv8n network model after the two lightweight modules used in the present invention are improved at the same time becomes lighter in both tests, the detection speed is increased, and various indicators are improved to varying degrees, which can meet the detection requirements of the instrument recognition project.

[0183] Comparative experiments with other existing models further verify the effectiveness of the improvements to YOLOv8n in instrument detection:

[0184] This embodiment selects the currently popular target detection algorithms YOLOv5s, YOLOv5m, YOLOv7, YOLOv7-tiny and YOLOv8s for comparison. The ablation experiment results of the dial detection dataset (the first labeled dataset) are shown in Table 4:

[0185] Table 4 Comparative experimental results of dial detection dataset

[0186]

[0187] According to the test results in Table 4, the lightweight improved YOLOv8n model is the lightest and has the highest detection accuracy compared with other models. The average detection accuracy of the improved YOLOv8n model reached 98.4%, which was 0.1%, 0.4%, 0.3%, 1%, and 0.2% higher than that of YOLOv5s, YOLOv5m, YOLOv7, YOLOv7-tiny, and YOLOv8s models, respectively. The amount of calculation decreased by 9.1 GLOPS, 41.2 GLOPS, 98.8 GLOPS, 6.9 GLOPS, and 17.8 GLOPS, respectively, and the number of parameters decreased by 4.586×10 6 , 18.444×10 6 ,34.77×10 6 ,3.588×10 6 ,7.671×10 6 It can be seen that not only the average detection accuracy of the model has been improved, but the amount of calculation and the amount of parameters have also been reduced to varying degrees, making the model lightweight, accurate and efficient.

[0188] The comparative experimental results of the pointer detection dataset (the second annotated dataset) are shown in Table 5:

[0189] Table 5. Comparative experimental results of pointer detection dataset

[0190]

[0191]

[0192] According to the experimental results in Table 5, the average detection accuracy of the improved YOLOv8n is the highest in all four categories in all comparative experiments: C1 reaches 99%, C2 reaches 98%, C3 reaches 99.2%, and C4 reaches 98.1%. The computational complexity is reduced by 9.1 GLOPS, 41.2 GLOPS, 98.8 GLOPS, 6.5 GLOPS, and 19.1 GLOPS compared with YOLOv5s, YOLOv5m, YOLOv7, YOLOv7-tiny, and YOLOv8s, respectively, and the number of parameters is reduced by 4.368×10 6 ,18.212×106 ,34.56×10 6 ,3.362×10 6 and 7.438×10 6 It can be seen that the improved YOLOv8n network model using the present invention has obvious improvements in detection accuracy and speed on the second data set.

[0193] like Fig.10 As shown in the figure, the detection comparison effect diagram of different models shows that the detection accuracy and precision of the improved lightweight model are still very high, and the difference is not very big compared with other models with large computational complexity, and there will be no missed detection or false detection. In summary, the improved model can better complete the task of instrument identification, and the computational complexity and parameter quantity are low, which is more suitable for deployment on devices with limited computing resources.

[0194] Example 4

[0195] like Fig.11 As shown, a pointer instrument detection, recognition and reading system based on improved YOLOv8n includes: a first labeling module, a model improvement module, a first training module, a second training module and a result output module;

[0196] A first annotation module is used to obtain pointer instrument image data and perform data expansion and dial annotation in sequence to obtain a first annotation data set;

[0197] The model improvement module is used to optimize and adjust the backbone network and some convolution modules in the neck network based on the YOLOv8n network structure, as well as the convolution modules in the head network, to obtain an improved YOLOv8n network model;

[0198] A first training module is used to train an improved YOLOv8n network model based on a first labeled data set to obtain a dial area detection model;

[0199] The second training module is used to input the first labeled data set into the dial area detection model to obtain a dial data set; perform secondary labeling based on the dial data set to obtain a second labeled data set and train the dial area detection model to obtain a target frame detection model;

[0200] The result output module is used to input the image of the instrument to be tested into the target frame detection model to obtain the target frame; and obtain the pointer reading of the instrument to be tested based on the target frame.

[0201] Preferably, the method process implemented by the modules in this embodiment corresponds one to one with the above method, and will not be described one by one here.

[0202] Example 5

[0203] Based on the same inventive concept, the present invention also provides a computer device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0204] Memory, used to store computer programs;

[0205] The processor, when used to execute the program stored in the memory, can implement a pointer instrument detection and identification reading method based on improved YOLOv8n as in Example 1 or 2.

[0206] The electronic device may include: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus. The processor may call the logic instructions in the memory to execute a pointer instrument detection and identification reading method based on improved YOLOv8n in embodiment 1 or 2.

[0207] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0208] Through the above technical solutions, it can be known that the present invention discloses a method and system for detecting and identifying readings of pointer-type instruments based on improved YOLOv8n, using the improved YOLOv8n model to locate the pointer dial, and then using the identified dial to identify the starting scale, center of circle, pointer, and end scale using the improved YOLOv8n model to greatly improve the accuracy. It has the following beneficial effects:

[0209] 1. The present invention improves the YOLOv8n network model and uses a lightweight convolution Double-Conv to replace some standard convolution modules in the backbone network and the neck network, thereby reducing the complexity of the model and improving the detection speed of the target object; at the same time, the Ghostconv convolution module is used to replace the standard convolution module in the detection head in the head network, while ensuring the model feature extraction capability, more feature maps are generated from the existing feature maps by using cheap linear transformations, greatly reducing the required calculation amount. Based on the above improvements, the computational complexity and parameter amount of the model are greatly reduced, and the accuracy and speed are improved to a certain extent.

[0210] 2. The data augmentation technology is used to increase the diversity of training samples, so that the model can better learn the performance of pointer instruments under different lighting conditions, angle changes and other factors, thereby enhancing the robustness and generalization ability of the model in actual application scenarios.

[0211] 3. The two-stage labeling strategy (first generating the first labeled data set for preliminary training to obtain the dial area detection model; then producing a more accurate second labeled data set based on the output results of this model for further training) can effectively reduce the false detection rate and has a better capture effect on the pointer position of small targets or occluded conditions.

[0212] 4. The present invention is not only suitable for single image analysis, but also can be easily extended to real-time monitoring tasks of continuous frames in video streams, providing a powerful support tool for equipment status monitoring in industrial production environments.

[0213] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0214] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A pointer instrument detection and recognition reading method based on improved YOLOv8n, characterized in that: include: Acquire pointer instrument image data and perform data expansion and dial annotation in sequence to obtain a first annotated data set; Based on the YOLOv8n network structure, the backbone network, some convolution modules in the neck network, and the convolution modules in the head network are optimized and adjusted to obtain an improved YOLOv8n network model; The improved YOLOv8n network model is trained based on the first labeled data set to obtain a dial area detection model; Based on the first labeled data set, input into the dial area detection model to obtain a dial data set; Perform secondary annotation based on the dial data set to obtain a second annotated data set and train the dial area detection model to obtain a target frame detection model; Inputting the image of the instrument to be tested into the target frame detection model to obtain the target frame; The pointer reading of the instrument to be tested is obtained based on the target frame.

2. According to a method for detecting and identifying pointer-type instruments based on improved YOLOv8n according to claim 1, it is characterized in that: The data expansion specifically includes: Divide the pointer instrument image data into A group of different data subsets; Randomly select the data subsets in group A-1 and perform different image enhancement operations to obtain different expanded data subsets in group A-1; An expanded data set is formed based on all the expanded data subsets and the data subset that has not been subjected to the image enhancement operation.

3. According to a method for detecting and identifying readings of a pointer instrument based on improved YOLOv8n according to claim 1, it is characterized in that: The optimization adjustment specifically includes: The second convolution module, the third convolution module, the fourth convolution module and the fifth convolution module in the backbone network based on the YOLOv8n network structure are replaced by the first Double-Conv module, the second Double-Conv module, the third Double-Conv module and the fourth Double-Conv module respectively; The sixth convolution module and the seventh convolution module in the neck network based on the YOLOv8n network structure are replaced by the fifth Double-Conv module and the sixth Double-Conv module respectively; All standard convolution modules in all detection heads in the head network based on the YOLOv8n network structure are replaced with GhostConv convolution modules.

4. According to claim 3, a pointer instrument detection and recognition reading method based on improved YOLOv8n is characterized in that: All Double-Conv modules include G groups of convolutional filters and the number of input channels is M; The convolution filter includes two identical branches, each of which includes: a first convolution part and a second convolution part, the first convolution part includes an M / G layer and a first convolution layer, and the first convolution layer includes a 3×3 convolution layer and a first 1×1 convolution layer connected in sequence; The second convolution part includes a second 1×1 convolution layer of the MM / G layer, and the first 1×1 convolution layer is located above the second 1×1 convolution layer; From the first group of convolution filters to the Gth group of convolution filters, the position of the first convolution part is shifted downward from the top by M / G layers in sequence until the first convolution part of the Gth group of convolution filters is located at the bottom.

5. According to claim 4, a pointer instrument detection and recognition reading method based on improved YOLOv8n is characterized in that: The computational cost of the Double-Conv module is FL DC for: Among them, Do represents the width and height dimensions of the output feature map, M represents the number of input feature map channels, and K 2 Represents the convolution kernel size, N represents the number of convolution filters, and G represents the number of convolution groups.

6. According to a method for detecting and identifying readings of a pointer instrument based on improved YOLOv8n according to claim 3, it is characterized in that: The GhostConv convolution module includes: a standard convolution layer, a linear operation layer and a splicing layer; Input the feature map to the standard convolution layer to obtain a first feature map; The first feature map is input into a linear operation layer for linear operation to obtain a plurality of second feature maps; All the second feature maps are input to the splicing layer and spliced ​​by channel to obtain a fused feature map.

7. According to a method for detecting and identifying readings of a pointer instrument based on improved YOLOv8n according to claim 1, it is characterized in that: The secondary annotation includes: Based on each dial image data in the dial data set, the starting scale, the end scale, the center position and the pointer position are marked, and are marked with numbers: C1, C2, C3 and C4.

8. A method for detecting and identifying readings of a pointer instrument based on improved YOLOv8n according to claim 7, characterized in that: Obtaining a pointer reading of the instrument to be tested based on the target frame specifically includes: Based on the target frame, the starting scale coordinates, the end scale coordinates, the center position coordinates and the pointer position coordinates are obtained; Obtaining a vector V1 based on the starting scale coordinates and the circle center position coordinates; Obtaining a vector V2 based on the pointer position coordinates and the circle center position coordinates; Obtaining vector V3 based on the end point scale coordinates and the circle center position coordinates; A first angle θ is calculated based on the vector V1 and the vector V2; The second angle θ is calculated based on the vector V1 and the vector V3 max ; Based on the first angle θ and the second angle θ max And the system range r to obtain the pointer reading of the instrument under test: Among them, α represents the pointer reading of the instrument to be tested.

9. According to a method for detecting and identifying readings of a pointer instrument based on improved YOLOv8n according to claim 1, it is characterized in that: Also includes: Based on recall rate, precision rate and average precision mean as preset evaluation indicators; The dial area detection model and the target frame detection model are evaluated respectively based on the evaluation index.

10. A pointer instrument detection, recognition and reading system based on improved YOLOv8n, applied to a pointer instrument detection, recognition and reading method based on improved YOLOv8n as claimed in any one of claims 1 to 9, characterized in that: include: A first labeling module, a model improvement module, a first training module, a second training module and a result output module; The first annotation module is used to obtain the pointer instrument image data and perform data expansion and dial annotation in sequence to obtain a first annotation data set; The model improvement module is used to optimize and adjust the backbone network and some convolution modules in the neck network and the convolution modules in the head network based on the YOLOv8n network structure to obtain an improved YOLOv8n network model; The first training module is used to train the improved YOLOv8n network model based on the first labeled data set to obtain a dial area detection model; The second training module is used to input the first labeled data set into a dial area detection model to obtain a dial data set; Perform secondary annotation based on the dial data set to obtain a second annotated data set and train the dial area detection model to obtain a target frame detection model; The result output module is used to input the image of the instrument to be tested into the target frame detection model to obtain the target frame; The pointer reading of the instrument to be tested is obtained based on the target frame.

Images