Instrument intelligent reading method based on deep learning
Through the deep learning-based instrument intelligent reading method, using YOLOv5 model and template matching technologies, the robustness of instrument reading recognition in the existing technology in the complex noise environment is solved, and high-precision and reliable instrument reading recognition is achieved, which is suitable for smart substations.
Patent Information
- Application Number
- CN202510003609.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-02
- Publication Date
- 2025-04-11
AI Technical Summary
The existing automatic reading recognition method of pointer instruments lacks robustness when facing complex noise, making it difficult to achieve accurate and reliable instrument reading recognition.
The instrument is used based on deep learning intelligent reading method, and the instrument is detected using the YOLOv5 model, and the instrument reading is accurately recognized through template matching, binarization, circle detection and pointer detection.
It improves the accuracy and robustness of instrument reading detection, realizes the integration of instrument detection and reading, suitable for deployment in low-cost edge devices, and is suitable for smart substation application scenarios.
Smart Images
Figure CN119810810A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present disclosure relate to the field of image processing technology, and in particular to an intelligent instrument reading method based on deep learning. Background Art
[0002] Substations are an important part of modern power systems to achieve voltage conversion, load regulation, power protection and control, and other functions. There are usually a large number of pointer instrument facilities inside substations, which are used to detect and evaluate the working conditions and health status of electrical equipment in real time. Therefore, how to accurately and reliably complete the reading of these instrument parameters is crucial to understanding the operation of the entire substation and thus ensuring the safety and stability of the substation.
[0003] In the reading of pointer instruments, the traditional reading method relies on manual work. However, the reading efficiency and accuracy of manual work largely depend on the work experience of the meter reader, and have high requirements on the professional ability of the practitioners. In addition, while manual meter reading consumes manpower, the reading accuracy and efficiency will often decrease with the extension of working hours, and it is easily disturbed by harsh environments such as wind, sand, rain, and snow. Therefore, studying an automatic, accurate, fast, and robust pointer instrument reading recognition method is of great significance to help achieve efficient management of substation automation.
[0004] In the existing research on automatic reading recognition technology of pointer instruments, the representative methods are detection methods such as using two-dimensional gamma function, convolutional neural network and region growth to locate the dial area and center. Among them, the two-dimensional gamma function can achieve effective brightness correction for images with uneven illumination, but in actual environments, low-illuminance images are often accompanied by a lack of clarity, and this method has certain limitations; and although the method of instrument reading recognition through convolutional neural network has derived a pointer automatic reading method combined with mask region (Mask R-CNN) key point detection, this method uses the captured image key point features for fitting and has a strong performance in the recognition accuracy of readings, but for practical applications, the instrument key point features in the image are often disturbed by illumination, so this method has low reliability in actual environments; Different from the above methods, the method based on the principle of region growth to locate the dial area and center uses the central projection method and boundary detection method to extract the pointer reading in the implementation of the algorithm, and has a good performance on the experimental data set, but because this method does not take into account the changes in the perspective and deformation of the dial in the actual environment in the production of the data set, this method lacks robustness when facing the contour noise of the dial.
[0005] In summary, the existing automatic identification method of pointer instrument readings still lacks the robustness to meet work expectations when facing complex noise. For the research on substation pointer instrument readings, it is still urgent to invent and design an accurate and reliable automatic identification algorithm and system for instrument readings to help accurately locate the instrument equipment and thus achieve accurate identification of the instrument readings.
[0006] Therefore, it is necessary to improve one or more problems existing in the above-mentioned related technical solutions.
[0007] It should be noted that this section is intended to provide background or context for the technical solutions of the present disclosure stated in the claims. The description herein is not admitted to be prior art by virtue of being included in this section. Summary of the invention
[0008] The purpose of the embodiments of the present disclosure is to provide a deep learning-based intelligent meter reading method, thereby overcoming one or more problems caused by the limitations and defects of related technologies at least to a certain extent.
[0009] According to an embodiment of the present disclosure, a method for intelligent meter reading based on deep learning is provided, the method comprising: Obtain instrument images containing pointer-type instrument targets in a power environment, annotate the instrument images to obtain label files, and divide the label files into a training set and a validation set according to a preset ratio; Construct a YOLOv5 model, and use the training set and test set to train and test the YOLOv5 model to obtain a trained YOLOv5 model; the YOLOv5 model includes a backbone network, a neck network, and a detection head; Use the YOLOv5 model to detect the image of the instrument to be detected to obtain the instrument detection result map; Performing template matching on the detection result image to obtain a template matching result image; Binarizing the template matching result image to obtain a binarized result image; Perform circle detection on the binarization result image to obtain a circle detection result image; Perform pointer detection on the circle detection result map to obtain a pointer detection map.
[0010] Further, the steps of obtaining an instrument image containing a pointer instrument target in a power environment, annotating the instrument image to obtain a label file, and dividing the label file into a training set and a validation set according to a preset ratio include: Obtain an instrument image containing a pointer instrument target in the power environment, and use the labelimg tool to annotate the image data to obtain the corresponding instrument file in XML format; the instrument file includes the folder name where the instrument image is saved, the image name corresponding to the label, the absolute path where the image is saved, the size of the label image, and the information of the channel and the annotated object; Convert the instrument file in XML format to a label file in YOLO format; The label file is divided into a training set and a validation set according to a preset ratio.
[0011] Furthermore, the backbone network includes: G-Ghost layer 1, SFB layer 1, G-Ghost layer 2, SFB layer 2, G-Ghost layer 3, SFB layer 3 and UpSample layer connected in series; wherein, The structures of SFB layer 1, SFB layer 2 and SFB layer 3 are the same, and all include: Channel split, concat module and Channel Shuffle connected in sequence, Channel split and concat are connected through a parallel first branch and a second branch, and the second branch includes a serial 1×1 depth convolution, a 3×3 convolution and a 1×1 depth convolution; The structures of G-Ghost layer 1, G-Ghost layer 2 and G-Ghost layer 3 are the same, and all include: 3×3 convolution, several parallel 1×1 convolutions and concat modules. The 3×3 convolutions are respectively connected to several parallel 1×1 convolutions, and several parallel 1×1 convolutions are all connected to the concat module.
[0012] Furthermore, the process of SFB layer 1 processing input data is as follows: Input data of size C×H×W is input into Channel split. Channel split inputs the input data into the first branch and the second branch with the same number of input channels, respectively, to obtain the first feature map of (C / 2)×H×W and the second feature map of (C / 2)×H×W, respectively; where C is the number of channels, H and W are the height and width of the feature map, respectively; The concat module fuses the first feature map and the second feature map, and uses Channel Shuffle to perform channel-level fusion operations to obtain the initial fusion features; Channel Shuffle performs channel-level fusion operations on the initial fusion features; The process of SFB layer 1 processing input data is as follows: Input the feature tensor of size Cin×H×W into the 3×3 convolution to generate a core feature tensor of size Cont×H×W; where Cin is the input channel; A set of lightweight linear transformations are performed on the core feature tensor using 1×1 convolution to generate 4 redundant feature tensors of size Cont×H×W; The core feature tensor and the generated redundant features are concatenated in the channel dimension to form the final output feature of size C×H×W.
[0013] Furthermore, the step of performing template matching on the detection result image to obtain a template matching result image includes: Grayscale the detection result image and the template image; The AKAZE algorithm is used to perform feature point detection on the grayscale detection result image and the template image to detect the first feature point set. And the second feature point set ; Traverse the first feature point set All feature points in the second feature point set are found The corresponding matching points in the first mapping relationship set are obtained ; Traverse the second feature point set All feature points in the second feature point set are found The corresponding matching points in the second mapping relationship set are obtained ; Obtaining a correct matching pair according to the first mapping relationship set and the second mapping relationship set; The random sampling consistency algorithm is used to screen out the correct matching pairs with large matching errors in the image and obtain the homography matrix; The obtained homography matrix is used to perform perspective transformation on the detection result image to obtain the template matching result image.
[0014] Furthermore, the step of binarizing the template matching result image to obtain a binarized result image includes: The template matching result image is converted into a grayscale image, and the grayscale values of the grayscale image are counted to obtain a grayscale histogram; Based on the grayscale histogram, K-means clustering is performed using the K-means algorithm to update the centroid of the grayscale histogram, calculate the average grayscale value of each category, and obtain a new centroid; Iteratively update the centroid and redistribute the grayscale values according to the new centroid until convergence to obtain two classes; The average value of the two classes is used to determine the binarization threshold. According to the binarization threshold, the original grayscale image is binarized to generate a binarization result image.
[0015] Furthermore, the step of performing circle detection on the binarization result image to obtain a circle detection result image includes: Based on the binarization result graph, the least squares method is used to fit the circle where the dial scale lines are located. The optimal function matching of a set of data is determined by minimizing the sum of square errors to determine the fitting circle parameters.
[0016] According to the fitting circle parameters, the circle detection result graph is obtained.
[0017] Furthermore, the step of performing pointer detection on the circle detection result map to obtain a pointer detection map includes: The pixel points of the circle detection result map are summed vertically, and the horizontal coordinate of the straight line where the pointer is located is the column with the smallest summed value. Combined with the pointer angle and range, the pointer detection map and instrument readings are obtained.
[0018] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects: In the embodiments of the present disclosure, through the above-mentioned deep learning-based intelligent instrument reading method, on the one hand, first, the YOLOv5 model is used to detect the instrument in the image, and the image of the dial to be read is registered based on the template matching algorithm; then, the image is binarized to extract the pointer area, and the position of the pointer line is obtained by accumulating and comparing pixel values; finally, the distance method of pixel accumulation is used to calculate the reading, which improves the detection accuracy of the instrument reading and realizes the integrated realization of instrument detection and reading, meeting the requirements of deployment on low-cost edge devices. On the other hand, this method has obvious improvements in real-time performance and intelligence level, and can realize intelligent reading of instruments, which is more suitable for smart substation application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.
[0020] Figure 1 A step diagram showing a method for intelligent meter reading based on deep learning in an exemplary embodiment of the present disclosure; Figure 2 The instrument detection result in the exemplary embodiment of the present disclosure is shown; Figure 3 The template matching result in the exemplary embodiment of the present disclosure is shown; Figure 4 The binarization result in the exemplary embodiment of the present disclosure is shown; Figure 5 shows a circle detection result in an exemplary embodiment of the present disclosure; Figure 6 A pointer detection diagram in an exemplary embodiment of the present disclosure is shown; Figure 7 A specific flow chart of a meter intelligent reading method based on deep learning in an exemplary embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0021] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
[0022] In addition, the accompanying drawings are only schematic illustrations of the embodiments of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated descriptions will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities.
[0023] This example implementation provides a meter intelligent reading method based on deep learning. Figure 1 As shown in , the deep learning-based intelligent meter reading method may include: steps S101 to S107.
[0024] Step S101: obtaining an instrument picture containing a pointer-type instrument target in an electric power environment, annotating the instrument picture to obtain a label file, and dividing the label file into a training set and a verification set according to a preset ratio; Step S102: constructing a YOLOv5 model, and respectively training and testing the YOLOv5 model using a training set and a test set to obtain a trained YOLOv5 model; wherein the YOLOv5 model includes a backbone network, a neck network, and a detection head; Step S103: Use the trained YOLOv5 model to detect the image of the instrument to be detected to obtain a detection result image of the instrument; Step S104: performing template matching on the detection result image to obtain a template matching result image; Step S105: binarizing the template matching result image to obtain a binarized result image; Step S106: performing circle detection on the binarization result image to obtain a circle detection result image; Step S107: accumulating pixels of the circle detection result image to obtain a pointer detection image.
[0025] Through the above-mentioned intelligent instrument reading method based on deep learning, on the one hand, first, the YOLOv5 model is used to detect the instrument in the image, and the image of the dial to be read is registered based on the template matching algorithm; then, the image is binarized to extract the pointer area, and the position of the pointer line is obtained by accumulating and comparing pixel values; finally, the distance method of pixel accumulation is used to calculate the reading, which improves the detection accuracy of the instrument reading and realizes the integration of instrument detection and reading, meeting the requirements of deployment on low-cost edge devices. On the other hand, this method has obvious improvements in real-time performance and intelligence level, which can realize intelligent reading of instruments and is more suitable for smart substation application scenarios.
[0026] Next, we will refer to Figures 1 to 7 Each step of the above-mentioned deep learning-based intelligent meter reading method in this example implementation is described in more detail.
[0027] In step S101, an instrument picture containing a pointer-type instrument target in an electric power environment is obtained, the instrument picture is annotated to obtain a label file, and the label file is divided into a training set and a verification set according to a preset ratio.
[0028] In one embodiment, pictures containing pointer-type instrument targets in an electric power environment are taken, each taken picture is labeled to obtain a label file, and the prepared data set is divided into a training set and a verification set.
[0029] More than 300 images containing instruments were selected through field collection and online collection. The labelimg tool was used to annotate the images and obtain the corresponding xml format file. The information stored in the file includes the name of the folder where the image is saved, the image name corresponding to the label, the absolute path where the image is saved, the size of the labeled image, the channel and the information of the annotated object, that is, which category the target object belongs to. Afterwards, since the model training in this article is based on the YOLOV5 network structure, the xml format file of the dataset is converted into a YOLO format label.
[0030] In step S102 and step S103, the backbone network includes: G-Ghost layer 1, SFB layer 1, G-Ghost layer 2, SFB layer 2, G-Ghost layer 3, SFB layer 3 and UpSample layer connected in series; wherein, SFB layer 1, SFB layer 2 and SFB layer 3 have the same structure and all include: Channel split, concat module and Channel Shuffle connected in sequence, Channel split and concat are connected through a parallel first branch and a second branch, and the second branch includes a serial 1×1 depth convolution, a 3×3 convolution and a 1×1 depth convolution.
[0031] The structures of G-Ghost layer 1, G-Ghost layer 2 and G-Ghost layer 3 are the same, and all include: 3×3 convolution, several parallel 1×1 convolutions and concat modules. The 3×3 convolutions are respectively connected to several parallel 1×1 convolutions, and several parallel 1×1 convolutions are all connected to the concat module.
[0032] The process of SFB layer 1 processing input data is as follows: Input data of size C×H×W is input into Channel split. Channel split inputs the input data into the first branch and the second branch with the same number of input channels, respectively, to obtain the first feature map of (C / 2)×H×W and the second feature map of (C / 2)×H×W, respectively; where C is the number of channels, H and W are the height and width of the feature map, respectively; the concat module fuses the first feature map and the second feature map, and uses Channel Shuffle to perform channel-level fusion operations on them to obtain the initial fused features; Channel Shuffle performs channel-level fusion operations on the initial fused features.
[0033] The process of SFB layer 1 processing input data is as follows: The feature tensor of size Cin×H×W is input into a 3×3 convolution to generate a core feature tensor of size Cont×H×W; where Cin is the input channel; a set of lightweight linear transformations are performed on the core feature tensor using 1×1 convolution to generate 4 redundant feature tensors of size Cont×H×W; the core feature tensor and the generated redundant features are concatenated in the channel dimension to form the final output feature of size C×H×W.
[0034] In one embodiment, the conventional YOLOV5 model backbone is first replaced with ShuffleNet v2 (SFB), and the backbone network introduces Channel split. The structure divides the input data of size C×H×W into two branches, where C is the number of channels, H and W are the height and width of the feature map respectively; the number of input channels of the two branches is equal, and both are half of that before using Channelsplit. One branch maintains the same mapping output and its output size is (C / 2)×H×W, and the other branch undergoes three convolutions with the same number of input and output channels, namely 1×1 depth convolution, 3×3 convolution, and 1×1 depth convolution, and the final output size is (C / 2)×H×W. Finally, the structure abandons the 1×1 grouped convolution to reduce the memory access caused by too many groups. When splicing the two branches, the element-by-element addition method is no longer used, but the channel-level fusion operation is performed through channel splicing, avoiding element-level operations. The idea of G-Ghost is introduced into the C3 module, and a lightweight module GG-C3 is constructed, which can replace the C3 module in YOLOv5, reduce redundant parameters, and maintain the ability to extract features while achieving compression of model parameters and computational complexity. Specifically, the input feature tensor of size Cin×H×W is firstly input, where Cin is the input channel, and H and W are the height and width of the feature map, and a core feature tensor of size Cont×H×W is generated through 3×3 convolution. Then, a set of lightweight linear transformations are performed on the core features using 1×1 convolution to generate 4 redundant feature tensors of size Cont×H×W. Finally, the core features and the generated redundant features are concatenated in the channel dimension to form the final output features of size C×H×W. Figure 2 This is the instrument test result.
[0035] In step S104, template matching is performed on the detection result image to obtain a template matching result image.
[0036] In one embodiment, each image to be read and the template image acquired in advance are grayed. The AKAZE algorithm is used to detect feature points, and a feature matching pair is obtained by bidirectional matching. The bidirectional matching strategy uses the two images as reference images, calculates the similarity of feature descriptors by Hamming distance, and finds the best matching pair of feature points respectively. Only when the feature descriptors of the two images are the same in both matches, is it considered that the correct matching pair is found. For the calculation of the Hamming distance, it is assumed that and is the set of feature points of the two images to be matched, where and The feature point is a binary feature descriptor, and the Hamming distance The calculation formula is as follows:
[0037] in, It is an XOR operation.
[0038] The detailed steps of the two-way matching strategy are as follows: The feature points of the two images are detected using the feature point detection algorithm, and their set is and ; Traversal All feature points in The corresponding matching points in the Again Do the same traversal operation to get the mapping relationship set ; If there are identical elements mapped in g and f, then the set of elements is a correct matching pair; Then, the Random Sample Consensus (RANSAC) algorithm is used to filter out feature points with large matching errors in the image and obtain the homography matrix. Finally, the obtained matrix is used to perform perspective transformation on the image to be read to achieve image registration. Figure 3 is the template matching result.
[0039] In step S105, the template matching result image is binarized to obtain a binarized result image.
[0040] In one embodiment, first, the image is converted into a grayscale image. Each pixel value of the grayscale image can be regarded as a point in the data set, and its value range is usually between 0 and 255. By counting the frequency of these grayscale values, the grayscale histogram of the image can be obtained, which is the input data of the K-means algorithm. Select the initial centroid. The K-means algorithm needs to initialize two centroids. The usual practice is to randomly select two different initial centroids from the grayscale value range. In order to speed up the convergence rate, the initial centroid can also be selected as certain extreme points (e.g., maximum and minimum values) of the image grayscale histogram. Perform K-means clustering and assign each grayscale value in the image to the centroid closest to it. Since we have selected two centroids, the grayscale values will be divided into two categories, which represent the foreground and the background respectively. Update the centroid, calculate the average value of the grayscale value of each category, and obtain a new centroid. Then, repeat the previous step and reallocate the grayscale values according to the new centroid. This process is iterated continuously until the change in the centroid is less than the preset threshold or the maximum number of iterations is reached. Determine the binarization threshold. After K-means clustering, we get two classes. In order to binarize the image, you need to determine a threshold, and classify the pixels with grayscale values below the threshold as background, and the pixels above the threshold as foreground. Usually, the average of the two centroids can be used as this threshold. Image binarization. Finally, according to the determined threshold, the original grayscale image is binarized to generate a binary image. The pixel values in the binary image are only 0 and 255 (or 1 and 0), corresponding to the background and foreground respectively. The following figure shows the result of k-means binarization. Figure 4 It is the K-means binarization result.
[0041] In step S106, circle detection is performed on the binarization result image to obtain a circle detection result image.
[0042] In one embodiment, the least square method is used to fit the circle where the dial scale lines are located. The least square method is to determine the optimal function matching of a set of data based on minimizing the sum of square errors. The specific steps of fitting the circle are as follows: Assume that the equation of the fitting circle is the formula
[0043] The center of the circle (a / 2, b / 2) can be obtained from the above formula, and the radius r is shown in the following formula:
[0044] Assume (x, y) is the coordinate of the extracted key sample point, i=1,2…N, N is the number of extracted key points, then the Euclidean distance between the point (x, y) in the sample set and the fitting circle is as shown below:
[0045] The cumulative sum calculated by the above formula for all points in the sample set is as follows:
[0046] Find the solution that makes The smallest a, b and d are the final solution, and the fitting circle parameters can be determined. Figure 5 is the circle detection result.
[0047] In step S107, the pointer position is determined by pixel accumulation and the reading is calculated. Since the pointer area is black and the pixel value is 0 after binarization, when the image pixel values are summed vertically, the sum of the pixel values of the straight line where the pointer tip is located in the image is the smallest. According to the characteristics obtained by the above analysis, the vertical sum of the obtained binary image pixel points is obtained, and the horizontal coordinate of the straight line where the pointer is located is the column with the smallest summed value. The meter reading is calculated based on the pointer angle and range. Figure 6 This is the pointer detection diagram.
[0048] like Figure 7 As shown in FIG. 1 , a specific flow chart of the intelligent instrument reading method based on deep learning is shown. If the trained YOLOv5 model detects an instrument, subsequent processing is performed. If no instrument is detected, the process ends and outputs that no instrument is detected.
[0049] In a specific embodiment, to verify the effectiveness of the method, a sample test is performed on the test set in the data set prepared in step 1. The test results are shown in Table 1. It can be seen that the average relative error of the algorithm detection is 2.11%, and the average running time of the algorithm is 0.347s. In terms of accuracy and speed, the requirements of real-time and accuracy of actual industrial deployment are met.
[0050] Table 1 Test results
[0051] Through the above-mentioned intelligent instrument reading method based on deep learning, on the one hand, first, the YOLOv5 model is used to detect the instrument in the image, and the image of the dial to be read is registered based on the template matching algorithm; then, the image is binarized to extract the pointer area, and the position of the pointer line is obtained by accumulating and comparing pixel values; finally, the distance method of pixel accumulation is used to calculate the reading, which improves the detection accuracy of the instrument reading and realizes the integration of instrument detection and reading, meeting the requirements of deployment on low-cost edge devices. On the other hand, this method has obvious improvements in real-time performance and intelligence level, which can realize intelligent reading of instruments and is more suitable for smart substation application scenarios.
[0052] It should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", etc. in the above description indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the embodiments of the present disclosure and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the embodiments of the present disclosure.
[0053] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.
[0054] In the embodiments of the present disclosure, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present disclosure can be understood according to specific circumstances.
[0055] In the embodiments of the present disclosure, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may include the first and second features being in direct contact, or may include the first and second features not being in direct contact but being in contact through another feature between them. Moreover, a first feature being "above", "above" and "above" a second feature includes the first feature being directly above and obliquely above the second feature, or simply indicates that the first feature is higher in level than the second feature. A first feature being "below", "below" and "below" a second feature includes the first feature being directly below and obliquely below the second feature, or simply indicates that the first feature is lower in level than the second feature.
[0056] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification.
[0057] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.
Claims
1. A deep learning-based intelligent instrument reading method, characterized in that: The method includes: Obtain instrument images containing pointer-type instrument targets in a power environment, annotate the instrument images to obtain label files, and divide the label files into a training set and a validation set according to a preset ratio; Construct a YOLOv5 model, and use the training set and test set to train and test the YOLOv5 model to obtain a trained YOLOv5 model; the YOLOv5 model includes a backbone network, a neck network, and a detection head; Use the trained YOLOv5 model to detect the image of the instrument to be detected to obtain the instrument detection result image; Performing template matching on the detection result image to obtain a template matching result image; Binarizing the template matching result image to obtain a binarized result image; Perform circle detection on the binarization result image to obtain a circle detection result image; The circle detection result map is pixel-accumulated to obtain the pointer detection map.
2. The instrument intelligent reading method based on deep learning according to claim 1 is characterized in that: The steps of obtaining an instrument image containing a pointer instrument target in a power environment, annotating the instrument image to obtain a label file, and dividing the label file into a training set and a validation set according to a preset ratio include: Obtain an instrument image containing a pointer instrument target in the power environment, and use the labelimg tool to annotate the image data to obtain the corresponding instrument file in XML format; the instrument file includes the folder name where the instrument image is saved, the image name corresponding to the label, the absolute path where the image is saved, the size of the label image, and the information of the channel and the annotated object; Convert the instrument file in XML format to a label file in YOLO format; The label file is divided into a training set and a validation set according to a preset ratio.
3. The instrument intelligent reading method based on deep learning according to claim 1 is characterized in that: The backbone network includes: G-Ghost layer 1, SFB layer 1, G-Ghost layer 2, SFB layer 2, G-Ghost layer 3, SFB layer 3 and UpSample layer connected in series; wherein, The structures of SFB layer 1, SFB layer 2 and SFB layer 3 are the same, and all include: Channel split, concat module and Channel Shuffle connected in sequence, Channel split and concat are connected through a parallel first branch and a second branch, and the second branch includes a serial 1×1 depth convolution, a 3×3 convolution and a 1×1 depth convolution; The structures of G-Ghost layer 1, G-Ghost layer 2 and G-Ghost layer 3 are the same, and all include: 3×3 convolution, several parallel 1×1 convolutions and concat modules. The 3×3 convolutions are respectively connected to several parallel 1×1 convolutions, and several parallel 1×1 convolutions are all connected to the concat module.
4. The instrument intelligent reading method based on deep learning according to claim 3 is characterized in that: The process of SFB layer 1 processing input data is as follows: Input data of size C×H×W is input into Channel split. Channel split inputs the input data into the first branch and the second branch with the same number of input channels, respectively, to obtain the first feature map of (C / 2)×H×W and the second feature map of (C / 2)×H×W, respectively; where C is the number of channels, H and W are the height and width of the feature map, respectively; The concat module fuses the first feature map and the second feature map, and uses Channel Shuffle to perform channel-level fusion operations to obtain the initial fusion features; Channel Shuffle performs channel-level fusion operations on the initial fusion features; The process of SFB layer 1 processing input data is as follows: Input the feature tensor of size Cin×H×W into the 3×3 convolution to generate a core feature tensor of size Cont×H×W; where Cin is the input channel; A set of lightweight linear transformations are performed on the core feature tensor using 1×1 convolution to generate 4 redundant feature tensors of size Cont×H×W; The core feature tensor and the generated redundant features are concatenated in the channel dimension to form the final output feature of size C×H×W.
5. The instrument intelligent reading method based on deep learning according to claim 1 is characterized in that: The step of performing template matching on the detection result image to obtain a template matching result image includes: Grayscale the detection result image and the template image; The AKAZE algorithm is used to perform feature point detection on the grayscale detection result image and the template image to detect the first feature point set. And the second feature point set ; Traverse the first feature point set All feature points in the second feature point set are found The corresponding matching points in the first mapping relationship set are obtained ; Traverse the second feature point set All feature points in the second feature point set are found The corresponding matching points in the second mapping relationship set are obtained ; Obtaining a correct matching pair according to the first mapping relationship set and the second mapping relationship set; The random sampling consistency algorithm is used to screen out the correct matching pairs with large matching errors in the image and obtain the homography matrix; The obtained homography matrix is used to perform perspective transformation on the detection result image to obtain the template matching result image.
6. The instrument intelligent reading method based on deep learning according to claim 1 is characterized in that: The step of binarizing the template matching result image to obtain a binarized result image includes: The template matching result image is converted into a grayscale image, and the grayscale values of the grayscale image are counted to obtain a grayscale histogram; Based on the grayscale histogram, K-means clustering is performed using the K-means algorithm to update the centroid of the grayscale histogram, calculate the average grayscale value of each category, and obtain a new centroid; Iteratively update the centroid and redistribute the grayscale values according to the new centroid until convergence to obtain two classes; The average value of the two classes is used to determine the binarization threshold. According to the binarization threshold, the original grayscale image is binarized to generate a binarization result image.
7. The instrument intelligent reading method based on deep learning according to claim 1 is characterized in that: The step of performing circle detection on the binarization result image to obtain a circle detection result image includes: Based on the binarization result graph, the least squares method is used to fit the circle where the dial scale lines are located. The optimal function matching of a set of data is determined by minimizing the sum of square errors to determine the fitting circle parameters. According to the fitting circle parameters, the circle detection result graph is obtained.
8. The instrument intelligent reading method based on deep learning according to claim 1 is characterized in that: The step of accumulating pixels of the circle detection result image to obtain a pointer detection image includes: The pixel points of the circle detection result map are summed vertically, and the horizontal coordinate of the straight line where the pointer is located is the column with the smallest summed value. Combined with the pointer angle and range, the pointer detection map and instrument readings are obtained.