Distribution room pointer type instrument reading identification method, electronic equipment and storage medium

Abstract

The invention is suitable for the technical field of intelligent detection, and provides a power distribution room pointer type instrument reading recognition method, electronic equipment and a storage medium, and the method comprises the steps: inputting a to-be-recognized image into a target detection model in pointer type instrument reading recognition models, and obtaining a dial region detection result and a text region detection result; inputting the character area detection result into a character recognition model in a pointer type instrument reading recognition model to obtain a character recognition result including a measurement unit, an initial scale value and a full scale value; inputting the dial plate area detection result into a pointer identification model in a pointer type instrument reading identification model to obtain a pointer information identification result including an angle value between the starting scale line and a pointer and an angle value between the starting scale line and a full-scale scale line; and the reading recognition result is determined based on the character recognition result and the pointer information recognition result, so that the reading recognition efficiency of the power distribution room pointer instrument is improved.

Description

Technical Field

[0001] This application belongs to the technical field of intelligent detection, and particularly relates to a method for reading and recognizing pointer-type instrument readings in a power distribution room, an electronic device, and a storage medium. Background Art

[0002] With the rapid development of instrument and meter technology, instruments and meters are widely used in various industries. The use of pointer-type instruments still has an irreplaceable position in many factories and transportation equipment in our country. Its advantages are being sturdy and durable, inexpensive, and the fluctuation of its indication (i.e., the swing of the pointer) can vividly represent the change of parameters. These instruments are used for real-time monitoring and recording of various key data, providing an important basis for production operation. However, the traditional method of reading instrument data relies on manual inspection tours. This method not only has low efficiency but also has problems such as copying errors and omissions, and it is difficult to discover and handle potential safety hazards in a timely manner.

[0003] With the development of industrial automation, image processing, and pattern recognition technologies, robot intelligent inspection tour technology has emerged to replace manual inspection tours, improving the inspection tour efficiency and reducing the risks and costs of manual inspection tours. An important task during the inspection tour is the reading and recognition of pointer-type instrument readings. In places such as power distribution rooms, there are a large number of different types of pointer-type instruments, corresponding to different measurement units and ranges, etc. In response to this situation, the currently common method is to separately train corresponding reading and recognition models for each type of pointer-type instrument and use different reading and recognition models to perform reading and recognition of the corresponding type of pointer-type instrument. However, this method has a large amount of calculation and low efficiency of reading and recognition.

[0004] Therefore, how to improve the efficiency of reading and recognizing pointer-type instrument readings in a power distribution room has become an urgent problem to be solved. Summary of the Invention

[0005] The embodiments of this application provide a method for reading and recognizing pointer-type instrument readings in a power distribution room, an electronic device, and a storage medium, aiming to improve the efficiency of reading and recognizing pointer-type instrument readings in a power distribution room.

[0006] First aspect, an embodiment of the present application provides a method for identifying the readings of pointer-type meters in a power distribution room. The method includes: obtaining an image to be identified, where the image to be identified includes a dial image of a target pointer-type meter, and the target pointer-type meter is any pointer-type meter in the power distribution room; inputting the image to be identified into a target detection model in a trained pointer-type meter reading identification model to obtain a dial area detection result and a text area detection result; inputting the text area detection result into a text recognition model in the pointer-type meter reading identification model to obtain a text recognition result, where the text recognition result includes the measurement unit, starting scale value, and full-scale value of the target pointer-type meter; inputting the dial area detection result into a pointer recognition model in the pointer-type meter reading identification model to obtain a pointer information recognition result, where the pointer information recognition result includes a first target angle value and a second target angle value, the first target angle value is the angle value of the included angle between the starting scale line and the pointer of the target pointer-type meter, and the second target angle is the angle value of the included angle between the starting scale line and the full-scale line of the target pointer-type meter; and determining the reading recognition result of the target pointer-type meter based on the text recognition result and the pointer information recognition result.

[0007] In a possible implementation, the pointer-type meter reading identification model includes a model input module, the target detection model, the text recognition model, the pointer recognition model, a first model output module, and a second model output module; the target detection model is a YOLO model, and its input is connected to the model input module; the text recognition model is a DBNet model, and its input is connected to the output of the target detection model; the pointer recognition model is a PFLD model, and its input is connected to the output of the target detection model; the first model output module is connected to the output of the text recognition model; the second model output module is connected to the output of the pointer recognition model.

[0008] In a possible implementation, the training process of the pointer-type meter reading identification model includes: obtaining a set of pointer-type meter dial images; constructing and training a YOLO model based on the set of pointer-type meter dial images to obtain the target detection model; constructing and training a DBNet model based on the set of pointer-type meter dial images to obtain the text recognition model; constructing and training a PFLD model based on the set of pointer-type meter dial images to obtain the pointer recognition model; and constructing the pointer-type meter reading identification model based on the target detection model, the text recognition model, and the pointer recognition model.

[0009] In a possible implementation, constructing and training a YOLO model based on the pointer-type instrument dial image set to obtain the target detection model includes: annotating the dial area and text area in each pointer-type instrument dial image in the pointer-type instrument dial image set to obtain a first annotation information set, where the first annotation information set includes the position information of the dial area and the position information of the text area in each pointer-type instrument dial image; constructing a first sample data set based on the pointer-type instrument dial image set and the first annotation information set; constructing and training a YOLO model based on the first sample data set to obtain the target detection model.

[0010] In a possible implementation, constructing and training a DBNet model based on the pointer-type instrument dial image set to obtain the text recognition model includes: annotating the text area and text content in each pointer-type instrument dial image in the pointer-type instrument dial image set to obtain a second annotation information set, where the second annotation information set includes the position information of the text area in each pointer-type instrument dial image and the text content corresponding to each text area; constructing a second sample data set based on the pointer-type instrument dial image set and the second annotation information set; constructing and training a DBNet model based on the second sample data set to obtain the text recognition model.

[0011] In a possible implementation, the pointer recognition model includes a key point recognition sub-model and a key angle recognition sub-model; constructing and training a PFLD model based on the pointer-type instrument dial image set to obtain the pointer recognition model includes: annotating the key points and key angles in each pointer-type instrument dial image in the pointer-type instrument dial image set to obtain a third annotation information set, where the third annotation information set includes the position information of the key points and the position information and angle values of the key angles in each pointer-type instrument dial image, the key points include the starting scale value point, the full-scale scale value point, the pointer tip point and the pointer bottom point, and the key angles include the angle between the starting scale line and the pointer and the angle between the starting scale line and the full-scale scale line; constructing a third sample data set based on the pointer-type instrument dial image set and the third annotation information; co-training the main branch network and the auxiliary network in the PFLD model based on the third sample data set to obtain the key point recognition sub-model and the key angle recognition sub-model; constructing the pointer recognition model based on the key point recognition sub-model and the key angle recognition sub-model.

[0012] In a possible implementation, inputting the dial area detection result into the pointer recognition model in the pointer-type instrument reading recognition model to obtain a pointer information recognition result includes: inputting the dial area detection result into the key point recognition sub-model in the pointer recognition model to obtain a dial key point recognition result, where the dial key point recognition result includes the position information of the starting scale value point, full-scale value point, pointer tip point, and pointer bottom point of the target pointer-type instrument; and inputting the dial key point recognition result into the key angle recognition sub-model in the pointer recognition model to obtain the pointer information recognition result.

[0013] In a possible implementation, determining the reading recognition result of the target pointer-type instrument based on the text recognition result and the pointer recognition result includes: calculating the measured value of the target pointer-type instrument based on the starting scale value, full-scale value, first target angle value, and second target angle value; and determining the reading recognition result based on the measurement unit and measured value of the target pointer-type instrument.

[0014] In a second aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method described in the first aspect or any one of its implementation manners is implemented.

[0015] In a third aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, where when the computer program is executed by a processor, the method described in the first aspect or any one of its implementation manners is implemented.

[0016] In a fourth aspect, an embodiment of the present application provides a computer program product, including a computer program, where when the computer program is executed by a processor, the steps of the method described in the first aspect or any one of its implementation manners are implemented.

[0017] The beneficial effects of the embodiments of the present application compared with the prior art are as follows: An image to be recognized containing the dial image of a target pointer-type instrument is input into the target detection model in the pointer-type instrument reading recognition model to obtain the dial area detection result and the text area detection result; the text area detection result is input into the text recognition model in the pointer-type instrument reading recognition model to obtain the text recognition result containing the measurement unit, starting scale value, and full-scale value of the target pointer-type instrument; the dial area detection result is input into the pointer recognition model in the pointer-type instrument reading recognition model to obtain the pointer information recognition result including the first target angle value and the second target angle value, where the first target angle value is the angle value of the included angle between the starting scale line and the pointer of the target pointer-type instrument, and the second target angle is the angle value of the included angle between the starting scale line and the full-scale line of the target pointer-type instrument; based on the text recognition result and the pointer information recognition result output by the model, the reading recognition result of the target pointer-type instrument is determined. Using a single pointer-type instrument reading recognition model can perform reading recognition on various pointer-type instruments with different measurement units and different ranges, without the need to separately train reading recognition models for different types of pointer-type instruments, saving computing resources and improving the efficiency of reading recognition of pointer-type instruments in the power distribution room.

[0018] It can be understood that the electronic device, computer-readable storage medium, and computer program product provided by the embodiments of the present application have the same beneficial effects as the above-mentioned method for recognizing the readings of pointer-type instruments in the power distribution room, and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0020] Figure 1 It is a schematic flowchart of a method for recognizing the readings of pointer-type instruments in the power distribution room provided by an embodiment of the present application;

[0021] Figure 2 It is a schematic structural diagram of a pointer-type instrument reading recognition model provided by an embodiment of the present application;

[0022] Figure 3 It is a schematic structural diagram of a target detection model provided by an embodiment of the present application;

[0023] Figure 4 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] In the following description, for the purpose of illustration rather than limitation, specific details such as specific system architectures, technologies, etc. are presented to provide a thorough understanding of the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0025] It should be understood that when used in the specification and appended claims of the present application, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0026] It should also be understood that the term "and / or" used in the specification and appended claims of the present application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0027] As used in the specification and appended claims of the present application, the term "if" can be interpreted as "when", "once", "in response to determining", or "in response to detecting" depending on the context. Similarly, the phrase "if determined" or "if [the described condition or event] is detected" can be interpreted as meaning "once determined", "in response to determining", "once [the described condition or event] is detected", or "in response to detecting [the described condition or event]" depending on the context.

[0028] In addition, in the description of the specification and appended claims of the present application, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0029] The reference to "one embodiment" or "some embodiments" etc. described in the specification of the present application means that a specific feature, structure, or characteristic described in connection with that embodiment is included in one or more embodiments of the present application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in another way. The terms "comprising", "including", "having", and their variants all mean "including but not limited to", unless otherwise specifically emphasized in another way.

[0030] For ease of understanding, the technical solutions of the present application will be introduced in detail below in conjunction with the accompanying drawings.

[0031] Figure 1 The figure is a schematic flow chart of a method for identifying the readings of pointer-type instruments in a distribution room according to an embodiment of the present application. For the sake of convenience of description, only the parts related to this embodiment are shown. The method provided in this embodiment includes the following steps:

[0032] S110. Obtain an image to be recognized, where the image to be recognized includes a dial image of a target pointer-type instrument, and the target pointer-type instrument is any pointer-type instrument in the distribution room.

[0033] Specifically, the image to be recognized is obtained by a distribution room inspection robot. The distribution room mainly contains 10-kilovolt (kV) medium-voltage switch cabinets, and the pointer-type instruments on the switch cabinets mainly include ammeters, voltmeters, pressure gauges, etc.; the distribution room inspection robot is equipped with a variety of sensors, including but not limited to cameras, infrared thermal imagers, smoke detectors, environmental sensors, lidar, ultrasonic sensors, and vision systems, to detect various possible faults or abnormal conditions, such as overheated equipment, leaks, fire hazards, etc.; the image to be recognized obtained by the distribution room inspection robot includes a complete dial image of any one or more pointer-type instruments in the distribution room.

[0034] In a specific implementation, the inspection path of the inspection robot is preset according to the layout of the distribution room to ensure that all key areas are covered, and the shooting position and shooting angle of the inspection robot when shooting the image to be recognized are set to ensure that the obtained image to be recognized includes a complete and clear dial image of the target pointer-type instrument. When the inspection robot performs the inspection task, it travels along the set inspection path, realizes automatic navigation and obstacle avoidance through the loaded lidar, ultrasonic sensors, and vision systems, and obtains the image to be recognized at the specified shooting position through the loaded camera. In addition, during the inspection process, the inspection robot collects a large amount of real-time data through its carried sensors and transmits these data to the background system for processing and analysis to help quickly locate potential problems and issue alarms in a timely manner.

[0035] S120. Input the image to be recognized into the target detection model in the trained pointer-type instrument reading recognition model to obtain the dial area detection result and the text area detection result.

[0036] In a possible implementation manner, such as Figure 2As shown in the figure, the pointer-type instrument reading recognition model includes a model input module, an object detection model, a text recognition model, a pointer recognition model, a first model output module, and a second model output module. Among them, the object detection model is a YOLO model, and its input is connected to the model input module; the text recognition model is a DBNet model, and its input is connected to the output of the object detection model; the pointer recognition model is a Parallel Facial Landmark Detection (PFLD) model, and its input is connected to the output of the object detection model; the first model output module is connected to the output of the text recognition model; the second model output module is connected to the output of the pointer recognition model.

[0037] Based on the model architecture of the above pointer-type instrument reading recognition model, the training process of the pointer-type instrument reading recognition model includes: obtaining a set of pointer-type instrument dial images; based on the set of pointer-type instrument dial images, constructing and training a YOLO model to obtain an object detection model; based on the set of pointer-type instrument dial images, constructing and training a DBNet model to obtain a text recognition model; based on the set of pointer-type instrument dial images, constructing and training a PFLD model to obtain a pointer recognition model; based on the object detection model, the text recognition model, and the pointer recognition model, constructing a pointer-type instrument reading recognition model.

[0038] In specific implementation, a large number of dial images of pointer-type instruments are collected and screened. The screened dial images should cover various instrument types (such as different measurement units, different ranges, different dial shapes, etc.), different shooting angles, lighting conditions, and backgrounds, and the images are clear, and the details of the dial and the pointer are easy to identify. They are constructed into a set of pointer-type instrument dial images to train the pointer-type instrument reading recognition model.

[0039] Further, the steps of constructing and training a YOLO model based on the set of pointer-type instrument dial images to obtain an object detection model include: annotating the dial area and the text area in each pointer-type instrument dial image in the set of pointer-type instrument dial images to obtain a first annotation information set, and the first annotation information set includes the position information of the dial area and the position information of the text area in each pointer-type instrument dial image; based on the set of pointer-type instrument dial images and the first annotation information set, constructing a first sample data set; based on the first sample data set, constructing and training a YOLO model to obtain an object detection model.

[0040] In a specific implementation, each pointer-type instrument dial image in the pointer-type instrument dial image set is converted into a black-and-white image to reduce the size of the training data, discard redundant features, improve the recognition accuracy, and shorten the training time. The set of converted pointer-type instrument dial images and the first annotation information set are constructed into a first sample data set. The first sample data set is divided into a test set and a validation set according to a preset ratio. The YOLO model is trained using the test set, and the performance of the model, including indicators such as accuracy and recall, is evaluated using the validation set. The trained YOLO model is used as the object detection model.

[0041] As an example, as Figure 3 shown, the object detection model includes a model input module, a first CBL convolutional layer, a first pooling layer, a hidden layer, a first convolutional layer, a second pooling layer, a second CBL convolutional layer, a second convolutional layer, a dropout layer, a third convolutional layer, a third pooling layer, a fully connected layer, and a model output module connected in sequence; it also includes a first bottleneck1 layer, a third CBL convolutional layer, a second bottleneck1 layer, a fourth CBL layer, and a spatial pyramid pooling (SPP) layer connected in sequence. Among them, the input of the first bottleneck1 layer is connected to the output of the first CBL convolutional layer, and the output of the SPP layer is connected to the third convolutional layer; it also includes a bottleneck2 layer, whose input is connected to the output of the hidden layer and whose output is connected to the dropout layer.

[0042] Exemplarily, the Bottleneck2 layer is mainly used to extract the deep semantic information and detailed features such as the dial pointer of the image. Through a design called the "bottleneck structure", the amount of calculation and the number of parameters are effectively reduced, while maintaining the expression ability of the model. Bottleneck2 is usually composed of two standard convolutional modules connected in series, and the output is short-circuited with the input to form a residual connection. This design helps to solve the problem of gradient disappearance when the network is deepened, enabling the model to be trained deeper.

[0043] Exemplarily, in a deep learning model, the CBL convolutional layer is a common network structure component, especially in computer vision tasks such as object detection. It includes a convolutional layer, a batch normalization layer, and a Leaky ReLU activation function layer, combining the advantages of these three layers. It can effectively extract image features, accelerate model training, and improve the generalization ability of the model. Among them, the convolutional layer is responsible for extracting features from the input data. By multiplying the convolutional kernel with the sliding window of the input data and summing (usually adding a bias term), a feature map is generated. The convolutional layer usually uses a convolutional kernel with a stride of 2 to perform downsampling of the feature map, reducing the size of the feature map while increasing the number of feature maps, making it easier to capture the features of the text area. The batch normalization layer normalizes the output of the convolutional layer to make its distribution more stable, which helps to accelerate the training speed, improve the model convergence, and reduce the risk of overfitting. The Leaky ReLU activation function layer introduces non-linearity, enabling the model to capture complex patterns in the input data.

[0044] Exemplarily, the convolutional kernel of the first convolutional layer is 3, the stride is 5, the padding is 2, and the activation function is the ReLU function; the convolutional kernel of the second convolutional layer is 3, the stride is 7, and the padding is 2; the convolutional kernel of the third convolutional layer is 5, the stride is 3, and the padding is 2.

[0045] In a possible implementation, the image to be recognized is input into the object detection model in the trained pointer-type instrument reading recognition model. The object detection model divides the input image to be recognized into a 36×36 grid, and each grid is responsible for predicting the object whose center point falls within the grid. Each grid will predict a fixed number of bounding boxes and the confidence scores of these bounding boxes. The confidence score reflects whether there is an object in the bounding box and the accuracy of the bounding box. In addition, the object detection model also predicts the conditional probabilities of two types of objects, namely the dial area and the text area, that may appear in each grid. In the inference stage, the object detection model calculates the class confidence score of each bounding box, that is, the confidence score of the bounding box multiplied by the probability that the object predicted by the bounding box belongs to a certain class. Then, through the non-maximum suppression algorithm, duplicate bounding boxes are removed. When screening the bounding boxes, the horizontal positions of the bounding boxes in the dial area are the same, and the vertical positions are equally spaced. Finally, the detection results of the dial area and the text area are output. The detection result of the dial area includes the position information and class information of the dial area, and the detection result of the text area includes the position information and class information of the text area. Among them, the class information includes the dial and the text.

[0046] S130, input the detection result of the text area into the text recognition model in the pointer-type instrument reading recognition model to obtain the text recognition result, which includes the measurement unit, the starting scale value, and the full-scale value of the target pointer-type instrument.

[0047] In a possible implementation, the steps of constructing and training a DBNet model based on a set of pointer-type instrument dial images to obtain a text recognition model include: annotating the text regions and text contents in each pointer-type instrument dial image in the set of pointer-type instrument dial images to obtain a second annotation information set, where the second annotation information set includes the position information of the text regions in each pointer-type instrument dial image and the text content corresponding to each text region; constructing a second sample data set based on the set of pointer-type instrument dial images and the second annotation information set; constructing and training a DBNet model based on the second sample data set to obtain a text recognition model.

[0048] Specifically, DBNet is a segmentation-based text detection algorithm that introduces a Differentiable Binarization module into the segmentation model, enabling the model to perform binarization through an adaptive threshold map, and the adaptive threshold map can calculate the loss, which can play an auxiliary role in optimizing the model training process.

[0049] In a specific implementation, each pointer-type instrument dial image in the set of pointer-type instrument dial images is converted into a black-and-white image, and the text regions in each pointer-type instrument dial image in the set of pointer-type instrument dial images are annotated using polygons or rectangular frames, and the text content corresponding to each text region is annotated to obtain a second annotation information set; the converted set of pointer-type instrument dial images and the second annotation information set are constructed into a second sample data set, and data preprocessing is performed on the second sample data set, including steps such as image parsing, real label processing, random cropping, and image enhancement, to improve the generalization ability of the model; according to the second annotation information set, expansion and contraction operations are performed on the converted set of pointer-type instrument dial images to generate a probability map and a threshold map for training, and the second sample data set after data preprocessing is used to optimize the network parameters through the backpropagation algorithm. During the training process, DBNet will simultaneously learn the generation of the probability map, the threshold map, and the approximate binary map. Its loss function usually includes Dice Loss (used to evaluate the similarity between the prediction result and the real label), Mask L1 Loss (calculating the L1 distance between the predicted value and the label), and Balance Loss (used to improve the problem of unbalanced positive and negative samples). These loss functions jointly guide the training of the network, enabling the network to more accurately predict the text region and the text content.

[0050] As an example, the text region detection result is input into the text recognition model in the pointer-type instrument reading recognition model, and the text recognition result is obtained, including the measurement unit ampere (A), the starting scale value 0, and the full-scale value 400. Usually, the type of the target pointer-type instrument can be determined by the measurement unit. For example, when the measurement unit is A, it is determined that the target pointer-type instrument is an ammeter.

[0051] S140. Input the dial region detection result into the pointer recognition model in the pointer-type instrument reading recognition model to obtain the pointer information recognition result. The pointer information recognition result includes a first target angle value and a second target angle value. The first target angle value is the angle value of the included angle between the starting scale line and the pointer of the target pointer-type instrument, and the second target angle is the angle value of the included angle between the starting scale line and the full-scale line of the target pointer-type instrument.

[0052] In a possible implementation manner, the pointer recognition model includes a key point recognition sub-model and a key angle recognition sub-model; then the steps of constructing and training the PFLD model based on the pointer-type instrument dial image set to obtain the pointer recognition model include: annotating the key points and key angles in each pointer-type instrument dial image in the pointer-type instrument dial image set to obtain a third annotation information set. The third annotation information set includes the position information of the key points, the position information and angle values of the key angles in each pointer-type instrument dial image. The key points include the starting scale value point, the full-scale value point, the pointer tip point, and the pointer bottom point. The key angles include the included angle between the starting scale line and the pointer and the included angle between the starting scale line and the full-scale line; constructing a third sample data set based on the pointer-type instrument dial image set and the third annotation information; co-training the main branch network and the auxiliary network in the PFLD model based on the third sample data set to obtain the key point recognition sub-model and the key angle recognition sub-model; constructing the pointer recognition model based on the key point recognition sub-model and the key angle recognition sub-model.

[0053] Specifically, the PFLD model includes a main branch network and an auxiliary network. The main branch network is used to predict the position of the key points. This network is based on the architecture of MobilenetV2 and enhances the feature extraction ability of the network through ordinary convolution, MobilenetV2Block, and multi-scale fusion; the auxiliary network is mainly used to predict angles to improve the stability and robustness of the key point positions. During the training process, the auxiliary network can supervise the training of the main branch network, enabling the model to pay more attention to those rare and overly large pose angle samples, thereby improving the prediction accuracy.

[0054] Furthermore, the PFLD model can achieve refined recognition, mainly due to the following key designs and optimizations:

[0055] 1) Parallel network structure

[0056] The PFLD model adopts a parallel network structure, including a keypoint localization network and a position estimation network. This design enables the model to obtain the position information of the dial through the position estimation network while detecting the keypoints of the dial, thus better understanding the rotation angle and position of the dial. This parallel structure not only improves the detection accuracy but also enhances the robustness of the model to complex scenarios.

[0057] 2) Multi-scale feature fusion

[0058] The PFLD model fuses features of three scales, large, medium, and small, during the feature extraction process. This multi-scale processing mechanism enables the model to adapt to dials of different sizes and distances, thus improving the generalization ability and adaptability of the detection. Through multi-scale feature fusion, the PFLD model can capture the detailed information of the dial structure more precisely.

[0059] 3) Lightweight backbone network

[0060] The backbone network of the PFLD model adopts the lightweight MobileNet structure instead of large models such as traditional VGG16 or ResNet. This design not only reduces the computational amount and storage requirements of the model but also enhances the expressive ability of the model through structural optimization. For example, the PFLD model adjusts the width parameter of MobileNet to achieve a good balance between size and performance.

[0061] 4) Auxiliary network and loss function optimization

[0062] The PFLD model introduces an auxiliary network to estimate the rotation information of the dial and designs a new loss function to increase the penalty for difficult samples. This design enables the model to pay more attention to rare samples and samples with larger position angles, thus improving the accuracy of keypoint localization.

[0063] 5) Dense keypoint information learning

[0064] The PFLD model learns dense keypoint information through a deep learning network, which can not only accurately locate the positions of the keypoints but also capture the subtle changes in the deformation of the dial. This ability makes the PFLD model show high stability and reliability in practical applications.

[0065] 6) Efficient training and optimization

[0066] The PFLD model is trained using a large amount of dial image data and adopts an effective optimization algorithm to improve the performance and generalization ability of the model. This training strategy enables the PFLD to achieve good detection results in the actual environment.

[0067] In a specific implementation, each pointer-type instrument dial image in the pointer-type instrument dial image set is converted into a black-and-white image, and each pointer-type instrument dial image is labeled. The labeling content includes the positions of key points, the positions and angular values of key angles. The labeling format is usually Extensible Markup Language (XML) or JSON. Open-source software such as LabelImg can be used as the labeling tool. Next, data preprocessing operations are performed, including cropping and scaling, normalization, and data augmentation. Cropping and scaling mean cropping each pointer-type instrument dial image into a region containing the dial and adjusting the image size to match the model input size (such as 112×112 pixels). Normalization is to normalize the pixel values of the image, usually scaling the pixel values to the range [0, 1]. To improve the generalization ability of the model, data augmentation operations can be performed on the image, such as random rotation, flipping, brightness adjustment, etc. Modify the configuration file of the PFLD model, specify the paths of the training and validation data sets, and adjust hyperparameters such as the learning rate, number of training epochs, and batch size. Run the training script to start training the PFLD model. Input the third sample data set into both the main branch network and the auxiliary network in the PFLD model for collaborative training. During the training process, the model will continuously optimize the weights through backpropagation to minimize the error between the predicted key points and the true labels. Monitor the training process, observe the changes in the loss function, and the prediction accuracy of key points and key angles. If it is found that the training is unstable or overfitting, the learning rate can be adjusted or data augmentation operations can be increased. Evaluate the performance of the model on the validation set, calculate metrics such as the average error (such as root mean square error) and accuracy of key point detection. Randomly select some validation set images, draw the key points and key angles predicted by the model on the images, and visually check the recognition effect of the model. According to the evaluation results of the validation set, further adjust hyperparameters such as the learning rate and data augmentation strategy to improve the performance of the model. If it is necessary to deploy the model on a resource-constrained device, pruning and quantization operations can be performed on the trained model. The main branch network after training is used as the key point recognition sub-model, and the auxiliary network is used as the key angle recognition sub-model.

[0068] As an example, the loss function used to train the PFLD model is:

[0069] Loss = MSE1 + MSE2,

[0070] where MSE1 represents the mean square error of key points, and MSE2 represents the mean square error of key angles.

[0071] In a possible implementation, the detection result of the dial area is input into the key point recognition sub-model in the pointer recognition model to obtain the dial key point recognition result. The dial key point recognition result includes the position information of the starting scale value point, the full-scale value point, the pointer tip point, and the pointer bottom point of the target pointer-type instrument. The dial key point recognition result is input into the key angle recognition sub-model in the pointer recognition model to obtain the pointer information recognition result, that is, the angle value between the starting scale line of the target pointer-type instrument and the pointer, and the angle value between the starting scale line and the full-scale line of the target pointer-type instrument.

[0072] S150. Based on the text recognition result and the pointer information recognition result, determine the reading recognition result of the target pointer-type instrument.

[0073] In a possible implementation, based on the starting scale value and the full-scale value in the text recognition result, and the first target angle value and the second target angle value in the pointer information recognition result, calculate the measured value of the target pointer-type instrument; then, based on the measurement unit and the measured value of the target pointer-type instrument, determine the reading recognition result.

[0074] As an example, based on the starting scale value X0 and the full-scale value X in the text recognition result m , and the first target angle value θ1 and the second target angle value θ in the pointer information recognition result m , calculate the measured value Y of the target pointer-type instrument through the following formula:

[0075]

[0076] Exemplarily, the text recognition result of the target pointer-type instrument is that the measurement unit is A, the starting scale value is 0, and the full-scale value is 400; the pointer recognition result is that the first target angle is 45 degrees and the second target angle is 90 degrees. Then, the measured value of the target pointer-type instrument can be calculated as 200 through the above formula. From the measured value and the measurement unit of the target pointer-type instrument, the final reading recognition result is 200A.

[0077] The technical solution provided by this application inputs the image to be recognized containing the dial image of the target pointer-type instrument into the target detection model in the pointer-type instrument reading recognition model to obtain the dial area detection result and the text area detection result; inputs the text area detection result into the text recognition model in the pointer-type instrument reading recognition model to obtain the text recognition result containing the measurement unit, starting scale value, and full-scale value of the target pointer-type instrument; inputs the dial area detection result into the pointer recognition model in the pointer-type instrument reading recognition model to obtain the pointer information recognition result including the first target angle value and the second target angle value, where the first target angle value is the angle value of the included angle between the starting scale line and the pointer of the target pointer-type instrument, and the second target angle is the angle value of the included angle between the starting scale line and the full-scale line of the target pointer-type instrument; determines the reading recognition result of the target pointer-type instrument based on the text recognition result and the pointer information recognition result output by the model. Using a pointer-type instrument reading recognition model can perform reading recognition on various pointer-type instruments with different measurement units and different ranges, without separately training reading recognition models for different types of pointer-type instruments, saving computing resources and improving the efficiency of reading recognition of pointer-type instruments in the power distribution room.

[0078] Figure 4 It is a schematic structural diagram of an electronic device provided by an embodiment of this application. As Figure 4 shown, the electronic device 4 of this embodiment includes: at least one processor 40 ( Figure 4 only one is shown in the figure), a memory 41, and a computer program 42 stored in the memory 41 and executable on at least one processor 40. When the processor 40 executes the computer program 42, it implements the steps in the above Figure 1 method embodiment.

[0079] The electronic device 4 may be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The electronic device 4 may include but is not limited to the processor 40 and the memory 41. Those skilled in the art can understand that Figure 4 this is only an example of the electronic device 4 and does not constitute a limitation on the electronic device 4. It may include more or fewer components than shown in the figure, or combine some components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0080] The processor 40 may be a Central Processing Unit (CPU), and the processor 40 may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0081] In some embodiments, the memory 41 may be an internal storage unit of the electronic device 4, such as the hard disk or memory of the electronic device 4. In other embodiments, the memory 41 may also be an external storage device of the electronic device 4, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc., equipped on the electronic device 4. Further, the memory 41 may also include both the internal storage unit and the external storage device of the electronic device 4. The memory 41 is used to store an operating system, application programs, a BootLoader, data, and other programs, such as the program code of a computer program. The memory 41 may also be used to temporarily store data that has been output or is to be output.

[0082] The embodiments of the present application also provide a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments can be implemented.

[0083] When an integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above method embodiments of this application, a computer program can be used to instruct relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can at least include: any entity or device that can carry the computer program code to an electronic device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electrical carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disc, etc.

[0084] A computer-readable storage medium provided by an embodiment of this application has the same beneficial effects as the above method for identifying the reading of a pointer-type instrument in a power distribution room.

[0085] An embodiment of this application provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments can be implemented.

[0086] A computer program product provided by an embodiment of this application has the same beneficial effects as the above method for identifying the reading of a pointer-type instrument in a power distribution room.

[0087] In the above embodiments, the descriptions of the respective embodiments have their own emphases. For parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0088] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0089] In the embodiments provided in the present application, it should be understood that the disclosed apparatus / devices and methods can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the apparatus or unit can be in electrical, mechanical or other forms.

[0090] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0091] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features. And these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for identifying the reading of a pointer instrument in a power distribution room, characterized in that: The method comprises: Acquire an image to be identified, wherein the image to be identified includes a dial image of a target pointer instrument, and the target pointer instrument is any pointer instrument in a power distribution room; Inputting the image to be recognized into the target detection model in the trained pointer meter reading recognition model to obtain the dial area detection result and the text area detection result; Inputting the text area detection result into the text recognition model in the pointer instrument reading recognition model to obtain a text recognition result, wherein the text recognition result includes the measurement unit, the starting scale value and the full scale value of the target pointer instrument; The dial area detection result is input into the pointer recognition model in the pointer instrument reading recognition model to obtain a pointer information recognition result, wherein the pointer information recognition result includes a first target angle value and a second target angle value, wherein the first target angle value is an angle value of an angle between a starting scale line and a pointer of the target pointer instrument, and the second target angle value is an angle value of an angle between a starting scale line and a full-scale scale line of the target pointer instrument; Based on the text recognition result and the pointer information recognition result, a reading recognition result of the target pointer instrument is determined.

2. The method according to claim 1, characterized in that The pointer meter reading recognition model includes a model input module, the target detection model, the text recognition model, the pointer recognition model, a first model output module and a second model output module; The target detection model is a YOLO model, and its input is connected to the model input module; The text recognition model is a DBNet model, whose input is connected to the output of the object detection model; The pointer recognition model is a PFLD model, whose input is connected to the output of the target detection model; The first model output module is connected to the output of the text recognition model; The second model output module is connected to the output of the pointer recognition model.

3. The method according to claim 2, characterized in that The training process of the pointer meter reading recognition model includes: Get the pointer instrument dial image collection; Based on the pointer instrument dial image set, a YOLO model is constructed and trained to obtain the target detection model; Based on the pointer instrument dial image set, a DBNet model is constructed and trained to obtain the text recognition model; Based on the pointer instrument dial image set, a PFLD model is constructed and trained to obtain the pointer recognition model; Based on the target detection model, the text recognition model and the pointer recognition model, the pointer meter reading recognition model is constructed.

4. The method according to claim 3, characterized in that The method of constructing and training a YOLO model based on the pointer instrument dial image set to obtain the target detection model includes: Annotating a dial area and a text area in each pointer-type instrument dial image in the pointer-type instrument dial image set to obtain a first annotation information set, wherein the first annotation information set includes position information of the dial area and position information of the text area in each of the pointer-type instrument dial images; Constructing a first sample data set based on the pointer instrument dial image set and the first annotation information set; Based on the first sample data set, a YOLO model is constructed and trained to obtain the target detection model.

5. The method according to claim 3, characterized in that: The method of constructing and training a DBNet model based on the pointer instrument dial image set to obtain the text recognition model includes: Annotating the text area and text content in each pointer instrument dial image in the pointer instrument dial image set to obtain a second annotation information set, wherein the second annotation information set includes position information of the text area in each pointer instrument dial image and text content corresponding to each text area; Constructing a second sample data set based on the pointer instrument dial image set and the second annotation information set; Based on the second sample data set, a DBNet model is constructed and trained to obtain the text recognition model.

6. The method according to claim 3, characterized in that The pointer recognition model includes a key point recognition sub-model and a key angle recognition sub-model; The method of constructing and training a PFLD model based on the pointer instrument dial image set to obtain the pointer recognition model includes: Annotate key points and key angles in each pointer-type instrument dial image in the pointer-type instrument dial image set to obtain a third annotation information set, wherein the third annotation information set includes position information of key points in each pointer-type instrument dial image and position information and angle values ​​of key angles, wherein the key points include a starting scale value point, a full-scale scale value point, a pointer tip point, and a pointer bottom point, and the key angles include an angle between a starting scale line and the pointer and an angle between the starting scale line and the full-scale scale line; Constructing a third sample data set based on the pointer instrument dial image set and the third annotation information; Based on the third sample data set, the main branch network and the auxiliary network in the PFLD model are collaboratively trained to obtain the key point recognition sub-model and the key angle recognition sub-model; The pointer recognition model is constructed based on the key point recognition sub-model and the key angle recognition sub-model.

7. The method according to claim 6, characterized in that The step of inputting the dial area detection result into the pointer recognition model in the pointer meter reading recognition model to obtain the pointer information recognition result includes: Input the dial area detection result into the key point recognition sub-model in the pointer recognition model to obtain the dial key point recognition result, wherein the dial key point recognition result includes the position information of the starting scale value point, the full scale value point, the pointer tip point and the pointer bottom point of the target pointer instrument; The dial key point recognition result is input into the key angle recognition sub-model in the pointer recognition model to obtain the pointer information recognition result.

8. The method according to any one of claims 1 to 7, characterized in that: The step of determining the reading recognition result of the target pointer instrument based on the text recognition result and the pointer recognition result includes: Calculating a measurement value of the target pointer instrument based on the starting scale value, the full-scale scale value, the first target angle value, and the second target angle value; The reading recognition result is determined based on the measurement unit and the measurement value of the target pointer instrument.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images