Digital display meter reading method and device, storage medium and computer equipment

Abstract

According to the digital display meter reading method and device, the storage medium and the computer equipment provided by the invention, when the digital display meter image is read, the target detection model can be obtained, and the changeable kernel convolution block in the model can enable the model to better adapt to complex light-variable scenes and the processing capability of irregular targets such as digits; the spatial pyramid pooling module can enhance the global perception capability of the model; the Monte Carlo attention module can capture related information of different scales, and the small target recognition capability of the network is enhanced. Therefore, the model is adopted to perform character detection on the digital display meter image, a high-precision detection result can be obtained, the detection result comprises a digital display meter position frame, a plurality of character position frames and corresponding category labels, and through each category label, character combination can be directly performed on each character position frame in the digital display meter position frame, so that the accuracy of character combination is improved. And a reading result is generated, so that a traditional multi-model cascade architecture is avoided, and the detection efficiency and the real-time performance are improved.

Description

Technical Field

[0001] The present application relates to the field of target detection technology, and in particular to a method, device, storage medium and computer equipment for reading a digital display meter. Background Art

[0002] In the flexible interface cabinet autonomous quality inspection system, real-time data collection from cabinet digital displays is crucial for assessing construction quality and power system operating status. As core monitoring equipment, digital displays directly reflect system operating conditions, including voltage, current, and power, and are crucial for construction quality verification and fault warning. Traditional manual inspection methods suffer from low efficiency and high errors, leading to the development of automated meter recognition using machine vision technology.

[0003] Most existing technical solutions build recognition processes based on deep learning target detection frameworks. For example, the YOLOv5 cascade ResNet architecture can be used to first locate the instrument area through YOLOv5, and then the cropped image is input into ResNet for digital recognition. This type of method achieves basic recognition functions through staged processing. Compared with traditional manual inspection methods, it can adapt to complex industrial scenarios in terms of detection accuracy and inference speed.

[0004] However, existing solutions still have inherent limitations. For one thing, when the digital display is small, there is light interference, or the camera angle is offset, the object detection model is prone to missed detections or positioning errors, resulting in subsequent digital recognition failures. Furthermore, the multi-model cascade architecture significantly increases the computational load, making it difficult to deploy in embedded devices or scenarios with high real-time requirements. In short, the current model-based detection approach still lacks the accuracy and system response speed to meet the intelligent needs of large-scale cabinet inspections. Summary of the Invention

[0005] The purpose of this application is to solve at least one of the above-mentioned technical defects, especially the technical defect that the separation detection method of the model in the existing technology is still difficult to meet the intelligent requirements of large-scale panel cabinet inspection in terms of detection accuracy and system response speed.

[0006] The present application provides a method for reading a digital display meter, the method comprising:

[0007] Collect digital display images of the target area through automatic inspection equipment;

[0008] Determining a target detection model, wherein the target detection model is constructed and trained using a variable kernel convolution block, a spatial pyramid pooling module, and a Monte Carlo attention module;

[0009] Performing character detection on the DRO image using the target detection model to obtain a plurality of position frames and a category label for each position frame, wherein the position frames include a DRO position frame and a plurality of character position frames;

[0010] Based on each category label, character combination is performed on each character position frame in the digital display position frame to generate a reading result.

[0011] Optionally, determining the target detection model includes:

[0012] Acquire a sample DRO image including a DRO, wherein the sample DRO image includes a DRO real position frame and a character real position frame respectively marking the DRO and each character in the display area of the DRO;

[0013] Constructing an initial detection model, and inputting the sample DRO image into the initial detection model, obtaining a plurality of predicted position frames output by the initial detection model for the sample DRO image and a predicted category label for each predicted position frame; the predicted position frames include a DRO predicted position frame and a character predicted position frame;

[0014] Training the initial detection model with the goal of making the digital display predicted position frame and the character predicted position frame approach the digital display real position frame and the character real position frame respectively;

[0015] When the initial detection model meets the preset training conditions, the trained initial detection model is used as the target detection model.

[0016] Optionally, the constructing of the initial detection model includes:

[0017] Obtain a pre-trained model based on YOLOv8, wherein the pre-trained model consists of a backbone network, a neck network, and a head network;

[0018] The Conv convolution blocks in the first, third, fifth, and seventh layers of the backbone network are replaced with variable kernel convolution blocks, and a global average pooling layer and a global maximum pooling layer are added to the SPPF module in the backbone network to form a spatial pyramid pooling module;

[0019] Replace all Conv convolution blocks in the neck network with variable kernel convolution blocks, and add Monte Carlo attention modules before all detection heads in the head network;

[0020] Based on the adjusted backbone network, neck network and head network, an initial detection model is generated.

[0021] Optionally, the predicted category label includes a category corresponding to the predicted position box and position coordinates of the predicted position box in the digital display image;

[0022] The training of the initial detection model with the goal of making the digital display predicted position frame and the character predicted position frame approach the digital display real position frame and the character real position frame respectively includes:

[0023] Calculate the classification loss value of each predicted position box according to the category of each predicted position box;

[0024] Calculate the position regression loss value of each predicted position frame based on the position coordinates of each predicted position frame;

[0025] Determining the relative position between the DRO prediction position frame and each character prediction position frame based on the category of the DRO position frame, and calculating a relative position loss value of each relative position;

[0026] Parameters in the initial detection model are updated based on the classification loss value, the position regression loss value, and the relative position loss value.

[0027] Optionally, the target detection model includes a backbone network, a neck network and a head network;

[0028] The method of using the target detection model to perform character detection on the digital display image to obtain a plurality of position frames and a category label for each position frame includes:

[0029] Inputting the digital display meter image into the backbone network, so that the backbone network extracts features of the digital display meter in the digital display meter image and outputs digital display meter features;

[0030] The neck network is used to fuse multiple digital display features output by the backbone network to obtain fused features;

[0031] The fused features are input into the head network to obtain multiple position boxes output by the head network and a category label for each position box.

[0032] Optionally, the category label includes the location coordinates, category and confidence of the location box;

[0033] The step of combining characters of the character position frames in the digital display position frame based on the respective category labels to generate a reading result of the digital display image includes:

[0034] Determining whether the confidence level of the digital display position frame is lower than a preset threshold;

[0035] If yes, the reading result of the digital display corresponding to the digital display position frame is not output;

[0036] If not, characters corresponding to each character position frame are generated according to each category, and each character is arranged and combined according to the position coordinates of each character position frame in the digital display position frame to obtain the reading result of the digital display corresponding to the digital display position frame.

[0037] Optionally, the method further comprises:

[0038] The reading result is marked in the digital display image based on the position coordinates of the digital display position frame, and the marked digital display image is sent to the console for storage.

[0039] The present application also provides a digital display meter reading device, comprising:

[0040] Image acquisition module, used to collect digital display images of target areas through automatic inspection equipment;

[0041] A model determination module is used to determine a target detection model, wherein the target detection model is constructed and trained using a variable kernel convolution block, a spatial pyramid pooling module, and a Monte Carlo attention module;

[0042] a model detection module, configured to perform character detection on the DRO image using the target detection model to obtain a plurality of position frames and a category label for each position frame, wherein the position frames include a DRO position frame and a plurality of character position frames;

[0043] The result generating module is used to perform character combination on each character position frame in the digital display position frame based on each category label to generate a reading result.

[0044] The present application also provides a storage medium, characterized in that: the storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the digital display meter reading method as described in any of the above embodiments.

[0045] The present application also provides a computer device, characterized by comprising: one or more processors, and a memory;

[0046] The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the digital display meter reading method as described in any one of the above embodiments are performed.

[0047] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0048] The present application provides a method, device, storage medium, and computer device for reading a digital display. When a digital display needs to be read, an image of the digital display in the target area can be captured by an automatic inspection device, and then a pre-trained target detection model can be obtained. The model is constructed and trained using a variable kernel convolution block, a spatial pyramid pooling module, and a Monte Carlo attention module. The variable kernel convolution block can dynamically adjust its shape and size based on the input data and local features, allowing the model to better adapt to complex scenes with changing lighting conditions and irregular targets such as numbers. The introduction of the spatial pyramid pooling module can enhance the model's global perception capabilities. The Monte Carlo attention module can capture relevant information at different scales, enhancing the network's ability to recognize small targets. Therefore, by using this model to perform character detection on the digital display image, high-precision detection results can be obtained. The detection results include multiple position frames and category labels for each position frame. Here, the position frame includes a digital display position frame and multiple character position frames. Through various category labels, this application can directly perform character combination on each character position frame in the digital display position frame to generate the reading results of the digital display image, thereby avoiding the traditional multi-model cascade architecture and improving detection efficiency and real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0050] Figure 1 A flowchart of a method for reading a digital display meter provided in an embodiment of the present application;

[0051] Figure 2 This is a diagram showing the effect of data enhancement results in an embodiment of the present application;

[0052] Figure 3 A schematic diagram of the structure of an initial detection model provided in an embodiment of the present application;

[0053] Figure 4 A schematic diagram of the marking of the confidence result of the position frame of a digital display provided in an embodiment of the present application;

[0054] Figure 5 A schematic structural diagram of a digital display reading device provided in an embodiment of the present application;

[0055] Figure 6 A schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0057] Existing solutions still have inherent limitations. For one thing, when the digital display is small, there is light interference, or the camera angle is offset, the object detection model is prone to missed detections or positioning errors, resulting in subsequent digital recognition failures. Furthermore, the multi-model cascade architecture significantly increases the computational load, making it difficult to deploy in embedded devices or scenarios with high real-time requirements. In short, the current model-based detection approach still lacks the accuracy and system response speed to meet the intelligent needs of large-scale cabinet inspections.

[0058] Based on this, this application proposes the following technical solutions, please refer to the following for details:

[0059] In one embodiment, Figure 1 As shown, Figure 1 A flowchart of a method for reading a digital display meter provided in an embodiment of the present application; the present application provides a method for reading a digital display meter, which specifically includes the following:

[0060] S110: Collecting a digital display image of the target area through the automatic inspection equipment.

[0061] In this step, when the digital display meter needs to be read, the automatic inspection equipment can be used to capture an image of the digital display meter in the target area to form a digital display meter image for subsequent automated reading.

[0062] The digital displays used for readings in this application encompass a variety of types, such as current, voltage, power, and temperature. Furthermore, to facilitate management of digital displays, digital displays in a target area are often deployed together. Therefore, the digital display images captured by the automated inspection equipment may include one or more digital displays, and the types of digital displays can be diverse, without limitation.

[0063] It is understandable that the present application can select automatic inspection equipment according to the environmental characteristics of the target area, such as fixed-mounted industrial cameras, drone inspection equipment, rail-mounted inspection robots, or mobile inspection robots. To elaborate, for fixed-mounted industrial cameras, multiple cameras can be arranged at appropriate locations to achieve continuous monitoring of digital displays; drone inspection equipment is suitable for inspections of large-scale or high-altitude equipment, and can flexibly adjust the shooting angle to cover hard-to-reach areas; rail-mounted inspection robots are usually deployed in equipment-intensive areas, running along a set path and taking pictures at predetermined time intervals, while mobile inspection robots can rely on visual navigation or lidar SLAM technology to move autonomously in complex environments, identify digital displays, and take precise pictures.

[0064] Specifically, when capturing DDR images, automated inspection equipment can be equipped with infrared or intelligent fill-light systems to adapt to varying lighting conditions and ensure DDR image clarity. Furthermore, by employing autofocus, intelligent angle adjustment, and dynamic exposure control technologies, the acquisition equipment can adapt to readings at varying angles and distances, improving image legibility. After capturing DDR images, they can be transmitted in real time to a computer via a wireless or wired network, enabling the computer to automatically interpret and interpret the images.

[0065] Furthermore, after receiving the digital display image, the computer device can also pre-process the digital display image. The pre-processing process includes but is not limited to sharpening and denoising the image. Sharpening refers to compensating for the image contour, enhancing the image edge and grayscale jump portion, and making the image clear. It can be divided into two categories: spatial domain processing and frequency domain processing. By highlighting the edges and contours of the objects on the image, or the features of certain linear target elements, the contrast between the edges of the objects and the surrounding pixels is improved. Denoising refers to the process of reducing noise in digital images. Generally, when image information is digital, and during the digitization and transmission of the image, it is often affected by the imaging device and the external environment noise interference, that is, the received image information generally includes noise, and this noise will become an important cause of image interference. By denoising the image, the noise in the image can be removed, further improving the authenticity and accuracy of the image.

[0066] S120: Determine a target detection model. The target detection model is constructed and trained using a variable kernel convolution block, a spatial pyramid pooling module, and a Monte Carlo attention module.

[0067] In this step, after the digital display image is acquired in step S110, the computer device can also determine the target detection model, which is constructed and trained using a variable kernel convolution block, a spatial pyramid pooling module, and a Monte Carlo attention module, thereby improving detection efficiency and accuracy.

[0068] It can be understood that in order to further shorten the detection time and improve the detection efficiency, the present application can use a sample digital display image containing a digital display to pre-train a preset initial detection model, and determine the initial detection model after training as the target detection model, that is, when the digital display image is acquired and read for the first time, the corresponding target detection model can be determined and saved. In this way, if the image data containing the digital display needs to be read subsequently, the target detection model can be used directly without the need to determine it again.

[0069] Specifically, the initial detection model here is constructed using a variable kernel convolution block, a spatial pyramid pooling module, and a Monte Carlo attention module. Among them, the variable kernel convolution block can dynamically adjust its shape and size according to the input data and local features, so that the model can better adapt to complex scenes with changing light and irregular targets such as numbers; and the introduction of the spatial pyramid pooling module can enhance the global perception ability of the model, capture feature information of different ranges, so that the network can take into account both large and small targets without changing the input size; the Monte Carlo attention module can capture relevant information of different scales and enhance the network's small target recognition ability. Therefore, by using this model to read digital display images, the present application can more accurately and quickly locate the character area of the digital display and its display area in the image data, and then can quickly and accurately identify the specific readings therein.

[0070] S130: Performing character detection on the DRO image using the target detection model to obtain a plurality of position frames and a category label for each position frame, wherein the position frames include a DRO position frame and a plurality of character position frames.

[0071] In this step, after determining the target detection model in step S130, the computer device can input the acquired digital display image into the target detection model so that the target detection model performs character detection on the digital display image to obtain multiple position frames and category labels for each position frame. The position frames here can include a digital display position frame and multiple character position frames located in the digital display position frame.

[0072] Among them, the type label can be used to describe the properties of the digital display and characters corresponding to each detected position frame, such as position coordinates, category, confidence level, etc., providing a reliable basis for subsequent automated reading.

[0073] It should be noted that when there is only one DRO in the DRO image, the target detection model detects only one DRO position frame; when there are multiple DROs in the DRO image, the target detection model detects multiple DRO position frames, each DRO corresponds to a DRO position frame, each DRO position frame is independent of each other, and each DRO position frame contains multiple character position frames.

[0074] Furthermore, when the computer device identifies and outputs the position frame through the target detection model, it can also simultaneously output the confidence level corresponding to each position frame, so that when subsequently identifying the specific reading of each digital display in the digital display image, it can be determined based on the confidence level of each position frame whether to output and display the recognition result of the corresponding position frame, thereby further improving the accuracy of the digital display reading.

[0075] S140: Based on each category label, character combinations are performed on each character position frame in the digital display position frame to generate a reading result.

[0076] In this step, after obtaining multiple position frames and the category label of each position frame through step S130, the computer device can perform character combination on each character position frame in the digital display position frame based on each category label to generate a reading result of the digital display corresponding to the digital display position frame, thereby avoiding the traditional multi-model cascade architecture and improving detection efficiency and real-time performance.

[0077] It is understood that the computer device can determine the spatial distribution of each position box and the classified category based on the various category labels, then sort the various character position boxes within the DRO position box according to the DRO reading method to obtain the corresponding DRO reading order. The characters corresponding to the categories in each character position box are then combined according to this reading order to form the final reading result. Unlike traditional multi-model cascade architectures, this method can complete character location and recognition with a single target detection model, avoiding additional character segmentation and independent OCR recognition steps, significantly improving detection efficiency and real-time performance.

[0078] In the above embodiment, when a digital display meter needs to be read, an image of the digital display meter in the target area can be collected by an automatic inspection device, and then a pre-trained target detection model is obtained. The model is constructed and trained using a variable kernel convolution block, a spatial pyramid pooling module, and a Monte Carlo attention module. Among them, the variable kernel convolution block can dynamically adjust its shape and size according to the input data and local features, so that the model can better adapt to complex scenes with changing light and irregular targets such as numbers; the introduction of the spatial pyramid pooling module can enhance the model's global perception ability; and the Monte Carlo attention module can capture relevant information at different scales, enhancing the network's small target recognition ability. Therefore, using this model to perform character detection on a digital display meter image can obtain high-precision detection results. The detection results include multiple position frames and category labels for each position frame. Here, the position frame includes a digital display meter position frame and multiple character position frames. Through each category label, the present application can directly combine the characters of each character position frame in the digital display meter position frame to generate a reading result for the digital display meter image, thereby avoiding the traditional multi-model cascade architecture and improving detection efficiency and real-time performance.

[0079] In one embodiment, the step of determining the target detection model in step S120 may include:

[0080] S121: Acquire a sample DRO image including a DRO, wherein the sample DRO image includes a DRO real position frame and a character real position frame respectively marking the DRO and each character in the display area of the DRO.

[0081] S122: Construct an initial detection model and input the sample digital display image into the initial detection model to obtain multiple predicted position frames output by the initial detection model for the sample digital display image and a predicted category label for each predicted position frame; the predicted position frames include a digital display prediction position frame and a character prediction position frame.

[0082] S123: Training an initial detection model with the goal of making the digital display's predicted position frame and the character's predicted position frame approach the digital display's true position frame and the character's true position frame respectively.

[0083] S124: When the initial detection model meets the preset training conditions, the trained initial detection model is used as the target detection model.

[0084] In this embodiment, when reading a digital display image for the first time, a target detection model needs to be trained. First, a computer device can obtain a sample digital display image containing a digital display, wherein the sample digital display image includes a digital display real position frame and a character real position frame respectively marked for the digital display and each character in the display area of the digital display; at the same time, an initial detection model can be constructed to input the sample digital display image into the initial detection model and obtain multiple predicted position frames output by the initial detection model for the sample digital display image and a predicted category label for each predicted position frame; the predicted position frames include a digital display predicted position frame and a character predicted position frame. Then, the computer device can train the initial detection model with the goal of making the digital display predicted position frame and the character predicted position frame approach the digital display real position frame and the character real position frame respectively. When the initial detection model meets the preset training conditions, such as when the number of iterations reaches a set value, the trained initial detection model is used as the target detection model.

[0085] Specifically, after obtaining a sample digital display meter image containing a digital display meter, the computer device can also perform data enhancement on the sample digital display meter image, including Gaussian blur, adjusting brightness and contrast, adding noise and other data enhancement operations, so as to simulate the digital display meter images under different lighting scenes, increase the number of samples in the data set, and thereby improve the robustness and accuracy of model training.

[0086] For Gaussian blur, computers can use the GaussianBlur function in the OpenCV library to smooth images. The ksize parameter in this function controls the size of the Gaussian kernel; a larger kernel produces a more pronounced blur. Here, Gaussian blur is applied to the original image to simulate varying degrees of blur. The size and shape of the blur kernel can also be varied to enhance the model's robustness to blurred images.

[0087] To adjust brightness and contrast, computers can use the convertScaleAbs function in the OpenCV library. The alpha parameter controls the contrast enhancement, while the beta parameter controls the brightness adjustment. By adjusting the brightness and contrast of an image, you can simulate the image of a digital display under different lighting conditions, enhancing the model's adaptability to varying brightness and contrast.

[0088] For adding noise, the computer device can use the add_noise function to add noise to the image. It supports three types of noise: Gaussian noise, salt and pepper noise, and Poisson noise. Adding Gaussian noise randomly generates Gaussian distributed noise and superimposes it on the image; adding salt and pepper noise randomly adds white and black pixels to the image; adding Poisson noise generates noise according to the Poisson distribution, which is suitable for simulating noise in optical imaging.

[0089] Schematically, as Figure 2 As shown, Figure 2 Figure 1 shows the effects of data enhancement results from an embodiment of this application. Figure (a) shows the original image of a digital display; Figure (b) shows the effect after Gaussian blurring the original image; Figure (c) shows the effect after brightness and contrast adjustments; and Figure (d) shows the effect after noise addition. Therefore, by performing data augmentation on the collected original image, the size of the original dataset can be effectively expanded, thereby enhancing the robustness and accuracy of model training.

[0090] Furthermore, when using data-augmented sample DRO images to train the initial detection model, the sample DRO images can be divided into a training set, a validation set, and a test set in a ratio of 7:2:1. This allows for sequential training, validation, and testing of the initial detection model. During training, appropriate loss functions, optimizers, and hyperparameters can be selected. Through cross-validation and model evaluation, the model structure and training strategy can be adjusted to ensure that the model achieves high detection accuracy on different test sets, ultimately resulting in an optimized object detection model.

[0091] In one embodiment, the process of constructing the initial detection model in step S122 may include:

[0092] S1221: Obtain a pre-trained model based on YOLOv8. The pre-trained model consists of a backbone network, a neck network, and a head network.

[0093] S1222: Replace the Conv convolution blocks in the first, third, fifth, and seventh layers of the backbone network with variable kernel convolution blocks, and add global average pooling layers and global maximum pooling layers to the SPPF module in the backbone network to form a spatial pyramid pooling module.

[0094] S1223: Replace all Conv convolution blocks in the neck network with variable kernel convolution blocks, and add Monte Carlo attention modules before all detection heads in the head network.

[0095] S1224: Generate an initial detection model based on the adjusted backbone network, neck network, and head network.

[0096] In this embodiment, when constructing the initial detection model, the computer device can build a pre-trained model consisting of a backbone network, a neck network and a head network based on YOLOv8 (You Only Look Once v8), and improve the structure of the model, including replacing the Conv convolution blocks in the first, third, fifth and seventh layers of the backbone network with AKConv convolution blocks (changeable kernel convolution blocks), and adding a global average pooling layer and a global maximum pooling layer to the original SPPF module, called the SPPF_Avg_Max module (spatial pyramid pooling module), and replacing all Conv convolution blocks in the neck network with AKConv convolution blocks (changeable kernel convolution blocks), and adding a MoCAttention attention module (Monte Carlo attention module) in front of all detection heads in the head network.

[0097] YOLOv8 is the latest version of the YOLO (You Only Look Once) series of object detection models. Compared to previous YOLO versions such as YOLOv5 and YOLOv7, YOLOv8 has been optimized and improved in terms of detection accuracy, inference speed, and model structure. It not only supports object detection, but also has functions such as instance segmentation, image classification, and key point detection, suitable for a variety of computer vision tasks. Based on this, this application can also further improve the pre-trained model built on YOLOv8 by introducing the AKConv convolution block, SPPF_Avg_Max module, and MoCAttention attention module to further improve the model's detection accuracy and efficiency.

[0098] Specifically, the AKConv convolution block is a flexible convolution mechanism that breaks through the limitations of traditional convolution, which relies on fixed kernels and local windows. Unlike traditional convolution operations that rely on kernels of fixed shape and size, the AKConv convolution block allows the kernel to dynamically adjust its shape and size based on the input data and local features. It can adjust the sampling position to more accurately capture target features at different locations and scales. This adaptive convolution approach enables the network to better adapt to complex scenes with changing lighting conditions and handle irregular objects such as numbers.

[0099] The MoCAttention module uses a pooling operation based on random sampling to generate a scale-independent attention map. This enables the network to capture relevant information at different scales and enhances the network's ability to recognize small objects. Given an input tensor x, the output attention map Am(x) of the MoCAttention module is calculated as follows:

[0100]

[0101] Where x represents the input tensor and represents the output size of the attention map; represents the association probability function, and f(x, i) represents the average pooling function.

[0102] The SPPF_Avg_Max module builds on the existing SPPF module by introducing global average pooling and global maximum pooling layers to enhance the network's global perception. Specifically, the global average pooling layer extracts global background information, helping the network better understand the overall context of the image; while the global maximum pooling layer extracts edge and salient feature information, enhancing the network's perception of detail.

[0103] Schematically, as Figure 3 As shown, Figure 3 A structural diagram of an initial detection model provided in an embodiment of the present application; wherein backbone represents the backbone network, Neck represents the neck network, and Head represents the head network.

[0104] In one embodiment, the predicted category label in step S123 may include the category corresponding to the predicted position frame and the position coordinates of the predicted position frame in the DRO image; wherein, with the goal of making the DRO predicted position frame and the character predicted position frame approach the DRO real position frame and the character real position frame, respectively, the process of training the initial detection model may include:

[0105] S1231: Calculate the classification loss value of each predicted position frame according to the category of each predicted position frame.

[0106] S1232: Calculate the position regression loss value of each predicted position frame based on the position coordinates of each predicted position frame.

[0107] S1233: Determine the relative position between the DRO prediction position frame and each character prediction position frame based on the category of the DRO position frame, and calculate the relative position loss value of each relative position.

[0108] S1234: Update the parameters in the initial detection model based on the classification loss value, the position regression loss value, and the relative position loss value.

[0109] In this embodiment, during the training of the initial detection model, after the initial detection model outputs a predicted location frame and a predicted category label including a category and location coordinates, the computer device can calculate a classification loss value and a position regression loss value for each predicted location frame based on the category and location coordinates of each predicted location frame. Simultaneously, based on the category of the digital display location frame, the computer device determines the relative position between the digital display location frame and each character predicted location frame therein, and calculates a relative position loss value for each relative position. Finally, the computer device can update the parameters of the initial detection model based on the classification loss value, the position regression loss value, and the relative position loss value.

[0110] Specifically, for the classification loss value, the computer device can use the classification loss function to calculate the classification loss value between the category corresponding to each predicted position box output by the model and the category corresponding to each real position box marked in the sample data table image; similarly, for the position regression loss value, the computer device can use the position regression loss function to calculate the classification loss value between the coordinate position corresponding to each predicted position box output by the model and the coordinate position corresponding to each real position box marked in the sample data table image; and for the relative position loss value, the computer device can determine the relative position between the digital display table prediction position box and each character prediction position box therein based on the category of the digital display table position box, and use the relative position loss function to calculate the relative position loss value of each relative position.

[0111] Furthermore, after obtaining the classification loss value, position regression loss value, and relative position loss value respectively, the computer device can derive each loss value to obtain the corresponding gradient, and perform back propagation through the chain calculation method to obtain the updated parameters, and then the updated parameters can be used to update the parameters in the initial detection model.

[0112] In one embodiment, the object detection model in step S130 may include a backbone network, a neck network, and a head network. The process of using the object detection model to perform character detection on the digital display image to obtain multiple position boxes and a category label for each position box may include:

[0113] S131: Inputting the DRO image into the backbone network, so that the backbone network extracts features of the DRO in the DRO image and outputs DRO features.

[0114] S132: The neck network is used to fuse multiple digital display features output by the backbone network to obtain fused features.

[0115] S133: Input the fused features into the head network to obtain multiple position boxes output by the head network and a category label for each position box.

[0116] In this embodiment, when using the target detection model to perform character detection on the digital display meter image, the computer device can first input the digital display meter image into the backbone network, so that the backbone network can extract features of the digital display meter in the digital display meter image and output the digital display meter features. Then, the neck network is used to fuse the multiple digital display meter features output by the backbone network to obtain fused features. Finally, the fused features can be input into the head network to obtain multiple position boxes output by the head network and the category label of each position box.

[0117] Specifically, a computer device can input a DRO image into the object detection model's backbone network. This network extracts key features from the DRO image and generates a multi-layer DRO feature map. This network typically consists of multiple convolutional layers, residual blocks, or a Transformer architecture to extract feature information at different scales and semantic levels, enabling the model to accurately identify the characteristics of the DRO and the characters in its display area.

[0118] After the backbone network extracts the DRO features, these features are passed to the neck network, which further fuses the multi-scale features from the backbone network and enhances the model's detection capabilities for objects of varying sizes. To ensure that DROs and characters can be clearly detected at various sizes, the neck network can also incorporate structures such as FPN (Feature Pyramid Network) or PAN (Path Aggregation Network). Through cross-layer connections and fusion, shallow features retain more spatial information, while deep features retain more semantic information. This allows the model to improve detection accuracy and robustness even for smaller characters by fusing features at different scales, preventing misdetection or omission of characters due to blur or lighting interference.

[0119] Finally, the fused features are input into the head network. The main task of the head network is to perform target detection based on the fused features and output multiple location boxes and category labels. The location boxes are used to identify the detected overall area of the digital display and the specific character area in the digital display, while the category labels can be used to describe the properties of the location boxes.

[0120] In one embodiment, the category label in step S140 may include the position coordinates, category, and confidence level of the position frame; wherein, the process of combining characters of each character position frame in the digital display position frame based on each category label to generate a reading result may include:

[0121] S141: Determine whether the confidence level of the digital display position frame is lower than a preset threshold.

[0122] S142: If yes, the reading result of the digital display corresponding to the digital display position frame is not output.

[0123] S143: If not, generate characters corresponding to each character position frame according to each category, and arrange and combine each character according to the position coordinates of each character position frame in the digital display position frame to obtain the reading result of the digital display corresponding to the digital display position frame.

[0124] In this embodiment, since the category label includes the position coordinates, category, and confidence level of the location frame, when generating the DRO reading result, the computer device can first determine whether the confidence level of the DRO location frame is below a preset threshold. If so, it indicates that the DRO reading result for that DRO location frame is less reliable, and the computer device may not output the DRO reading result corresponding to that DRO location frame. Otherwise, it indicates that the DRO reading result for that DRO location frame is more accurate, and the computer device can generate characters corresponding to each character location frame based on each category, and arrange and combine the characters according to their position coordinates within the DRO location frame to obtain the DRO reading result corresponding to the DRO location frame.

[0125] Specifically, the category label for each location box can be output in the form of {x1, y1, x2, y2, label, confidence}. x1 and y1 represent the coordinates of the top-left corner of the location box, x2 and y2 represent the coordinates of the bottom-right corner of the location box, label represents the detected category, and confidence represents the confidence score. For digital display location boxes, label can be represented by "digital." For character location boxes, the labels for digits 0-9 can be the digits themselves, the decimal point "." can be represented by "point," and the minus sign can be represented by "minus."

[0126] It is understandable that the present application can determine whether the reading result of the corresponding digital display is reliable by the confidence of the digital display position frame, and decide whether to output it. Figure 4 As shown, Figure 4 A diagram illustrating the confidence results of a digital display position frame provided in an embodiment of the present application; as can be seen from the figure, the confidence scores of the position frames labeled "digital" all exceed the preset threshold value of 0.9, so the computer device can directly obtain the characters corresponding to each character position frame in each digital display and arrange and combine them to obtain the reading results and output them.

[0127] Because the computer can directly retrieve the specific value of each character from the category in the category label returned by the model, it avoids the potential misreading issues that might arise from using an additional OCR model, and improves the real-time and accuracy of the readings. Furthermore, for each DRO, the computer can sort the corresponding characters according to the horizontal and vertical coordinates of each character position box, from left to right per row, ensuring that the character order matches the actual display order and that the reading order is consistent with the content displayed on the DRO, thereby accurately obtaining the DRO reading results.

[0128] In one embodiment, the method may further include:

[0129] S150: Marking the reading result in the digital display image based on the position coordinates of the digital display position frame, and sending the marked digital display image to the console for storage.

[0130] In this embodiment, after reading the reading result of the digital display, the computer device can mark the reading result in the digital display image based on the position coordinates of the digital display position frame, and send the marked digital display image to the console for storage for subsequent data analysis and application.

[0131] Specifically, the computer device can combine the position coordinates of the digital display position frame output by the target detection model to accurately mark the reading results on the original digital display image to achieve visual display. During the marking process, the computer device can use different marking methods, such as superimposing a text box above or below the digital display, or adding a semi-transparent label to a specific area in the image to ensure that the marked information is both clear and readable and does not affect the observation of the digital display itself. Furthermore, in order to enhance the visualization effect, the computer device can also adjust the color or font size of the marked text according to the confidence level of the digital display position frame, such as using green marking for high-confidence readings and yellow or red for readings with lower confidence levels to indicate that they require manual review.

[0132] In addition, computer equipment can appropriately compress or convert the marked image to improve storage and transmission efficiency, and send the final generated marked digital display image to the console for storage and management. It can also link data with other devices to expand the application scope of the data.

[0133] The digital display meter reading device provided in an embodiment of the present application is described below. The digital display meter reading device described below and the digital display meter reading method described above can be referenced to each other.

[0134] In one embodiment, Figure 5 As shown, Figure 5This is a structural diagram of a digital display meter reading device provided in an embodiment of the present application. The present application also provides a digital display meter reading device, including an image acquisition module 210, a model determination module 220, a model detection module 230, and a result generation module 240, specifically including the following:

[0135] The image acquisition module 210 is used to acquire a digital display image of a target area through an automatic inspection device.

[0136] The model determination module 220 is used to determine the target detection model. The target detection model is constructed and trained using a variable kernel convolution block, a spatial pyramid pooling module, and a Monte Carlo attention module.

[0137] The model detection module 230 is used to perform character detection on the DRO image using the target detection model to obtain multiple position frames and a category label for each position frame. The position frames include the DRO position frame and multiple character position frames.

[0138] The result generating module 240 is used to perform character combination on each character position frame in the digital display position frame based on each category label to generate a reading result.

[0139] In the above embodiment, when a digital display meter needs to be read, an image of the digital display meter in the target area can be collected by an automatic inspection device, and then a pre-trained target detection model is obtained. The model is constructed and trained using a variable kernel convolution block, a spatial pyramid pooling module, and a Monte Carlo attention module. Among them, the variable kernel convolution block can dynamically adjust its shape and size according to the input data and local features, so that the model can better adapt to complex scenes with changing light and irregular targets such as numbers; the introduction of the spatial pyramid pooling module can enhance the model's global perception ability; and the Monte Carlo attention module can capture relevant information at different scales, enhancing the network's small target recognition ability. Therefore, using this model to perform character detection on a digital display meter image can obtain high-precision detection results. The detection results include multiple position frames and category labels for each position frame. Here, the position frame includes a digital display meter position frame and multiple character position frames. Through each category label, the present application can directly combine the characters of each character position frame in the digital display meter position frame to generate a reading result for the digital display meter image, thereby avoiding the traditional multi-model cascade architecture and improving detection efficiency and real-time performance.

[0140] In one embodiment, the model determination module 220 may include:

[0141] The sample acquisition submodule is used to acquire a sample DRO image including a DRO. The sample DRO image includes a DRO real position frame and a character real position frame respectively marking the DRO and each character in the DRO display area.

[0142] The model construction submodule is used to build an initial detection model and input the sample digital display image into the initial detection model to obtain multiple predicted position frames output by the initial detection model for the sample digital display image and the predicted category label of each predicted position frame; the predicted position frame includes the digital display prediction position frame and the character prediction position frame.

[0143] The model training submodule is used to train the initial detection model with the goal of making the digital display prediction position frame and the character prediction position frame approach the digital display true position frame and the character true position frame respectively.

[0144] The model generation submodule is used to use the trained initial detection model as the target detection model when the initial detection model meets the preset training conditions.

[0145] In one embodiment, the model building submodule may include:

[0146] The model building unit is used to obtain a pre-trained model based on YOLOv8. The pre-trained model consists of a backbone network, a neck network, and a head network.

[0147] The first module replacement unit is used to replace the Conv convolution blocks in the first, third, fifth, and seventh layers of the backbone network with variable kernel convolution blocks, and to add global average pooling layers and global maximum pooling layers to the SPPF module in the backbone network to form a spatial pyramid pooling module.

[0148] The second module replacement unit is used to replace all Conv convolution blocks in the neck network with changeable kernel convolution blocks, and to add Monte Carlo attention modules to all detection heads in the head network.

[0149] The model generation unit is used to generate an initial detection model based on the adjusted backbone network, neck network and head network.

[0150] In one embodiment, the predicted category label may include the category corresponding to the predicted location box and the position coordinates of the predicted location box in the digital display image; the model building submodule may further include:

[0151] The first loss value calculation unit is used to calculate the classification loss value of each predicted position frame according to the category of each predicted position frame.

[0152] The second loss value calculation unit is configured to calculate a position regression loss value for each predicted position frame based on the position coordinates of each predicted position frame.

[0153] The third loss value calculation unit is used to determine the relative position between the DRO prediction position frame and each character prediction position frame based on the category of the DRO position frame, and calculate the relative position loss value of each relative position.

[0154] The parameter updating unit is used to update the parameters in the initial detection model based on the classification loss value, the position regression loss value, and the relative position loss value.

[0155] In one embodiment, the target detection model may include a backbone network, a neck network, and a head network; the model detection module 230 may include:

[0156] The feature extraction submodule is used to input the digital display meter image into the backbone network so that the backbone network extracts the features of the digital display meter in the digital display meter image and outputs the digital display meter features.

[0157] The feature fusion submodule is used to use the neck network to fuse multiple digital display features output by the backbone network to obtain fused features.

[0158] The category detection submodule is used to input the fused features into the head network to obtain multiple position boxes output by the head network and the category label of each position box.

[0159] In one embodiment, the category label may include the location coordinates, category, and confidence level of the location box; the result generation module 240 may include:

[0160] The threshold judgment submodule is used to judge whether the confidence of the digital display position frame is lower than a preset threshold.

[0161] The first result submodule is configured to not output the reading result of the digital display meter corresponding to the digital display meter position frame when the confidence level is lower than a preset threshold.

[0162] The second result submodule is used to generate characters corresponding to each character position frame according to each category when the confidence level is not lower than a preset threshold, and to arrange and combine each character according to the position coordinates of each character position frame in the digital display position frame to obtain a reading result of the digital display corresponding to the digital display position frame.

[0163] In one embodiment, the apparatus may further include:

[0164] The image marking module is used to mark the reading results in the digital display image based on the position coordinates of the digital display position frame, and send the marked digital display image to the console for storage.

[0165] In one embodiment, the present application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the digital display meter reading method as described in any of the above embodiments.

[0166] In one embodiment, the present application also provides a computer device having computer-readable instructions stored therein. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the digital display meter reading method as described in any one of the above embodiments.

[0167] Schematically, as Figure 6 As shown, Figure 6 This is a schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. The computer device 300 can be provided as a server. Figure 6 Computer device 300 includes a processing component 302, which further includes one or more processors, and memory resources represented by memory 301 for storing instructions executable by processing component 302, such as application programs. The application programs stored in memory 301 may include one or more modules, each corresponding to a set of instructions. Furthermore, processing component 302 is configured to execute the instructions to perform the digital display meter reading method according to any of the above-described embodiments.

[0168] The computer device 300 may further include a power supply component 303 configured to perform power management of the computer device 300, a wired or wireless network interface 304 configured to connect the computer device 300 to a network, and an input / output (I / O) interface 305. The computer device 300 may operate based on an operating system stored in the memory 301, such as Windows Server™, Mac OS X™, Unix™, Linux™, Free BSD™, or the like.

[0169] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0170] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0171] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referenced to each other.

[0172] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for reading a digital display meter, characterized in that: The method comprises: Collect digital display images of the target area through automatic inspection equipment; Determining a target detection model, wherein the target detection model is constructed and trained using a variable kernel convolution block, a spatial pyramid pooling module, and a Monte Carlo attention module; Performing character detection on the DRO image using the target detection model to obtain a plurality of position frames and a category label for each position frame, wherein the position frames include a DRO position frame and a plurality of character position frames; Based on each category label, character combination is performed on each character position frame in the digital display position frame to generate a reading result.

2. The method for reading a digital display meter according to claim 1, wherein: Determining the target detection model includes: Acquire a sample DRO image including a DRO, wherein the sample DRO image includes a DRO real position frame and a character real position frame respectively marking the DRO and each character in the display area of the DRO; Constructing an initial detection model, and inputting the sample DRO image into the initial detection model, obtaining a plurality of predicted position frames output by the initial detection model for the sample DRO image and a predicted category label for each predicted position frame; the predicted position frames include a DRO predicted position frame and a character predicted position frame; Training the initial detection model with the goal of making the digital display predicted position frame and the character predicted position frame approach the digital display real position frame and the character real position frame respectively; When the initial detection model meets the preset training conditions, the trained initial detection model is used as the target detection model.

3. The method for reading a digital display meter according to claim 2, wherein: The constructing of the initial detection model includes: Obtain a pre-trained model based on YOLOv8, wherein the pre-trained model consists of a backbone network, a neck network, and a head network; The Conv convolution blocks in the first, third, fifth, and seventh layers of the backbone network are replaced with variable kernel convolution blocks, and a global average pooling layer and a global maximum pooling layer are added to the SPPF module in the backbone network to form a spatial pyramid pooling module; Replace all Conv convolution blocks in the neck network with variable kernel convolution blocks, and add Monte Carlo attention modules before all detection heads in the head network; Based on the adjusted backbone network, neck network and head network, an initial detection model is generated.

4. The method for reading a digital display meter according to claim 2, wherein: The predicted category label includes the category corresponding to the predicted position frame and the position coordinates of the predicted position frame in the digital display image; The training of the initial detection model with the goal of making the digital display predicted position frame and the character predicted position frame approach the digital display real position frame and the character real position frame respectively includes: Calculate the classification loss value of each predicted position box according to the category of each predicted position box; Calculate the position regression loss value of each predicted position frame based on the position coordinates of each predicted position frame; Determining the relative position between the DRO prediction position frame and each character prediction position frame based on the category of the DRO position frame, and calculating a relative position loss value of each relative position; Parameters in the initial detection model are updated based on the classification loss value, the position regression loss value, and the relative position loss value.

5. The method for reading a digital display meter according to claim 1, wherein: The target detection model includes a backbone network, a neck network and a head network; The method of using the target detection model to perform character detection on the digital display image to obtain a plurality of position frames and a category label for each position frame includes: Inputting the digital display meter image into the backbone network, so that the backbone network extracts features of the digital display meter in the digital display meter image and outputs digital display meter features; The neck network is used to fuse multiple digital display features output by the backbone network to obtain fused features; The fused features are input into the head network to obtain multiple position boxes output by the head network and a category label for each position box.

6. The method for reading a digital display meter according to claim 1, wherein: The category label includes the location coordinates, category and confidence of the location box; The step of combining characters of each character position frame in the digital display position frame based on each category label to generate a reading result includes: Determining whether the confidence level of the digital display position frame is lower than a preset threshold; If yes, the reading result of the digital display corresponding to the digital display position frame is not output; If not, the characters corresponding to each character position frame are generated according to each category, and the characters are arranged and combined according to the position coordinates of each character position frame in the digital display position frame to obtain the reading result of the digital display corresponding to the digital display position frame.

7. The method for reading a digital display meter according to claim 1, wherein: The method also includes: The reading result is marked in the digital display image based on the position coordinates of the digital display position frame, and the marked digital display image is sent to the console for storage.

8. A digital display reading device, characterized in that: include: Image acquisition module, used to collect digital display images of target areas through automatic inspection equipment; A model determination module is used to determine a target detection model, wherein the target detection model is constructed and trained using a variable kernel convolution block, a spatial pyramid pooling module, and a Monte Carlo attention module; a model detection module, configured to perform character detection on the DRO image using the target detection model to obtain a plurality of position frames and a category label for each position frame, wherein the position frames include a DRO position frame and a plurality of character position frames; The result generating module is used to perform character combination on each character position frame in the digital display position frame based on each category label to generate a reading result.

9. A storage medium, characterized in that: The storage medium stores computer-readable instructions, which, when executed by one or more processors, enable the one or more processors to perform the steps of the digital display meter reading method according to any one of claims 1 to 7.

10. A computer device, characterized in that: include: one or more processors, and memory; The memory stores computer-readable instructions, which, when executed by the one or more processors, execute the steps of the digital display meter reading method according to any one of claims 1 to 7.

Images