LAA-GAN-based low-voltage ammeter detection and positioning method and system in low-light environment

Abstract

The invention discloses an LAA-GAN-based low-voltage ammeter detection and positioning method and system in a low-light environment, relates to the technical field of image processing and identification, and solves the problems that the ammeter position and the digital area are difficult to position and the dial number error identification rate is high during ammeter detection in the low-light environment. According to the embodiment of the invention, the brightness perception attention module is arranged in the decoder of the U-Net generator, and the image enhancement effect is adaptively adjusted according to the brightness distribution of the electric meter image through the brightness attention mechanism of the brightness perception attention module, so that the low-brightness area is enhanced more strongly to improve the visibility, and the user experience is improved. A high-brightness area avoids excessive enhancement, and detail loss caused by overexposure is prevented. The decoder is responsible for executing low-light enhancement and gradually recovering a high-resolution electric meter image based on the features extracted by the encoder, and the obtained enhanced image is more convenient for a detection network to identify and position the position of the electric meter and a digital area.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing and recognition, in particular to a low-voltage meter detection and positioning method and system in low-light environment based on LAA-GAN. BACKGROUND

[0002] With the rapid development of global economy, the intelligent global energy internet has become the direction and trend of the development of the power industry. The goal of the smart grid is to build a panoramic real-time system covering all aspects of the power system, including power generation, transmission, transformation, distribution, power utilization and dispatching, etc. Using advanced technologies such as information communication, computer and control, the efficiency of each part of the power system is improved, the cost and environmental pollution are reduced, and the reliability, stability and self-healing ability of the power system are improved as much as possible.

[0003] The smart meter is one of the basic devices for data acquisition of the smart grid (especially the smart distribution network), and undertakes the tasks of raw electric energy data acquisition, metering and transmission, and is the basis for information integration, analysis and optimization and information display.

[0004] However, due to environmental light, noise and other factors, the collected smart meter image has the problems of dark brightness and detail loss, which will affect the subsequent data processing. Therefore, it is necessary to ensure the accuracy of smart meter detection in low-light environment to provide reliable data support for meter data analysis.

[0005] Traditional low-light enhancement methods such as Retinex series algorithms rely on physical illumination-reflectance decomposition model and do not consider dynamic noise interference, resulting in artifacts in the dark area of the enhanced image, which makes it difficult to locate the meter position and digital area in image detection, and affects the image recognition result.

[0006] Therefore, a low-voltage meter detection and positioning method and system in low-light environment based on luminosity-aware attention (LAA)-generative adversarial network (GAN) are needed. SUMMARY

[0007] In view of the problem of difficult positioning of meter position and digital area in smart meter detection in low-light environment in the prior art, the present application provides a low-voltage meter detection and positioning method and system in low-light environment based on LAA-GAN, which can more accurately locate the meter position and digital area. The specific technical solutions are as follows:

[0008] In the first aspect, the present application provides a low-voltage meter detection and positioning method in low-light environment based on LAA-GAN, comprising:

[0009] An electric meter image is acquired, and the electric meter image is subjected to low-light enhancement by a U-Net generator; an encoder of the U-Net generator comprises 5 levels of down-sampling layers, and a decoder of the U-Net generator comprises 5 levels of up-sampling layers, the up-sampling layers comprising a brightness-aware attention module and an up-sampling module, the brightness-aware attention module being configured to connect feature maps extracted by the down-sampling layers and feature maps extracted by the up-sampling module through a skip connection and assign weights to the connected feature maps; a calculation formula for assigning feature map weights by the brightness-aware attention module comprises:

[0010] w l =σ(Conv 1×1 ([F enc ,F dec ]))⊙Sigmoid(I dark +∈);

[0011] wherein I dark is a low-illumination region mask, σ is a Sigmoid activation function, Conv 1×1 is a 1×1 convolution layer, F enc and F dec are respectively a feature map of a first down-sampling layer in the 5 levels of down-sampling layers and a feature map of a first up-sampling layer in the 5 levels of up-sampling layers, the first down-sampling layer and the first up-sampling layer corresponding to each other, and ∈ is a noise disturbance term; the enhanced electric meter image is input into a preset detection network to obtain an electric meter position and a digital region in the enhanced electric meter image.

[0012] Preferably, a method for generating the low-illumination region mask I dark comprises: extracting a feature map of the electric meter image; performing binaryzation on a V channel in a YUV space of the feature map based on a preset threshold to obtain an initial mask; and superimposing Gaussian noise on the initial mask to obtain the low-illumination region mask I dark .

[0013] Preferably, the preset threshold is 0.1.

[0014] Preferably, a third down-sampling layer of the encoder and a third up-sampling layer of the decoder both comprise an adaptive Gaussian filtering layer and the depthwise separable convolution layer, and a dynamic adjustment formula of a filter kernel standard deviation of the adaptive Gaussian filtering layer comprises:

[0015]

[0016] wherein σ is the filter kernel standard deviation, μ(I dark ) is a mean value of an illumination distribution of the electric meter image, and max(I dark ) is a maximum value in a low-illumination mask region of the electric meter image; and an output calculation formula of the third down-sampling layer and the third up-sampling layer comprises:

[0017] F out = a F gauss + (1-a) F conv ;

[0018] wherein, F out is the output of the third down-sampling layer or the third up-sampling layer, F gauss is the output of the adaptive Gaussian filtering layer, F conv is the output of the depth separable convolution layer, and a is the brightness attention map of the meter image; wherein, the pixels of the brightness attention map are used to indicate the brightness size of the pixels at the corresponding positions in the feature map of the meter image, and the pixel value of the pixels in the brightness attention map is inversely proportional to the brightness of the corresponding pixels in the feature map of the meter image.

[0019] Preferably, the loss function of the detection network comprises:

[0020]

[0021] wherein, is the loss value, IoU is the intersection over union of the predicted bounding box and the real bounding box, the predicted bounding box is the bounding box corresponding to the predicted meter position, and the real bounding box is the corresponding bounding box of the meter position in the meter image; ρ 2 (b pred ,b gt ) represents the Euclidean distance between the centers of the predicted bounding box and the real bounding box, c represents the length of the diagonal of the smallest rectangle containing the predicted bounding box and the real bounding box, a is a hyperparameter, and v is a regression constraint representing the fixed aspect ratio of the meter; the calculation formula of the regression constraint v comprises:

[0022]

[0023] wherein, w gt and h gt are the width and height of the real bounding box, respectively, and w red and h pred are the width and height of the predicted bounding box, respectively.

[0024] Preferably, the U-Net generator adopts INT8 symmetric quantization; and the detector head of the detection network adopts FP16 decimal coordinate precision.

[0025] Preferably, before the meter image is subjected to low-light enhancement by the U-Net generator, the method further comprises: pre-training the U-Net generator for 500 times, wherein the first learning rate of the pre-training of the U-Net generator is 2x10 -4; maintain the parameters of the U-Net generator unchanged, train the detection network for 300 epochs, and the second learning rate of the detection network is 1x10 -3 ; perform 100 epochs of end-to-end joint fine-tuning on the U-Net generator and the detection network, and the third learning rate of the end-to-end joint fine-tuning is 5x10 -5 ; the total loss function of the U-Net generator and the detection network is:

[0026]

[0027] wherein, is the total loss, is the adversarial loss of the U-Net generator, is the perceptual loss of the U-Net generator, is the structural similarity loss of the U-Net generator, is the detection loss of the detection network.

[0028] In a second aspect, the embodiments of the present application provide a low-voltage meter detection and positioning system in a low-light environment based on LAA-GAN, which is applied to the method of any one of claims 1-7. The system comprises:

[0029] An acquisition module is configured to acquire a meter image.

[0030] An enhancement module is configured to perform low-light enhancement on the meter image by using a U-Net generator. The encoder of the U-Net generator comprises five down-sampling layers, and the decoder of the U-Net generator comprises five up-sampling layers. The up-sampling layer comprises a brightness perception attention module and an up-sampling module. The brightness perception attention module is configured to connect the feature map extracted by the down-sampling layer and the feature map extracted by the up-sampling module through a skip connection, and assign a weight to the connected feature map. The calculation formula of the weight comprises:

[0031] w l =σ(Conv 1×1 ([F enc ,F dec ]))⊙Sigmoid(I dark +∈);

[0032] wherein, I dark is a low-illumination region mask, σ is a Sigmoid activation function, Conv 1×1 is a 1x1 convolution layer, F enc and F dec are respectively a feature map of a first down-sampling layer in the five down-sampling layers and a feature map of a first up-sampling layer in the five up-sampling layers, and the first down-sampling layer and the first up-sampling layer correspond to each other; ∈ is a noise disturbance term.

[0033] The detection module is configured to input the enhanced meter image into a preset detection network to obtain a meter position and a digital region in the enhanced meter image.

[0034] In a third aspect, an embodiment of the present application provides a computing device, comprising: a memory configured to store a program; and a processor configured to load the program to execute the method in the first aspect.

[0035] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which comprises a stored program, wherein the program, when executed, controls a device where the computer-readable storage medium is located to perform the method in the first aspect.

[0036] Compared with the prior art, the present application has the following beneficial effects: the embodiments of the present application set a brightness-aware attention module in the decoder of the U-Net generator, and through the brightness attention mechanism of the brightness-aware attention module, the image enhancement effect is adaptively adjusted according to the brightness distribution of the meter image, so as to ensure that the low-brightness area is more strongly enhanced to improve the visibility, and the high-brightness area is prevented from being excessively enhanced to prevent the loss of details caused by excessive exposure. The decoder is responsible for gradually restoring the high-resolution meter image based on the features extracted by the encoder while performing low-light enhancement, and the obtained enhanced image is more convenient for the detection network to identify and locate the meter position and the digital region. BRIEF DESCRIPTION OF DRAWINGS

[0037] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the drawings needed in the specific embodiments or the prior art description will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn according to the actual proportions.

[0038] Figure 1 A flowchart of a low-voltage meter detection and positioning method in a low-light environment based on LAA-GAN provided by an embodiment of the present application;

[0039] Figure 2 A data flow diagram of a low-voltage meter detection and positioning method in a low-light environment based on LAA-GAN provided by an embodiment of the present application;

[0040] Figure 3 A data flow diagram of a LAA-GAN enhanced image provided by an embodiment of the present application;

[0041] Figure 4 A joint training flowchart provided by an embodiment of the present application;

[0042] Figure 5A structure schematic diagram of a low-voltage power meter detection and positioning system in a low-light environment based on LAA-GAN is provided for an embodiment of the present application.

[0043] Figure 6 A structure schematic diagram of a computing device is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0044] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present application.

[0045] It should be understood that, when used in the specification and the appended claims, the terms “comprise” and “include” indicate the presence of described features, integers, steps, operations, elements, and / or components, but do not exclude one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0046] It should also be understood that the terms used in the present application specification are only for the purpose of describing particular embodiments and are not intended to limit the present application. As used in the present application specification and the appended claims, unless otherwise clearly indicated by the context, the singular forms “a”, “an” and “the” are intended to include the plural forms.

[0047] It should be further understood that the term “and / or” used in the present application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0048] To solve the problem of difficulty in positioning the position and digital area of a power meter in a low-light environment in the traditional scheme, the present application provides a low-voltage power meter detection and positioning method and system in a low-light environment based on LAA-GAN, which can more accurately position the position and digital area of a power meter.

[0049] Please refer to Figure 1 , Figure 1 A flowchart of a low-voltage power meter detection and positioning method in a low-light environment based on LAA-GAN is provided for an embodiment of the present application, which is applied to a computing device; as shown in Figure 1 the method comprises the following steps:

[0050] Step 101, the computing device acquires a power meter image.

[0051] The computing device can be a server, a personal computer, a smart terminal, or other device with data processing and computing capabilities. The meter image refers to visual image data of a meter obtained by shooting, scanning, or the like, which can intuitively present the appearance, reading, and related information of the meter.

[0052] The computing device can obtain the meter image directly through the image acquisition device or obtain the meter image stored in the storage device.

[0053] Specifically, when the computing device obtains the meter image directly through the image acquisition device, the AE algorithm can be used for automatic exposure control, and the meter image can be pre-cut to obtain a 640*640 region of interest (ROI) region.

[0054] To effectively detect, locate, and identify the meter image in a low-light environment, the computing device can deploy a deep learning, machine learning, or other artificial intelligence model to learn a large amount of meter image data to accurately locate various regions on the meter, such as the reading region and the function button region, without being affected by factors such as angle deviation and uneven illumination in the image. At the same time, the model can also learn the features and patterns of numbers, so that even if the meter numbers have stains, wear, or irregular fonts, the model can still identify the numbers more accurately than a human being, reducing reading errors.

[0055] Preferably, the computing device deploys a luminosity-aware attention (LAA)-generative adversarial network (GAN) model, and implements image enhancement of the meter image in a low-light environment based on the LAA-GAN model.

[0056] For example, Figure 2 , Figure 2 A data flow diagram of a low-voltage meter detection and positioning method in a low-light environment based on LAA-GAN provided by an embodiment of the present application. As shown in Figure 2 , the computing device inputs the original image of the meter in a low-light environment into the LAA-GAN model to obtain an enhanced image output by the LAA-GAN model; and then inputs the enhanced image into a detector to obtain a detection result output by the detector, the detection result including the meter position and the number region in the meter image. Further, the computing device can perform number recognition on the number region to obtain data recorded by the meter.

[0057] Specifically, for example, Figure 3 , Figure 3A data flow diagram for enhancing an image by an LAA-GAN is provided for an embodiment of the present application. It can be understood that, in order to make the drawings clear, Figure 3 The connecting lines between the same layer actions are omitted in Figure 3 In a specific example of the LAA-GAN model, the LAA-GAN model includes 3 down-sampling layers and 3 up-sampling layers, the down-sampling layers include a convolution & max pooling module and a depthwise separable convolution module, and the up-sampling layers include a convolution & up-sampling & concatenation module, an LAA module, and a convolution & up-sampling layer module.

[0058] After receiving the original image, the convolution & max pooling module of the down-sampling layer will first perform convolution and max pooling operations on the original image to obtain a first feature map; the depthwise separable convolution module of the same layer will perform a depthwise separable convolution operation on the first feature map to obtain a second feature map; then the depthwise separable convolution module will pass the second feature map to the corresponding up-sampling layer and the next down-sampling layer; after passing through three down-sampling layers, the LAA-GAN model starts the up-sampling process.

[0059] In the up-sampling process, the convolution & concatenation module will concatenate the received second feature map and the third feature map obtained by up-sampling to obtain a fourth feature map; the LAA module will perform feature weight distribution based on brightness on the fourth feature map to obtain a fifth feature map; the convolution & up-sampling module will perform convolution and up-sampling operations on the fifth feature map to obtain a sixth feature map, and pass the sixth feature map to the previous up-sampling layer. After passing through three up-sampling layers, the LAA-GAN model outputs an enhanced image.

[0060] The LAA module can adaptively adjust the enhancement effect through the brightness distribution of the fourth feature map, and the feature weight distribution under the spatial attention mechanism can effectively improve the brightness of the low-light area while avoiding the problem of overexposure, making the low-light enhancement more fine. The LAA-GAN model gradually recovers a high-resolution image from the features extracted by the down-sampling layer while performing low-light enhancement to obtain the final enhanced image.

[0061] Step 102, the computing device performs low-light enhancement on the meter image by the U-Net generator.

[0062] The LAA-GAN model includes a U-Net generator and a detection network.

[0063] For example, the U-Net generator can adopt a channel compression strategy: first, increase the number of channels at each level of the encoder in a step-by-step manner, and the number of channels of the 5 down-sampling layers is 16, 32, 64, 128 and 256 respectively; the decoder uses deconvolution + channel pruning (pruning rate 20%) to recover and reconstruct the features extracted by the encoder. This channel compression strategy significantly reduces the parameter quantity and computational complexity, making the model more suitable for real-time operation on edge devices. Combined with depth separable convolution instead of standard convolution, the parameter quantity can be further reduced.

[0064] In the U-Net generator, the feature map storage uses FP16, which can reduce the memory occupation; and the FP32 precision is retained during loss calculation to prevent gradient underflow.

[0065] Preferably, the encoder of the U-Net generator includes 5 down-sampling layers, and the decoder of the U-Net generator includes 5 up-sampling layers, the up-sampling layer includes a brightness-aware attention module and an up-sampling module, the brightness-aware attention module is used to connect the feature map extracted by the down-sampling layer and the feature map extracted by the up-sampling module through a skip connection, and assign a weight to the connected feature map; the calculation formula of the weight includes:

[0066] w l =σ(Conv 1×1 ([F enc ,F dec ]))⊙Sigmoid(I dark +∈);

[0067] wherein, I dark is a low-light area mask, σ is a Sigmoid activation function, Conv 1×1 is a 1×1 convolution layer, F enc and F dec are the feature map of the first down-sampling layer in the 5 down-sampling layers and the feature map of the first up-sampling layer in the 5 up-sampling layers respectively, and the first down-sampling layer and the first up-sampling layer correspond to each other; ∈ is a noise disturbance term.

[0068] Specifically, the LAA module realizes the weight promotion of the low-light area (the value of the low-light area mask is high) and enhances the dark details through feature concatenation and pixel-by-pixel weighting of brightness awareness; the weight attenuation of the highlight area (the value of the low-light area mask is low) and the suppression of overexposure. The U-Net generator realizes low-light enhancement through the brightness attention mechanism, and injects spatial attention through the skip connection to compensate for the information loss in the down-sampling process.

[0069] Specifically, the LAA-GAN model further includes a discriminator for distinguishing real images and generated enhanced images, helping the generator to produce more natural results. Illustratively, the discriminator adopts a multi-scale PatchGAN architecture, including 3 parallel branches (scales 64x64, 32x32, 16x16); each branch includes 4 convolutional layers (Conv2D-BN-LeakyReLU); and the output layer fuses the multi-scale discrimination results.

[0070] Illustratively, the detection network is a YOLOv8-Tiny model for an actual target detection task.

[0071] Preferably, the convolution kernel size in the encoder is 5x5, the convolution kernel size in the decoder is 3x3, and the number of attention heads in each LAA module is 4.

[0072] Preferably, the batch normalization momentum is 0.98. The batch normalization momentum refers to a parameter used to calculate the moving average and variance in the batch normalization layer, which controls the degree of influence of new batch statistics on cumulative statistics, is used to stabilize the dynamic adjustment of feature distribution in the training process, and prevents gradient explosion. The greater the value, the stronger the dependence on historical data.

[0073] Preferably, the learning rate decay period is 50 rounds.

[0074] Preferably, the low-illumination region mask I dark is generated by extracting a feature map of the electric meter image, binarizing a V channel in a YUV space of the feature map based on a preset threshold to obtain an initial mask, and superimposing Gaussian noise on the initial mask to obtain the low-illumination region mask I dark .

[0075] Preferably, the feature map is extracted by processing the electric meter image through three convolutional layers.

[0076] Preferably, the preset threshold is 0.1.

[0077] Preferably, the third down-sampling layer of the encoder and the third up-sampling layer of the decoder each include an adaptive Gaussian filtering layer and the depth separable convolutional layer, and a filtering kernel standard deviation dynamic adjustment formula of the adaptive Gaussian filtering layer includes:

[0078]

[0079] wherein σ is the filtering kernel standard deviation, μ(I dark ) is a mean value of an illumination distribution of the electric meter image, and max(I dark ) is a maximum value in a low-illumination mask region of the electric meter image; and an output calculation formula of the third down-sampling layer and the third up-sampling layer includes:

[0080] F out =α·F gauss +(1-α)·F conv ;

[0081] wherein, F out is the output of the third down-sampling layer or the third up-sampling layer, F gauss is the output of the adaptive Gaussian filtering layer, F conv is the output of the depth separable convolution layer, and a is the brightness attention map of the meter image; wherein, the pixel of the brightness attention map is used to indicate the brightness size of the pixel at the corresponding position in the feature map of the meter image, and the pixel of the brightness attention map is inversely proportional to the brightness of the corresponding pixel in the feature map of the meter image.

[0082] When the extreme dark area in the low-illumination mask is significantly different from the overall brightness, the value of σ is increased to 5 to enhance the smoothing ability of the Gaussian filter kernel and suppress noise; otherwise, σ tends to 1, which can preserve the texture details.

[0083] The brightness attention map is dynamically generated by the LAA module according to the illumination distribution of the current feature map, and is used to control the enhancement degree of different regions based on the illumination distribution of the feature map. The value range of the pixel of the brightness attention map is [0, 1]; specifically, the pixel corresponding to the extreme dark area of the feature map tends to 1, and the Gaussian filtering result is preferentially used to suppress noise; the pixel corresponding to the normal brightness area tends to 0, and the detail features extracted by convolution are preserved. The LAA module generates a brightness attention map containing a dynamic weight coefficient, which is used to fuse the outputs of the adaptive Gaussian filtering layer and the depth separable convolution layer.

[0084] In step 103, the computing device inputs the enhanced meter image into a preset detection network to obtain the meter position and the digital region in the enhanced meter image.

[0085] The detection network can detect the meter position and the digital region in the enhanced meter image through different mechanisms.

[0086] For example, the detection network can first perform feature extraction on the image through convolution layers and pooling layers, etc., to convert the image into a set of high-dimensional feature vectors, which contain the semantics and spatial information of various objects in the image; then, based on the extracted features, target detection is directly performed on the feature map, the class probability and position information of the target object are obtained by predicting each position on the feature map; and then, based on the class probability and position information, a bounding box is generated to determine the region where the target object is located.

[0087] Preferably, the loss function of the detection network includes:

[0088]

[0089] wherein, is a loss value, IoU is an intersection over union of a predicted bounding box and a real bounding box, the predicted bounding box is a bounding box corresponding to a predicted meter position, and the real bounding box is a corresponding bounding box of the meter position in the meter image; p 2 (b pred ,b gt ) represents the Euclidean distance between the center of the predicted bounding box and the center of the real bounding box, c represents the length of the diagonal of the smallest rectangle containing the predicted bounding box and the real bounding box, a is a hyperparameter, and v is a regression constraint representing a fixed aspect ratio of the meter; the calculation formula of the regression constraint v includes:

[0090]

[0091] wherein, w gt and h gt are the width and height of the real bounding box, respectively, and w red and h pred are the width and height of the predicted bounding box, respectively.

[0092] Preferably, the U-Net generator adopts INT8 symmetric quantization; and the detector head of the detection network adopts FP16 to reserve decimal coordinate accuracy.

[0093] Through INT8 symmetric quantization, data is converted from higher precision (such as FP32) to INT8, and model weights and activation values are represented by 8-bit integers, which can significantly reduce the memory space required for model storage and transmission. In calculation, the multiplication and addition operations of INT8 are more efficient than those of FP32, which can reduce the calculation amount and speed up the model inference speed. Especially on some devices with limited computing resources (such as mobile terminals and embedded devices), the running efficiency of the device can be improved, and the energy consumption can be reduced. The U-Net generator is usually used for image generation and other tasks, and the accuracy requirement is relatively not extremely high. INT8 quantization can maintain the performance of the model to a certain extent, and through quantization strategy and optimization, the generated results can be similar to the results generated by the original precision model in vision. Moreover, the data can be compressed and regularized to a certain extent during the quantization process, which helps to reduce the overfitting phenomenon of the model and improve the generalization ability of the model.

[0094] FP16 uses 16 bits to represent floating-point numbers, which can save half of the storage space and calculation amount compared with FP32. The detector head of the detection network is mainly responsible for the positioning and classification of targets, and has a higher requirement for coordinate accuracy. FP16 can retain a certain decimal accuracy, which can provide more accurate results for the coordinate prediction of the target, thereby improving the accuracy of target detection, reducing the positioning error, and avoiding problems such as inaccurate detection frame position caused by too low quantization accuracy.

[0095] For example, the original model includes a generator 983 MB + a detector 217 MB = 1.2 GB, and after quantization, the generator 312 MB (INT8) + the detector 68 MB (FP16) = 380 MB.

[0096] Preferably, the computing device can determine the dynamic range by counting the distribution of the activation values of each layer of the generator (KL divergence < 0.5): Qmin = μ-2σ, Qmax = μ+3σ, where Q refers to the quantization parameter, and the determination of the dynamic range refers to setting the quantization range according to the statistical distribution of the activation values (mean μ and standard deviation σ) to ensure the accuracy of the quantized model.

[0097] Through the mixed-precision quantization and dynamic range determination described above, the acceleration process can be quantized, and the edge deployment of the model can be optimized.

[0098] After obtaining the meter position and the digital area, the computing device can further identify the meter data in the digital area and upload the detection result through the Modbus protocol; when the meter data indicates that the meter is in an abnormal state, the computing device can send warning information to the manager or the maintenance personnel through the connected LED lamp.

[0099] Preferably, before the low-light enhancement of the meter image by the U-Net generator, the computing device can pre-train the U-Net generator for 500 epochs, the first learning rate for pre-training the U-Net generator is 2×10 -4 ; the parameters of the U-Net generator are maintained unchanged, and the detection network is trained for 300 epochs, the second learning rate for training the detection network is 1×10 -3 ; the U-Net generator and the detection network are jointly fine-tuned for 100 epochs, the third learning rate for the end-to-end joint fine-tuning is 5×10 -5 ; and the total loss function of the U-Net generator and the detection network is:

[0100]

[0101] wherein, the total loss is Ltotal = Ldet + Lper + Lstruct + Ladv, the adversarial loss of the U-Net generator is Ladv, the perceptual loss of the U-Net generator is Lper, the structural similarity loss of the U-Net generator is Lstruct, and the detection loss of the detection network is Ldet.

[0102] Specifically, the generator is used to generate images with higher photorealism, and the detection network is used to enhance the quality based on the feature similarity constraint. for optimizing image local structure, for optimizing target detection performance of the detection network.

[0103] Specifically, the calculation formula of the adversarial loss includes: wherein x is a real data sample, D() is a discriminator result, z is a random noise vector, and G(z) represents a fake data sample generated by the generator based on the random noise vector.

[0104] Specifically, the calculation formula of the perceptual loss includes: wherein x gt is a real data sample that the generator attempts to imitate and approximate, VGG 54 () represents the output features of the fourth convolutional layer in the fifth convolutional block in the VGG network, and ||·|| represents the square of the norm.

[0105] Specifically, the calculation formula of the structural similarity loss includes: wherein G and gt represent generated data samples and real data samples respectively, μ G and μ gt represent the pixel mean of the generated data samples and the real data samples respectively, and σ Ggt represent the pixel variance of the generated data samples and the real data samples respectively, σ t represents the covariance of the generated data samples and the real data samples, and c1 and c2 are constant terms.

[0106] Specifically, the calculation formula of the detection loss includes: wherein is a classification loss for solving the imbalance problem of positive and negative samples in the digital region of the electric meter; λ1 and λ2 are weight coefficients. The calculation formula of the classification loss includes:

[0107]

[0108] wherein p t is the predicted probability of the model for the sample belonging to the positive class, α t is a weight coefficient for balancing positive and negative samples, and γ is a regulation factor. Exemplarily, γ = 2 and α = 0.8.

[0109] Exemplarily, please refer to Figure 4 , Figure 4 a training process of the U-Net generator and the detection network in an embodiment is shown. As Figure 4As shown, the computing device first pre-trains the U-Net generator by the low-illumination ammeter image in the training phase 1, and then calculates the loss values of the above-mentioned adversarial loss, perceptual loss and structural loss based on the training results and the MSE loss function; and then adjusts the parameters of the U-Net generator and the parameters of the LAA module based on the loss values.

[0110] Then enter the training phase 2, the computing device freezes the parameters of the U-Net generator unchanged, and trains the YOLOv8-Tiny detection network based on the output result set of the U-Net generator in the training phase 1 and the joint loss function.

[0111] Then enter the training phase 3, the computing device jointly optimizes the U-Net generator and the detection network based on the labeled data set, the enhanced image output by the U-Net generator after dynamic Gaussian filtering as the input of the YOLOv8-Tiny detection network, and calculates the joint loss function based on the detection result of the detection network. Wherein the loss weight distribution ratio of the multi-task in the joint optimization process is: the joint loss function weight coefficient is 1.2, and the MSE loss function weight coefficient is 0.6.

[0112] In the joint optimization phase, the computing device can synchronize the parameters of the U-Net generator and the detection network once every 5 training rounds, and update the weights of the U-Net generator and the detection network.

[0113] The above describes the method provided by the embodiments of the present application, and the system provided by the embodiments of the present application will be described below.

[0114] The embodiments of the present application also provide a hardware acceleration scheme, which can be used for the deployment of the above-mentioned low-illumination low-voltage ammeter detection and positioning method based on LAA-GAN on hardware devices. The hardware acceleration scheme specifically includes:

[0115] 1. TensorRT optimization steps:

[0116] (1) Fuse Conv+BN+ReLU sequence: merge the continuous convolution (Conv), batch normalization (BN) and activation function ReLU operations into a single calculation node. The specific process is as follows: first, traverse the entire calculation graph, and identify the continuous operation mode of Conv-BN-ReLU; then extract the core parameters of the BN layer, including the scaling factor γ, the offset factor β and the mean μ; then through mathematical transformation, integrate these parameters into the weights and bias of the convolution kernel. Finally, delete the original BN and ReLU nodes, and only keep the Conv node after parameter integration. This operation effectively reduces the complexity of the calculation graph, reduces the number of memory accesses, and improves the inference efficiency.

[0117] (2) Input format optimization: For the original YUV420 format data collected by the camera (usually needs to be converted to RGB format before being processed by the model), targeted optimization is performed in TensorRT. The model input format is directly defined as YUV420, and the YUV to RGB conversion process is fused with the subsequent convolution operation through plugins or custom layers. This approach avoids additional external preprocessing steps and reduces performance loss caused by data conversion.

[0118] Tensor layout adjustment: Carefully check the transpose operation in the input preprocessing link. During the model building stage in TensorRT, directly specify the input tensor layout as NHWC or NCHW format according to the actual needs of the model, eliminate unnecessary transpose operations from the source, ensure that the storage and access mode of data in memory matches the model operation logic, and further improve the computing efficiency.

[0119] 2. Deep layer fusion optimization

[0120] In the LAA module, through innovative fusion strategy, the feature concatenation operation is deeply integrated with 1x1 convolution, and the Conv+BN+ReLU sequence is merged into an efficient single calculation node, greatly simplifying the calculation graph structure. In addition, the adaptive Gaussian filter is reconstructed as a separable convolution kernel, which can significantly reduce the computational complexity. Combined with FP16 mixed precision calculation and memory pooling technology, not only the inference speed is improved by 2.3 times, but also the peak memory usage is optimized to 413MB, effectively balancing performance and resource consumption.

[0121] 3. Multi-thread inference pipeline design

[0122] Adopting CPU and GPU cooperative parallel computing architecture, an efficient inference pipeline is constructed. Specifically, image acquisition is bound to CPUCore0 through an independent thread, and real-time acquisition of Camera raw image data is realized; in the preprocessing stage, CPUCore1 is used to perform parallel conversion from YUV space to RGB space and histogram equalization, and after data preprocessing, the data is immediately transmitted to the GPU pipeline.

[0123] Dual GPU stream parallel operation is enabled, where GPUStream1 executes the LAA-GAN enhancement algorithm to optimize image features, and GPUStream2 synchronously runs the YOLOv8 target detection model to realize multi-task efficient parallel processing.

[0124] 4. Low latency result feedback.

[0125] With the help of dual GPU stream technology, the end-to-end inference pipeline is deeply optimized, and the overall delay is controlled within 14ms, ensuring the real-time performance of the detection results and the system response efficiency, meeting the application requirements in high concurrency scenarios.

[0126] To better illustrate the model training part in the embodiments of the present application, a specific training scheme in the embodiments of the present application will be described below:

[0127] 1. Data set construction:

[0128] 5000 pairs of low-illumination and normal-illumination images are collected to construct a high-quality data set. At the same time, a diversified data enhancement strategy is used to improve the generalization ability of the model: through random illumination adjustment, the image brightness is dynamically transformed in the 0.1-0.6 times interval; the simulated fogging effect is introduced, the concentration is controlled in the 0.2-0.8 range, and different degrees of harsh environment are simulated; sensor noise is injected to simulate real shooting noise under ISO 1600-25600 sensitivity, and the diversity of data samples is fully enriched.

[0129] 2. Training environment configuration:

[0130] A high-performance cluster composed of 8 A100 GPUs is built to ensure efficient processing of large-scale data. The APEX toolkit is used to realize mixed precision training, which greatly improves the calculation efficiency while ensuring the training accuracy. The batch size per card is set to 32 samples to balance the calculation resource utilization rate and memory consumption, and to optimize the data throughput and model convergence speed in the training process.

[0131] 3. Loss function setting: λ adv = 1.0; λ perc = 0.8; λ ssim = 0.6; λ det = 1.2.

[0132] 4. Training result comparison:

[0133] For the same test set, the meter positioning and dial number recognition based on the method (LAA-GAN model) of the embodiments of the present application, Retinex-Net algorithm and EnlightenGAN algorithm are carried out, and the test data obtained are shown in the following table:

[0134]

[0135]

[0136] It can be seen that the method of the embodiments of the present application has great advantages in number recognition accuracy, positioning accuracy, single needle power consumption and memory usage.

[0137] Please refer to Figure 5 , Figure 5 a structure diagram of a low-voltage meter detection and positioning system in a low-light environment based on LAA-GAN provided by the embodiments of the present application, as Figure 5As shown, the system 50 includes:

[0138] An acquisition module 501 is used to acquire an image of an electric meter;

[0139] Enhancement module 502 is used to perform low-light enhancement on the electricity meter image using a U-Net generator. The encoder of the U-Net generator includes a 5-level downsampling layer, and the decoder of the U-Net generator includes a 5-level upsampling layer. The upsampling layer includes a brightness perception attention module and an upsampling module. The brightness perception attention module is used to connect the feature map extracted by the downsampling layer and the feature map extracted by the upsampling module through a skip connection, and assign a weight to the connected feature map; the calculation formula for assigning the weight includes:

[0140] w l =σ(Conv 1×1 ([F enc ,F dec ]))⊙Sigmoid(I dark +∈);

[0141] Among them, I dark is the low illumination area mask, σ is the Sigmoid activation function, Conv 1×1 is a 1×1 convolutional layer, F enc 、F dec are the feature maps of the first downsampling layer in the five-level downsampling layer and the feature maps of the first upsampling layer in the five-level upsampling layer, respectively, and the first downsampling layer corresponds to the first upsampling layer; ∈ is the noise perturbation term;

[0142] The detection module 503 is configured to input the enhanced meter image into a preset detection network to obtain the meter position and digital area in the enhanced meter image.

[0143] Preferably, the enhancement module 502 is specifically used to extract the feature map of the electric meter image; binarize the V channel in the YUV space of the feature map based on a preset threshold to obtain an initial mask; superimpose Gaussian noise on the initial mask to obtain the low illumination area mask I dark .

[0144] Preferably, the preset threshold is 0.1.

[0145] Preferably, the third downsampling layer of the encoder and the third upsampling layer of the decoder both include an adaptive Gaussian filter layer and the depthwise separable convolution layer, and the dynamic adjustment formula of the filter kernel standard deviation of the adaptive Gaussian filter layer includes:

[0146]

[0147] Among them, σ is the standard deviation of the filter kernel, μ(I dark) is the mean value of the illumination distribution of the meter image, max(I dark ) is the maximum value in the low-illumination mask area of the meter image; the output calculation formula of the third down-sampling layer and the third up-sampling layer includes:

[0148] F out = α·F gauss + (1-α)·F conv ;

[0149] wherein, F out is the output of the third down-sampling layer or the third up-sampling layer, F gauss is the output of the adaptive Gaussian filtering layer, F conv is the output of the depth separable convolution layer, and α is the brightness attention map of the meter image; wherein the pixels of the brightness attention map are used to indicate the brightness of the pixels at the corresponding positions in the feature map of the meter image, and the pixel value of the pixels in the brightness attention map is inversely proportional to the brightness of the corresponding pixels in the feature map of the meter image.

[0150] Preferably, the loss function of the detection network includes:

[0151]

[0152] wherein, L CIoU is the loss value, IoU is the intersection over union of the predicted bounding box and the real bounding box, the predicted bounding box is the bounding box corresponding to the predicted meter position, and the real bounding box is the corresponding bounding box of the meter position in the meter image; ρ 2 (b pred ,b gt ) represents the Euclidean distance between the centers of the predicted bounding box and the real bounding box, c represents the diagonal length of the smallest rectangle containing the predicted bounding box and the real bounding box, α is a hyperparameter, and v is a regression constraint representing the fixed aspect ratio of the meter; the calculation formula of the regression constraint v includes:

[0153]

[0154] wherein, w gt and h gt are the width and height of the real bounding box, respectively, and w pred and h pred are the width and height of the predicted bounding box, respectively.

[0155] Preferably, the U-Net generator adopts INT8 symmetric quantization; and the detector head of the detection network adopts FP16 decimal coordinate precision.

[0156] Preferably, the system 50 further includes a training module 504 for pre-training the U-Net generator 500 rounds, where the first learning rate of the pre-training U-Net generator is 2×10 -4 Keep the parameters of the U-Net generator unchanged, train the detection network for 300 rounds, and set the second learning rate of the detection network to 1×10 -3 The U-Net generator and the detection network are fine-tuned end-to-end for 100 rounds, and the third learning rate of the end-to-end joint fine-tuning is 5×10 -5 ; The total loss function of the U-Net generator and the detection network is:

[0157]

[0158] in, is the total loss, is the adversarial loss of the U-Net generator, is the perceptual loss of the U-Net generator, is the structural similarity loss of the U-Net generator, is the detection loss of the detection network.

[0159] The low-voltage electricity meter detection and positioning system in a low-light environment based on LAA-GAN provided in the embodiment of the present application can be understood by referring to the corresponding content of the aforementioned method embodiment part, and will not be repeated here.

[0160] like Figure 6 As shown, Figure 6 A possible logical structure diagram of a computing device provided in an embodiment of the present application. The computing device 60 includes: a processor 601, a communication interface 602, a memory 603, and a bus 604. The processor 601, the communication interface 602, and the memory 603 are interconnected via the bus 604. In the embodiment of the present application, the processor 601 is used to control and manage the actions of the computing device 60. For example, the processor 601 is used to execute Figure 1 The steps in the embodiments and / or other processes for the technology described herein. The communication interface 602 is used to support the computing device 60 to communicate. The memory 603 is used to store program codes and data of the computing device 60.

[0161] The processor 601 can be a central processing unit, a general purpose processor, a digital signal processor, an application specific integrated circuit, a field programmable gate array, or other programmable logic device, transistor logic, hardware components, or any combination thereof. It can implement or execute various example logical blocks, modules, and circuits described in connection with the disclosure. The processor can also be a combination of computing functionality, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, or the like. The bus 604 can be a peripheral component interconnect (PCI) bus, an extended industry standard architecture (EISA) bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, or the like. For ease of representation, Figure 6 Only one bus is shown in the figure, but there can be more than one bus, or types of buses.

[0162] In another embodiment of the present application, a computer readable storage medium is also provided, which includes instructions, when the instructions are run on a computer, cause the computer to execute the method described in the above embodiments. Figure 1 In another embodiment of the present application, a computer readable storage medium is also provided, which includes instructions, when the instructions are run on a computer, cause the computer to execute the method described in the above embodiments.

[0163] Those skilled in the art can appreciate that the units of each example described in connection with the embodiments disclosed herein can be realized in electronic hardware, computer software, or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components of each example have been described in general terms in the above description. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0164] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0165] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. For example, the described device embodiments are merely illustrative. For example, the division of the units is only a logical function division. There can be another division manner for the actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0166] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0167] In addition, each functional unit in the various embodiments of the present application can be integrated into a processing unit, or each unit can be a physically independent unit, or two or more units can be integrated into a unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0168] If the functions are realized in the form of software functional units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the essential part or all or part of the technical solutions that make contributions to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a read-only memory (ROM, read-only memory), a random access memory (RAM, random access memory), a mobile hard disk, a magnetic disk or an optical disk, and various program code storage media.

[0169] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application, and they should be covered in the scope of the claims and the description of the present application.

Claims

1. A low-voltage meter detection and positioning method in low-light environments based on brightness-aware attention LAA-generative adversarial network GAN, characterized by: include: Get the meter image; Performing low-light enhancement on the electricity meter image through a U-Net generator; The encoder of the U-Net generator includes a 5-level downsampling layer, the decoder of the U-Net generator includes a 5-level upsampling layer, the upsampling layer includes a brightness perception attention module and an upsampling module, the brightness perception attention module is used to connect the feature map extracted by the downsampling layer and the feature map extracted by the upsampling module through a skip connection, and assign weights to the connected feature maps; The calculation formula for allocating feature map weights to the brightness perception attention module includes: w l =σ(Conv 1×1 ([F enc ,F dec ]))⊙Sigmoid(I dark +∈); Among them, I dark is the low illumination area mask, σ is the Sigmoid activation function, Conv 1×1 is a 1×1 convolutional layer, F enc 、F dec are the feature maps of the first downsampling layer in the five-level downsampling layer and the feature maps of the first upsampling layer in the five-level upsampling layer, respectively. The level of the first downsampling layer corresponds to the level of the first upsampling layer; ∈ is the noise perturbation term; w l is the feature map weight; The enhanced electric meter image is input into a preset detection network to obtain the electric meter position and digital area in the enhanced electric meter image.

2. The method according to claim 1, characterized in that The low illumination area mask I dark The generation methods include: extracting a feature map of the electric meter image; Binarize the V channel in the YUV space of the feature map based on a preset threshold to obtain an initial mask; Gaussian noise is superimposed on the initial mask to obtain the low illumination area mask I dark .

3. The method according to claim 2, characterized in that The preset threshold is 0.

1.

4. The method according to any one of claims 1 to 3, characterized in that The third downsampling layer of the encoder and the third upsampling layer of the decoder both include an adaptive Gaussian filter layer and the depthwise separable convolution layer. The dynamic adjustment formula of the filter kernel standard deviation of the adaptive Gaussian filter layer includes: Among them, σ is the standard deviation of the filter kernel, μ(I dark ) is the mean value of the illumination distribution of the meter image, max(I dark ) is the maximum value in the low-light mask area of ​​the meter image; The output calculation formulas of the third downsampling layer and the third upsampling layer include: F out =α·F gauss +(1-α)·F conv ; Among them, F out is the output of the third downsampling layer or the third upsampling layer, F gauss is the output of the adaptive Gaussian filter layer, F conv is the output of the depthwise separable convolutional layer, α is the brightness attention map of the meter image; Among them, the pixels of the brightness attention map are used to indicate the brightness of the pixels at the corresponding positions in the feature map of the electric meter image, and the pixel values ​​of the pixels in the brightness attention map are inversely proportional to the brightness of the corresponding pixels in the feature map of the electric meter image.

5. The method according to any one of claims 1 to 3, characterized in that The loss function of the detection network includes: in, is the loss value, IoU is the intersection over union (IoU) of the predicted bounding box and the true bounding box, the predicted bounding box is the bounding box corresponding to the predicted meter position, and the true bounding box is the bounding box corresponding to the meter position in the meter image; ρ 2 (b pred ,b gt ) represents the Euclidean distance between the center of the predicted bounding box and the center of the true bounding box, c represents the diagonal length of the smallest rectangle containing the predicted bounding box and the true bounding box, α is a hyperparameter, and v is a regression constraint indicating that the aspect ratio of the meter is fixed; The calculation formula of the regression constraint v includes: Among them, w gt and hgt are the width and height of the ground-truth bounding box, respectively, w pred and hpred are the width and height of the predicted bounding box, respectively.

6. The method according to any one of claims 1 to 3, characterized in that The U-Net generator uses INT8 symmetric quantization; the detector head of the detection network uses FP16 to retain decimal coordinate accuracy.

7. The method according to any one of claims 1 to 3, characterized in that Before performing low-light enhancement on the electricity meter image by using the U-Net generator, the method further includes: The U-Net generator is pre-trained for 500 rounds, and the first learning rate of the U-Net generator is 2×10 -4 ; Keeping the parameters of the U-Net generator unchanged, the detection network was trained for 300 rounds, and the second learning rate of the detection network was 1×10 -3 ; The U-Net generator and the detection network are fine-tuned end-to-end for 100 rounds, and the third learning rate of the end-to-end joint fine-tuning is 5×10 -5 ; The total loss function of the U-Net generator and the detection network is: in, is the total loss, is the adversarial loss of the U-Net generator, is the perceptual loss of the U-Net generator, is the structural similarity loss of the U-Net generator, is the detection loss of the detection network.

8. A low-voltage meter detection and positioning system in low-light environments based on brightness-aware attention LAA-generative adversarial network GAN, characterized by: The method according to any one of claims 1 to 7, wherein the system comprises: An acquisition module, used for acquiring an electric meter image; An enhancement module is configured to perform low-light enhancement on the electricity meter image using a U-Net generator; the encoder of the U-Net generator includes a 5-level downsampling layer, the decoder of the U-Net generator includes a 5-level upsampling layer, the upsampling layer includes a brightness perception attention module and an upsampling module, the brightness perception attention module is configured to connect the feature map extracted by the downsampling layer and the feature map extracted by the upsampling module through a skip connection, and assign weights to the connected feature maps; the calculation formula for assigning feature map weights by the brightness perception attention module includes: w l =σ(Conv 1×1 ([F enc ,F dec ]))⊙Sigmoid(I dark +∈); Among them, I dark is the low illumination area mask, σ is the Sigmoid activation function, Conv 1×1 is a 1×1 convolutional layer, F enc 、F dec are the feature maps of the first downsampling layer in the five-level downsampling layer and the feature maps of the first upsampling layer in the five-level upsampling layer, respectively, the first downsampling layer corresponds to the first upsampling layer; ∈ is the noise perturbation term; The detection module is used to input the enhanced meter image into a preset detection network to obtain the meter position and digital area in the enhanced meter image.

9. A computing device, characterized in that include: Memory, used to store programs; A processor, configured to load the program to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 7.

Images