Electric meter bottom identification method and system

Abstract

This application relates to the field of automatic meter reading technology, specifically providing a method and system for identifying the pre-meter reading. The method includes: acquiring first and second image data of multiple meters in a calibration chamber, forming image fusion data through dual-channel stitching operations, and processing to generate an enhanced multi-scale feature map; performing deformation detection and adjustment on the enhanced multi-scale feature map to obtain a corrected multi-scale feature map adapted to the dense arrangement of multiple meters; segmenting the corrected multi-scale feature map to obtain a small-scale distribution probability map containing multiple channels; using an adaptive upsampling algorithm to optimize the image region edges corresponding to each target element in the small-scale classification probability map to obtain a high-definition segmentation probability map; converting it into an initial binary mask to determine the region coordinate information containing five target elements; and finally using a dual-channel OCR fusion algorithm to identify the region coordinate information to obtain a readable string corresponding to the target element.

Description

Technical Field

[0001] This application relates to the field of automatic meter reading technology, and in particular to a method and system for identifying the reading of an electricity meter. Background Technology

[0002] The initial reading of a smart meter refers to the value recorded when the meter is installed or activated. This initial reading serves as the benchmark for subsequent electricity consumption calculations, and its core function is to determine the starting point of the user's actual electricity consumption, preventing the inclusion of pre-installation data in billing. The verification process for electricity meters requires reading the meter barcode, total reading, peak, valley, and average initial reading data, and taking photos to generate a verification report. However, currently, the method for identifying and reading the initial reading of the meter display during verification still relies on manual operation, involving switching the meter screen, reading, recording, photographing, and compiling the report for submission. This entire process consumes a significant amount of manpower.

[0003] Existing automatic meter reading identification technologies mainly rely on a single type of image input (such as color or black and white) and use traditional OCR technology for processing. However, these methods have significant limitations when dealing with the dense simultaneous processing of multiple meters on the verification pipeline. Their processing efficiency is low and they are extremely sensitive to interference such as uneven lighting, screen reflection, character tilt, and broken numbers. Under complex verification conditions, the accuracy of single meter recognition fluctuates drastically. Summary of the Invention

[0004] To address the aforementioned issues, this application provides a method and system for identifying the low accuracy of electricity meters. By performing synchronous positioning and adaptive identification and segmentation of multiple electricity meters densely packed within the calibration chamber, high-precision multi-element identification of multiple electricity meters can be achieved.

[0005] To achieve the objectives of this application, the following technical solution is provided: In a first aspect, this application provides a method for identifying the reading of an electricity meter, including: Acquire first and second image data of multiple meters in the calibration chamber; The first image data and the second image data are combined through a dual-channel concatenation operation to form image fusion data. Multi-scale feature extraction and fusion processing are then performed on the image fusion data to generate an enhanced multi-scale feature map. The enhanced multi-scale feature map is deformed and adjusted based on the deployed deformable convolutional layer to obtain a corrected multi-scale feature map that adapts to the dense arrangement of the multiple meters. The corrected multi-scale feature map is segmented based on the MobileNetV3 backbone network to obtain a small-scale distribution probability map containing multiple channels; wherein, each channel corresponds to a type of target element, and the target element includes barcode, total degree, spike, peak, and valley. An adaptive upsampling algorithm is used to optimize the image region edges corresponding to each target element in the small-scale classification probability map to obtain a high-resolution segmentation probability map; The high-resolution segmentation probability map is converted into an initial binary mask. After morphological optimization of the initial binary mask, connected component filtering and orientation correction are performed to obtain the region coordinate information containing the five target elements. A dual-channel OCR fusion algorithm is used to identify the coordinate information of the region to obtain a readable string corresponding to the target element.

[0006] A further improvement of this application is that the acquisition of first image data and second image data of multiple meters in the calibration chamber includes: acquiring a color image and a black-and-white image of the multiple meters in the calibration chamber; preprocessing the color image to obtain the first image data; preprocessing the black-and-white image using histogram equalization to obtain the second image data; the preprocessing of the color image to obtain the first image data includes: performing adaptive grayscale processing on the color image to obtain first sub-image data; wherein, the adaptive grayscale weight function is defined as follows: ; ; In the formula, For the adaptive grayscale weight of the red channel, The standard deviation of the pixel values ​​in the red channel. The standard deviation of pixel values ​​in the green channel. The standard deviation of the pixel values ​​in the blue channel. For the adaptive grayscale weights of the green channel, For the adaptive grayscale weight of the blue channel, Coordinates in the final grayscale image The grayscale value of the pixel at that location. This is the red channel pixel value of that pixel. This is the green channel pixel value for that pixel. The first image data is obtained by using the blue channel pixel value of the pixel and performing contrast enhancement on the first sub-image data using a piecewise adaptive gamma correction model.

[0007] A further improvement of this application is that the step of performing multi-scale feature extraction and fusion processing on the image fusion data to generate an enhanced multi-scale feature map includes: inputting the image fusion data into a lightweight CSPDarknet-tiny backbone network for feature hierarchical abstraction, specifically: first through... The convolutional layer compresses the number of channels in the fused image data to 64, and then sequentially processes it through layer-by-layer processing with downsampling rates of 2, 4, and 8, outputting three levels of original multi-scale feature maps. The first-level feature map has a size of 960×540×128 and is used to capture edge and texture features; the second-level feature map has a size of 480×270×256 and is used to extract character stroke structure features; the third-level feature map has a size of 240×135×512 and is used to focus on the overall outline and spatial layout features of the meter. Using these three levels of original multi-scale feature maps as input, the FPN feature enhancement algorithm is used for bidirectional feature pyramid fusion processing to obtain three sets of enhanced feature maps. The constructed bidirectional feature pyramid multi-scale fusion expression is as follows: ; In the formula, For the first The enhanced multi-scale feature map obtained after the first-level feature map is fused by the FPN feature enhancement algorithm includes... , , Three sets of enhanced feature maps; For deconvolution operation, For convolution operations, For the first Level 1 original multi-scale feature map, This is a feature fusion operation that adds elements one by one. For the first Level original multi-scale feature map.

[0008] A further improvement in this application is that the deformation detection and adjustment of the enhanced multi-scale feature map based on the deployed deformable convolutional layer to obtain a corrected multi-scale feature map adapted to the dense arrangement of multiple electricity meters includes: detecting the boundary spacing between adjacent electricity meters in the electricity meter image region corresponding to the enhanced multi-scale feature map based on the deformable convolutional layer to obtain the detected value of the boundary spacing between adjacent electricity meters; wherein, a deformable convolutional layer DCNv2 is deployed on the enhanced multi-scale feature map, and the kernel size of the deformable convolutional layer DCNv2 is... The initial offset range is set to ±5 pixels; the detected distance between adjacent electricity meter boundaries is compared with a preset threshold; if the detected distance between adjacent electricity meter boundaries is less than the preset threshold, a dynamic offset expansion mechanism is activated, and an expansion coefficient is calculated to expand the upper limit of the sampling point offset to ±8 pixels; if the detected distance between adjacent electricity meter boundaries is greater than or equal to the preset threshold, the initial offset range is maintained; based on the adjusted sampling point parameters, convolution operations are performed using a deformable convolution core formula to obtain a corrected multi-scale feature map adapted to the dense arrangement of multiple electricity meters; specifically: ; In the formula, To output the position coordinates, For the first Each sampling point is offset from the center of the standard convolution kernel. For learning space offset, This represents the upper limit of the number of sampling points in the convolution kernel. For the first The convolution weight parameters corresponding to each sampling point The first in the convolution kernel The modulation scalar corresponding to each sampling point, where... The sigmoid function is used to constrain the interval to [0,1]. The input feature map is defined by the parameters within the parentheses, which represent the spatial locations of the sampling points within the input feature map. The output position coordinates in the output feature map are The pixel feature values; where a deformation constraint function is introduced during training: ; In the formula, The offset constraint cost function value. It is the set of offsets for all sampling points within the convolution kernel. This is the index of the sampling points within the convolution kernel. This represents the upper limit of the number of sampling points in the convolution kernel. For the first The L2 norm of the offset of each sampling point ,in, This represents the horizontal offset component. This represents the offset component in the vertical direction.

[0009] A further improvement in this application is that, during the model training process, the loss function is calculated as follows: ; ; ; In the formula, To jointly optimize the loss value, For bounding box regression loss, This is the ratio of the intersection area to the union area of ​​the predicted bounding box and the actual bounding box of the electricity meter. Predict the bounding box for the electricity meter. This is the actual annotation of the meter's bounding box. This is the balancing coefficient for the bounding box regression loss, used to balance the contributions of the center distance term and the aspect ratio term. The normalization factor for the center distance term, A parameter used to measure the consistency of the aspect ratio between the predicted bounding box and the ground truth bounding box. These are the weighting coefficients for the bounding box regression loss. To optimize the classification loss for class imbalance, For category weight coefficients, This represents the probability value predicted by the model that a sample belongs to the target category. Let be the modulation coefficient, where For focusing coefficient, ; Represents the natural logarithm operation. These are the weighting coefficients for the classification loss. The L2 norm of the offset of the deformable convolution sampling points. These are the weighting coefficients for the deformable convolution offset constraint loss.

[0010] A further improvement of this application is that the step of segmenting the corrected multi-scale feature map based on the MobileNetV3 backbone network to obtain a small-scale probability distribution map containing multiple channels includes: normalizing the image pixel values ​​of the corrected multi-scale feature map to... The interval is then uniformly scaled using bicubic interpolation. The preprocessed image is obtained by analyzing pixels. The preprocessed image is then input into the MobileNetV3 backbone network, passing through three depthwise separable convolutional modules to extract local feature maps containing character strokes and barcode patterns. These local feature maps are then input into the MobileViT module, where they are first enlarged by 3×3 convolutions and divided into patch units. A Transformer encoder is then used to model global context features, outputting global context-coded features containing cross-regional correlation information. The CNN local features and the global context-coded features are concatenated channel by channel to generate a fused feature map. Channel processing is then performed on the fused feature map. Dimensionality reduction yields a simplified feature map; then, spatial context features of the simplified feature map are extracted using a 3×3 convolutional layer, followed by a sigmoid activation function to generate a spatial attention weight map in the 0-1 range; the spatial attention weight map and the simplified feature map are multiplied element-wise to obtain an effective feature map with spatial attention weighting; based on the effective feature map, 3×3 and 5×5 depthwise separable convolutions and dilated convolutions with a dilation rate of 3 are used in parallel to capture features at different scales, and after channel concatenation and shuffling, 1×1 convolutions are used to map the channels to 5 (each channel corresponds to a class of data elements (barcode, total degree, peak, valley, flat)) to obtain a small-scale classification probability map; specifically: ; ; In the formula, This is the feature map after multi-scale convolution fusion. This represents a depth-separable convolution operation of size 3×3. This represents a depth-separable convolution operation with a size of 5×5. This represents a depthwise separable dilated convolution operation with a dilation rate of 3. This indicates a channel splicing operation. The effective feature map after spatial attention weighting; This is a small-scale classification probability map. This represents the Softmax activation function. This represents a channel transform convolution operation with a 1×1 size convolution kernel. For the reason Multiscale fused feature map obtained through channel washing operation.

[0011] A further improvement in this application is that the step of using an adaptive upsampling algorithm to optimize the image region edges corresponding to each target element in the small-scale classification probability map to obtain a high-resolution segmentation probability map includes: using an adaptive upsampling algorithm to reconstruct a high-resolution segmentation structure map based on the small-scale classification probability map and learnable weights; specifically: ; In the formula, The coordinates in the final generated high-definition segmentation structure diagram are Pixel values; The adaptive weighting coefficients are relative to the coordinates within the sampling window. The corresponding learnable weights are as follows: for key areas such as the edges of digit strokes, the weight values ​​are higher to strengthen edge features and avoid breakage, while for background areas, the weight values ​​are lower to suppress noise interference. For all within the adaptive sampling window Summation is performed on index points; This is a small-scale classification probability map; Let be the pixel value at the corresponding position in the small-scale probability map, where This is a floor function used to round down the coordinates of the high-resolution segmented structure image. Mapped to the corresponding baseline position in the small-scale probability map; and This indicates that the relative index of the sampling window is used in conjunction with the reference position. Determine the specific pixel positions in the small-scale probability map that participate in the weighted calculation.

[0012] A further improvement of this application is that the step of converting the high-definition segmentation probability map into an initial binary mask includes: for each channel, calculating the local probability mean and standard deviation in the neighborhood of each pixel based on a 25×25 window to generate a dynamic binarization threshold, and then obtaining an initial binary mask through binarization. After morphological optimization of the initial binary mask, connected component filtering and orientation correction are performed to obtain region coordinate information containing the five target elements. This includes: performing closing and opening operations on the initial binary mask, extracting all connected components, eliminating abnormal regions, and determining candidate regions that conform to the shape of the meter characters; calculating the minimum bounding rectangle of the candidate regions; and, if the tilt angle of the minimum bounding rectangle is greater than a preset angle, performing rotation correction using bilinear interpolation to align the character axis with the image coordinate axis, thereby obtaining region coordinate information containing the five target elements.

[0013] A further improvement in this application is that the method of using a dual-channel OCR fusion algorithm to identify the region coordinate information and obtain a readable string corresponding to the target element includes: aligning the region coordinate information with the original preprocessed image through coordinate mapping, and obtaining the element region image through cropping; performing recognition based on the PP-OCRv3 backbone network in the main channel, and conducting font adversarial training to obtain the main channel recognition result; wherein, a direction-aware module is embedded in the PP-OCRv3 backbone network, first calculating the gradient histogram features of the element region image, concatenating the gradient histogram features and CNN local features through channel concatenation, and then calculating the direction-adaptive convolution kernel weights through convolution operations and the Softmax activation function to improve the slanted character recognition rate; specifically: ; In the formula, For orientation-adaptive convolutional kernel weights, This represents the Softmax activation function. This represents the convolution operation. The gradient histogram features of the image of the element region are... The CNN local features extracted by the PP-OCRv3 backbone network Indicates a channel splicing operation; In the auxiliary channel, a character region enhancement model based on ViT is used for recognition to obtain the recognition results for the auxiliary channel; specifically: ; In the formula, To assist in text recognition via the auxiliary channel, To assist in channel confidence, Indicates based on Character region enhancement model, It is a 12-layer Transformer structure with a patch size of 4×4; For the element region image, This is a spatial attention map. The feature channel splicing operation is represented; the main channel recognition result and the auxiliary channel recognition result are aligned at the character level using a dynamic time warping algorithm to obtain the aligned main channel recognition result and auxiliary channel recognition result; the aligned main channel recognition result and auxiliary channel recognition result are dynamically weighted and fused to generate a readable string corresponding to the target element.

[0014] Secondly, this application provides an electricity meter reading identification system for implementing the above-mentioned electricity meter reading identification method, comprising: The acquisition module is used to acquire the first and second image data of multiple meters in the calibration chamber; The fusion module is used to combine the first image data and the second image data through a dual-channel stitching operation to form image fusion data, and to perform multi-scale feature extraction and fusion processing on the image fusion data to generate an enhanced multi-scale feature map. An adaptation module is used to perform deformation detection and adjustment on the enhanced multi-scale feature map based on the deployed deformable convolutional layer to obtain a corrected multi-scale feature map that adapts to the dense arrangement of the multiple meters. The segmentation module is used to segment the corrected multi-scale feature map based on the MobileNetV3 backbone network to obtain a small-scale probability distribution map containing multiple channels; wherein, each channel corresponds to a type of target element, and the target element includes barcode, total degree, spike, peak, and valley. The optimization module is used to optimize the image region edges corresponding to each target element in the small-scale classification probability map using an adaptive upsampling algorithm to obtain a high-resolution segmentation probability map. The coordinate generation module is used to convert the high-definition segmentation probability map into an initial binary mask, and after morphological optimization of the initial binary mask, to perform connected component filtering and orientation correction to obtain region coordinate information containing the five target elements. The recognition module is used to recognize the coordinate information of the region using a dual-channel OCR fusion algorithm to obtain a readable string corresponding to the target element.

[0015] Compared with the prior art, this application has the following beneficial effects: The meter bottom recognition method and system provided in this application introduce deformable convolutional layers to perform deformation detection and spatial adaptive correction on multi-scale feature maps, accurately compensating for perspective distortion and local distortion caused by viewing angle differences in densely arranged meters, thus ensuring physical consistency of subsequent segmentation results. Secondly, MobileNetV3 is used as a lightweight backbone to output a small-scale distribution probability map containing multiple channels. Then, a high-definition segmentation probability map is reconstructed through an adaptive upsampling algorithm and transformed into regional coordinate information containing five target elements to directly drive dual-channel OCR to complete highly robust text recognition. Thus, this application constructs a closed-loop end-to-end system of dual-channel image fusion, deformable feature correction, MobileNetV3 multi-channel segmentation, adaptive upsampling, dynamic threshold + geometric correction, and dual-channel OCR, achieving high-precision multi-element recognition for multi-meter verification under low computational complexity. Attached Figure Description

[0016] The accompanying drawings are provided to further understand this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. Figure 1 A schematic diagram of an optional first process for the meter reading identification method provided in the embodiments of this application; Figure 2 A schematic diagram of an optional second process for the meter reading identification method provided in the embodiments of this application; Figure 3 A schematic diagram of an optional third process for the meter reading identification method provided in the embodiments of this application; Figure 4 A schematic diagram of the lightweight CSPDarknet-tiny backbone network provided in the embodiments of this application; Figure 5 This is a schematic diagram of the structure of the MobileNetV3 backbone network provided in an embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature; in the description of this application, unless otherwise stated, "multiple" means two or more.

[0019] Significant progress has been made in the automated verification technology of electricity meters. Traditional electricity meter verification mainly relies on manual operation, which is inefficient and prone to errors. Automated verification systems, on the other hand, combine automatic control, information technology, and metrological verification practices to achieve large-scale, fully automated operation of measuring instrument verification and testing, thereby greatly reducing resource input in terms of personnel, equipment, site, and verification costs.

[0020] The initial reading (also known as the "base value" or "initial reading") of a smart meter refers to the initial value recorded when the meter is installed or activated, serving as the benchmark for subsequent electricity consumption calculations. Its core function is to determine the starting point of a user's actual electricity consumption, preventing the inclusion of pre-installation data in billing. Current verification processes require reading the meter barcode, total reading, peak, valley, and flat reading data, and taking photos to generate a verification report. However, there is currently a lack of fully automated and mature technical solutions for identifying and reading the meter's displayed base reading during verification. The process still relies on manual operation for switching meter displays, reading, recording, photographing, and compiling reports. This entire process consumes significant manpower.

[0021] Existing electricity meter reading recognition technologies primarily rely on single-type image input (such as color or black and white) and use traditional Optical Character Recognition (OCR) technology for processing. Newer solutions utilize CNN (Convolutional Neural Network) algorithms to analyze and identify key meter reading information, further improving the efficiency and reliability of manual recognition. However, these methods have significant limitations when dealing with simultaneous processing of multiple meters on a verification pipeline. For example, the number of meters processed and recognized in a single run is relatively small, making it unsuitable for parallel multi-meter scenarios on a pipeline, resulting in low processing efficiency. Environmental adaptability is poor; character recognition accuracy drops sharply to 68% (traditional OCR methods) when facing complex backgrounds and changing lighting conditions. Especially when the edge meter variation rate reaches 15%, the traditional perspective correction algorithm has a high probability of failure. Furthermore, considering the emergence of numerous new meter products from existing meter manufacturers, the number of compatible fonts for traditional methods is insufficient to meet practical needs (the variety of font types and features increases the maintenance cost of the required template library). Additionally, the numerous preprocessing steps required by traditional methods consume significant computing power, reduce efficiency, and fail to meet real-time requirements.

[0022] To address the aforementioned technical problems, this application proposes the following technical solutions and corresponding embodiments.

[0023] The following is combined Figures 1 to 5 The embodiments shown illustrate the technical solutions of this application: Example 1 This application provides an embodiment of a method for identifying the reading of an electricity meter, referring to... Figure 1 As shown, the content includes the following steps S101 to S107: Step S101: Obtain the first image data and the second image data of multiple meters in the calibration chamber.

[0024] In this embodiment, a dual-channel optical module is used to photograph multiple electricity meters in the calibration chamber, acquiring one color image and one black mark image for each meter. The color image is preprocessed to obtain first image data; simultaneously, the black mark image is preprocessed to obtain second image data. In this embodiment, refer to... Figure 2 As shown, the dual-channel optical module (dual-spectrum image acquisition) includes a color camera, a monochrome camera, and a synchronization trigger module. The color camera preferably uses a 20-megapixel CMOS (Complementary Metal-Oxide-Semiconductor) sensor to acquire RGB three-channel images; the monochrome camera preferably uses a 5-megapixel global shutter sensor to acquire single-channel grayscale images. The synchronization trigger module employs FPGA (Field Programmable Gate Array) hardware-level timing control to ensure that the exposure time deviation between the two cameras is ≤0.5ms. For example, the resolution of the images acquired by the dual-channel optical module is set to 1920×1080, and the frame rate is 30fps.

[0025] As a feasible implementation method, an adaptive grayscale combined with contrast enhancement method is used to preprocess the color image, wherein the weight function for adaptive grayscale is defined as follows: ; ; In the formula, For the adaptive grayscale weight of the red channel, The standard deviation of the pixel values ​​in the red channel. The standard deviation of pixel values ​​in the green channel. The standard deviation of the pixel values ​​in the blue channel. For the adaptive grayscale weights of the green channel, The adaptive grayscale weight for the blue channel; Coordinates in the final grayscale image The grayscale value of the pixel at that location. This is the red channel pixel value of that pixel. This is the green channel pixel value for that pixel. This is the blue channel pixel value for that pixel. Therefore, by dynamically allocating weights to enhance the sharing of a certain color channel in the color graphic, the retention rate of meter reading information and edge information in the graphic is further improved.

[0026] To further improve image recognizability, this embodiment employs the following piecewise adaptive gamma correction model to enhance the contrast of color images, specifically: ; In the formula, The location in the image after piecewise adaptive gamma correction contrast enhancement is The enhanced grayscale value of the pixel. For the spatial location index of pixels in the image, For position is The original grayscale value of the pixel. For example, the grayscale threshold in a piecewise adaptive gamma correction model, The average gray level of the image; These are the gamma correction coefficients corresponding to the low grayscale range in the piecewise adaptive gamma correction model. The gamma correction coefficients for the high grayscale range in the piecewise adaptive gamma correction model are dynamically calculated using the following method: ; ; In the formula, For natural logarithm operations, This is the grayscale threshold in the piecewise adaptive gamma correction model.

[0027] This further enhances the differentiation between bright and dark areas in the color graphic, making it easier to extract key information during subsequent processing.

[0028] In this embodiment of the application, histogram equalization is used to preprocess the black and white image to obtain the second image data. In this embodiment, before photographing multiple meters in the calibration chamber, the meters to be tested are conveyed to the chamber via a conveyor belt. Inside the chamber, a positioning sensor, based on a photoelectric encoder, detects the meter's position in real time, and a clamping device performs three-axis attitude correction to ensure that the perpendicularity error between the meter's display screen and the camera's optical axis is ≤0.5°. The corrected positioning accuracy is fed back to the PLC controller by a laser rangefinder, ensuring repeatability. For example, the clamping device uses a pneumatic flexible clamp with a dynamically adjustable clamping force range of 5-20N.

[0029] Step S102: The first image data and the second image data are combined through a dual-channel stitching operation to form image fusion data, and the image fusion data is subjected to multi-scale feature extraction and fusion processing to generate an enhanced multi-scale feature map.

[0030] In the embodiments of this application, reference is made to Figure 3 As shown, the preprocessed first and second image data are fused together using a dual-channel stitching method to form image fusion data. This image fusion data then undergoes multi-scale feature layer extraction and multi-scale feature enhancement fusion to generate an enhanced multi-scale feature map, which serves as input for the next feature extraction step. A feature pyramid-based localization algorithm is then used to generate high-precision meter location data for densely packed multi-meter locations. Specifically, the preprocessed first and second image data are fused and processed in a dual-channel manner to output a multi-scale feature map, which undergoes feature value enhancement and dynamic edge optimization to improve feature accuracy. In this embodiment, the fusion of the first and second image data specifically involves: ; In the formula, The image fusion data (four-channel feature map) preserves complementary information of color features (such as shell color and local detail color) and grayscale texture (such as character edges and texture information). This is for channel splicing operations. This is the first image data. For complete preprocessed second image data, It refers to the dimension as The real tensor space.

[0031] In the embodiments of this application, reference is made to Figure 4As shown, a lightweight CSPDarknet-tiny network is used as the backbone network. Depthwise separable convolutions are used to perform hierarchical abstraction of the fused features, outputting multi-scale feature maps. This facilitates the subsequent extraction of low-level features such as edges and corners, and high-level semantic features such as the overall shape of the meter and text regions. In this embodiment, a three-level feature map is preferably output, with resolutions of 1 / 2, 1 / 4, and 1 / 8 of the input image, corresponding to 128, 256, and 512 channels, respectively. Weighted concatenation is used during feature fusion. The lightweight CSPDarknet-tiny network consists of three CSP (Cross Stage Partial) local modules. Each module contains depthwise separable convolutional layers, pointwise convolutions, and channel attention mechanisms. Each module contains two sets of depthwise separable convolutional kernels (DWConv3×3, stride 1), with a channel compression ratio set to 2:1. All activation functions use LeakyReLU, with a negative axis slope of 0.1. After the image fusion data is input into the backbone network, it is first processed... The convolutional layer is compressed to 64 channels, and then sequentially processed through layer-by-layer downsampling with rates of 2, 4, and 8 to output a multi-scale feature map (three-level original multi-scale feature map). Its multi-scale feature description is as follows: ; In the formula, This is the first-level feature map. This is the second-level feature map. This is the third-level feature map.

[0032] In this embodiment of the application, the first-level feature map The size is 960×540×128, capturing low-level features such as edges and textures; second-level feature maps Dimensions: 480×270×256. Extraction of intermediate semantic features from character stroke structure; third-level feature map. The dimensions are 240×135×512, focusing on the overall outline and high-level features of the meter's spatial layout. Furthermore, in this embodiment, each output stage utilizes a cross-stage feature reuse mechanism. Within each CSP module, 40% of the feature channels of the previous layer's input feature map are retained and directly transmitted to the current module's computation path via a direct connection. The previous layer features and the current layer features are then concatenated at a 4:1 ratio to reduce computational load while ensuring the integrity of multi-granularity information.

[0033] In this embodiment, the FPN feature enhancement algorithm is used to perform feature fusion processing on the multi-scale feature map to obtain an enhanced multi-scale feature map. Specifically, the multi-scale fusion expression of the constructed bidirectional feature pyramid is as follows: ; In the formula, For the first The enhanced multi-scale feature map is obtained by fusing the first-level feature map with the FPN feature enhancement algorithm, and the final output is... , , Three sets of feature maps; For deconvolution operation, For convolution operations, For the first Level 1 original multi-scale feature map, This is a feature fusion operation that adds elements one by one. For the first Level original multi-scale feature map.

[0034] In this way, through the fusion processing of feature enhancement algorithms, high-level semantic features (such as text information) and low-level detail features (edges) are mutually referenced and complemented, which improves the positioning accuracy of edge meters and overcomes the problem that small targets are easily missed. This enhances the detection capability of small targets at the edge of the meter and can improve the feature availability rate in scenarios where meters are densely arranged in the verification chamber.

[0035] Step S103: Based on the deployed deformable convolutional layer, perform deformation detection and adjustment on the enhanced multi-scale feature map to obtain a corrected multi-scale feature map that adapts to the dense arrangement of the multiple meters.

[0036] In this embodiment, based on obtaining sufficiently effective feature data, the dense arrangement of meters in the inspection chamber leads to overlap due to insufficient spacing between meters. This overlap affects the final meter positioning accuracy. Therefore, deformable convolution is used to model spatial deformation (overlap) and enhance the identification of meter boundaries in the image. Specifically, the convolution operation of the introduced deformable convolutional layer (DCNv2) for positioning is as follows: ; In the formula, To output the position coordinates, For the first Each sampling point is offset from the center of the standard convolution kernel. For learning space offset, This represents the upper limit of the number of sampling points in the convolution kernel. For the first The convolution weight parameters corresponding to each sampling point The first in the convolution kernel The modulation scalar corresponding to each sampling point is used to control the offset weights, where... The gradient is constrained to the [0,1] interval by the Sigmoid function to prevent gradient explosion. The input feature map is defined by the parameters within the parentheses, which represent the spatial locations of the sampling points within the input feature map. The output position coordinates in the output feature map are The pixel feature values.

[0037] In the embodiments of this application, in the feature map , and Deploy deformable convolutional layers DCNv2 on top, the kernel size of deformable convolutional layers DCNv2 is... The initial offset range is ±5 pixels, and the modulation scalar is... The initial value is 1.0, and the Adam optimizer is used during training with a learning rate of 1e-4 and an L2 regularization coefficient of 0.001. In this embodiment, deformable convolutional layers are preferentially deployed in the shallow layers of the network.

[0038] In this embodiment, for the meter stacking area, the distribution of convolution kernel sampling points is dynamically adjusted: when the detection value of the distance between adjacent meter boundaries is less than 15 pixels, the offset is adjusted. Automatically expands to ±8 pixels to enhance deformation modeling capabilities for overlapping areas. In other words, the dynamic offset expansion mechanism in this embodiment detects the distance between adjacent meter boundaries. Activated when the offset is less than 15 pixels, the upper limit of the offset is expanded to ±8 pixels, where the expansion factor is... Calculated dynamically using the following formula: ; In the formula, For expansion coefficient, This is the measured value of the distance between adjacent electricity meters at the boundary.

[0039] In this embodiment, a deformation constraint function is introduced. In the formula, The offset constraint cost function value. It is the set of offsets for all sampling points within the convolution kernel. This is the index of the sampling points within the convolution kernel. This represents the upper limit of the number of sampling points in the convolution kernel. For the first The L2 norm of the offset of each sampling point ,in, This represents the horizontal offset component. This refers to the vertical offset component. Therefore, by limiting feature distortion caused by excessive offset, the center deviation of the positioning box is ensured to be ≤2 pixels. In this embodiment, the L2 regularization intensity... Stable training was achieved using the Adam optimizer, resulting in a final bounding box center deviation of ≤1.2 pixels.

[0040] In this way, through convolution operations, the spacing between densely arranged meters can be dynamically adapted, reducing the impact of positioning errors caused by unconventional layouts such as meter skew and stacking.

[0041] After completing the feature extraction, multi-scale fusion, and deformation detection in steps S102 to S103, this embodiment defines a loss function to jointly optimize multiple actions. It optimizes bounding box regression for meter occlusion (robotic arms, etc.), improves center localization in dense scenes, and also addresses the imbalance problem between different samples (increasing loss weights for difficult samples, i.e., tilted or blurred meter images) and constrains features to prevent distortion caused by extreme deformation. Specifically, the loss function is defined as follows: ; ; ; In the formula, To jointly optimize the loss value, For bounding box regression loss, This is the ratio of the intersection area to the union area of ​​the predicted bounding box and the actual bounding box of the electricity meter. Predict the bounding box for the electricity meter. This is the actual annotation of the meter's bounding box. This is the balancing coefficient for the bounding box regression loss, used to balance the contributions of the center distance term and the aspect ratio term. The normalization factor for the center distance term, A parameter used to measure the consistency of the aspect ratio between the predicted bounding box and the ground truth bounding box. These are the weighting coefficients for the bounding box regression loss. To optimize the classification loss for class imbalance, For category weight coefficients, This represents the probability value predicted by the model that a sample belongs to the target category. Let be the modulation coefficient, where For focusing coefficient, ; Represents the natural logarithm operation. These are the weighting coefficients for the classification loss. The L2 norm of the offset of the deformable convolution sampling points. These are the weighting coefficients for the deformable convolution offset constraint loss.

[0042] Step S104: Segment the corrected multi-scale feature map based on the MobileNetV3 backbone network to obtain a small-scale distribution probability map containing multiple channels; wherein, each channel corresponds to a type of target element, and the target element includes barcode, total degree, spike, peak, and valley.

[0043] In this embodiment, five typical data elements of a smart meter—barcode, total meter reading, peak, peak, and valley / flat—are segmented. Considering that different elements are located in different regions, this embodiment uses MobileViT as the backbone network to extract features from the input corrected multi-scale feature map. The extracted features are then fed into the segmentation head for processing, generating a probability map of the segmentation pattern. An adaptive upsampling combined with dynamic threshold segmentation is used to form an adaptive binary mask that can handle different region contrast pairs (different data have different backgrounds and contrast levels), providing the foundation for the final step of generating the coordinate information of the five data element regions.

[0044] In this embodiment of the application, the corrected multi-scale feature map is first normalized to The interval is used to uniformly scale multiple fine-positioning image data to a uniform scale using bicubic interpolation. Pixels are used to eliminate the impact of scale differences on segmentation.

[0045] In the embodiments of this application, reference is made to Figure 5 As shown, local feature maps of multi-level CNN (Convolutional Neural Network) are extracted through the MobileNetV3 backbone network. The MobileNetV3 backbone network is configured with three depthwise separable convolutional modules (3×3 kernels, stride 2). Specifically, the finely localized image data is input into the initialized MobileNetV3 backbone network and processed sequentially by the three depthwise separable convolutional modules. The resulting local feature maps, with sizes of 128×128×32, 64×64×64, and 32×32×128 respectively, capture detailed features such as character strokes and barcode textures. The final local feature maps output by the MobileNetV3 backbone network are then input into the MobileViT module for global feature modeling. First, the final local feature maps are upscaled to 64 channels using a 3×3 convolution operation, specifically: ; In the formula, For the process The feature map obtained after convolutional upscaling; for Convolution operation with different kernel sizes This is the final local feature map output by the MobileNetV3 backbone network. Representative dimension is The real tensor space.

[0046] Specifically, after the final local feature map is upscaled to 64 channels by a 3×3 convolution operation, the upscaled feature map (32×32×64) is divided into blocks according to the 4×4 size rule, resulting in 16 patch units. Each patch unit corresponds to a local region on the feature map, providing the basic unit for the subsequent self-attention calculation of the Transformer encoder.

[0047] In the second-level feature extraction, the feature maps of the pre-divided patch units are input into the MobileViT module to initiate the global context feature modeling process. This involves introducing a Transformer encoder to perform secondary patch division on the original local feature maps (32×32×128) output from the MobileNetV3 backbone network, again following a 4×4 size rule, resulting in 64 patch units. Subsequently, a self-attention mechanism is used to calculate the association weights between different patch units, establishing cross-regional feature associations and outputting the final global context encoded features. Its dimensions are 32×32×256.

[0048] In this embodiment of the application, CNN local features are used. and global context encoding features The components are concatenated by channel and dynamically weighted using a spatial attention module, specifically: ; In the formula, This is the fused feature map obtained after channel stitching and spatial attention weighting; It is the Sigmoid activation function. For convolution operations, For CNN local features, Encoding features for global context, This indicates element-wise multiplication. Here, after obtaining the fused feature map, key feature channels are focused based on this map, such as the difference between numbers and barcodes, to enhance higher-value regional features and suppress reflective noise. Simultaneously, considering the correlation between the meter readings, this embodiment utilizes feature recombination and pixel classification to establish the correlation between the regions containing different elements, ultimately outputting pixel-level classification probability data. In this embodiment, for the fused feature map containing CNN local features and global context encoding features, a channel dimensionality reduction operation is performed through a 1×1 convolutional layer to compress the high-dimensional fused features from 128 channels to 32 channels, resulting in a simplified feature map. Then, using this simplified feature map as input, its spatial context features are first extracted through a 3×3 convolutional layer, and then a spatial attention weight map in the 0~1 interval is generated through a Sigmoid activation function. Subsequently, this spatial attention weight map is multiplied element-wise with the simplified feature map to obtain the effective feature map after spatial attention weighting. This spatial attention mechanism further focuses on key areas of the barcode, total degree, and peak / valley characters on the meter display, suppressing the feature response of background noise areas. Specifically: ; ; In the formula, This is a simplified feature map after channel compression. For channel-reduction convolution operations with a 1×1 size convolution kernel, To fuse feature maps; The effective feature map after spatial attention weighting retains only the strong response features of key areas such as characters and numbers on the meter screen, which are directly used as input for the downstream meter bottom recognition task. This indicates element-wise multiplication. This represents the Sigmoid activation function. This represents a convolution operation with a 3×3 kernel.

[0049] Based on the above, this embodiment continues to use a 1×1 convolutional layer to apply attention-weighted effective feature maps. The feature dimensions are mapped from 32 channels to 5 channels (each channel corresponding to barcode, total degree, spike, peak, and valley / flat categories), adapting the feature dimensions to the specific classification task. Finally, the feature values ​​of each pixel in the above 5 classification dimensions are normalized using the Softmax function, outputting a pixel-level classification probability map. The resolution of the probability map is increased to the original meter image size (256×256).

[0050] Specifically, this embodiment uses three types of convolutional kernels to capture features at different scales, distinguish different element regions, and then generate a small-scale classification probability map. This provides a classification basis for subsequent adaptive upsampling, and its convolution formula is as follows: ; ; In the formula, This is the feature map after multi-scale convolution fusion. This represents a depth-separable convolution operation of size 3×3. This represents a depth-separable convolution operation with a size of 5×5. This represents a depthwise separable dilated convolution operation with a dilation rate of 3. This indicates a channel splicing operation. The effective feature map after spatial attention weighting; This is a small-scale classification probability map. This represents the Softmax activation function. This represents a channel transform convolution operation with a 1×1 size convolution kernel. For the reason Multiscale fused feature map obtained through channel washing operation.

[0051] Step S105: An adaptive upsampling algorithm is used to optimize the image region edges corresponding to each target element in the small-scale classification probability map to obtain a high-resolution segmentation probability map.

[0052] Because elements such as total degree, barcode, peaks, valleys, etc., have different typical characteristics and include types such as Arabic numerals, stripes, logos, and manufacturer information, the edges of the image regions to which different elements belong need to be re-optimized and reconstructed based on the features and probability maps from the previous step to prepare for the next segmentation step. In this embodiment, an adaptive upsampling method is used to address the problem of digit breakage (the digit 8 is prone to breakage into 0) compared to the traditional interpolation method, and a final high-definition segmentation structure map is generated. The sampling algorithm formula is as follows: ; In the formula, The coordinates in the final generated high-definition segmentation structure diagram are Pixel values; The adaptive weighting coefficients are relative to the coordinates within the sampling window. The corresponding learnable weights are as follows: for key areas such as the edges of digit strokes, the weight values ​​are higher to strengthen edge features and avoid breakage, while for background areas, the weight values ​​are lower to suppress noise interference. For all within the adaptive sampling window Summation is performed on index points; This is a small-scale classification probability map; Let be the pixel value at the corresponding position in the small-scale probability map, where This is a floor function used to round down the coordinates of the high-resolution segmented structure image. Mapped to the corresponding baseline position in the small-scale probability map; and This indicates that the relative index of the sampling window is used in conjunction with the reference position. Determine the specific pixel positions in the small-scale probability map that participate in the weighted calculation.

[0053] Step S106: Convert the high-definition segmentation probability map into an initial binary mask, and after morphological optimization of the initial binary mask, perform connected component filtering and orientation correction to obtain region coordinate information containing the five target elements.

[0054] In this embodiment, a dynamic thresholding method is used to perform dynamic thresholding on the high-resolution segmented structure image, outputting a binary mask. After some simple morphological optimization (connected region area filter orientation correction), the final specific region coordinate information of the five elements is output. The specific steps are implemented through the following steps S1061 to S1065: Step S1061: Calculation of local statistics; In this embodiment of the application, the probability graph Slide a 25×25 window and calculate the mean value within the neighborhood of each pixel. with standard deviation The window step size is set to 5 pixels.

[0055] Step S1062: Determine the adaptive threshold; In this embodiment, the dynamic binarization threshold is generated according to the following formula: ; In the formula, For pixels in the image The corresponding dynamic binarization threshold, In pixels The local mean of all pixel values ​​within a 25×25 sliding window centered on the pixel. In pixels The local standard deviation of all pixel values ​​within a 25×25 sliding window centered on the pixel.

[0056] In the embodiments of this application, when When a region is identified as a valid region, an initial binary mask is generated. .

[0057] Step S1063: Perform a closing operation on the initial binary mask to fill the holes inside the characters, and then perform an opening operation to remove isolated noise points and smooth the region boundaries.

[0058] The closing operation uses a 3×3 elliptic kernel and iterates twice; the opening operation uses a 2×2 rectangular kernel.

[0059] Step S1064: Connected component filtering; In this embodiment, all connected components are extracted, and abnormal regions with an area A < 50 pixels or an aspect ratio R > 3.0 (corresponding to scratches or reflective interference) are removed, while candidate regions that conform to the shape of the meter characters are retained.

[0060] Step S1065, Orientation correction; In this embodiment of the application, the minimum bounding rectangle is calculated for the reserved area. If the rectangle's tilt angle is... Then, bilinear interpolation is used for rotation correction to ensure that the character axis is aligned with the image coordinate axis, thereby improving the OCR recognition accuracy.

[0061] In this way, the specific region coordinates of the final five elements are output, i.e., the segmented regions.

[0062] Step S107: Use a dual-channel OCR fusion algorithm to identify the coordinate information of the region and obtain a readable string corresponding to the target element.

[0063] In this embodiment, given the specific region segmentation coordinates of the five elements, a dual-channel OCR fusion algorithm is used to accurately identify the element characters and enhance compatibility with different fonts. Specifically, the specific content of the identified characters is achieved through the following: Step S1071: Character region alignment; In this embodiment, the segmented regions (region coordinate information) are aligned with the original preprocessed image through coordinate mapping, and the element region image is obtained through a cropping operation. Here, to further improve the alignment accuracy, sub-pixel level bilinear interpolation (interpolation coefficient 0.5) is performed to ensure that the coordinate mapping error between the cropped element region (ROI) and the original preprocessed image is ≤0.5 pixels, ensuring that the subsequent main and auxiliary channel OCR processes the same ROI region and eliminating recognition errors caused by coordinate deviation. Wherein: ; In the formula, For the element region image, The original preprocessed image, To divide the region, This is for image cropping operations.

[0064] Step S1072, Main channel identification (based on the PP-OCRv3 improved scheme); In this embodiment, a direction-aware module is embedded in the backbone of PP-OCRv3. First, the gradient histogram features of the element region image are calculated. These gradient histogram features are then concatenated with the local features of the CNN. Next, direction-adaptive convolution kernel weights are calculated using convolution operations and the Softmax activation function. The direction weights of the convolution kernels are dynamically adjusted to improve the recognition rate of tilted characters. Specifically: ; In the formula, For orientation-adaptive convolutional kernel weights, This represents the Softmax activation function. This represents the convolution operation. This refers to the histogram of oriented gradients (HOG) features of the element region image. These are CNN local features extracted from the PP-OCRv3 backbone network. This indicates a channel splicing operation.

[0065] In this embodiment, a 15-type meter font discriminator is introduced. The adversarial training loss function is constructed as follows: ; In the formula, To counteract the training loss value, This represents the mathematical expectation operation. Represents the natural logarithm operation. Indicates the font discriminator for 15 types of electricity meters ( The discriminant output function, Represents a generator function; The forged font samples serve as the input source for adversarial training. This determines the probability that a generated feature belongs to a specific font class for the discriminator. The generator (...) The adversarial game between the generator and the discriminator enables the generator to output font-independent robust features, improving compatibility with different meter fonts.

[0066] Element region images are processed using the improved PP-OCRv3 model. The main channel recognition text is obtained. and main channel confidence .

[0067] Step S1073, Auxiliary channel identification (based on Vision Transformer). In this embodiment of the application, the cropped element region image is... The code is divided into 4×4 pixel patch units (4096 in total). After embedding position encoding in each patch, it is input into a 12-layer Transformer encoder. In this embodiment, the Transformer's multi-head attention mechanism (number of heads = 8) applies a 2x attention weight to the character backbone region (the central 50% area) to emphasize the core features of the character and suppress edge noise interference. Simultaneously, when outputting the character recognition probability, temperature scaling (T = 0.8) is used for confidence calibration to reduce confidence error.

[0068] In this embodiment, the splicing features of the element region image and the spatial attention map are processed by a ViT-based character region enhancement model to obtain the text recognized by the auxiliary channel and the confidence score of the auxiliary channel; specifically: ; In the formula, To assist in text recognition via the auxiliary channel, To assist in channel confidence, Indicates based on Character region enhancement model, It is a 12-layer Transformer structure with a patch size of 4×4; For the element region image, This is a spatial attention map. This indicates a feature channel concatenation operation. In this way, by using dual-channel, relatively independent recognition patterns, the bias of a single model can be avoided.

[0069] In this embodiment, dynamic weighted fusion is performed on the results of the dual-channel algorithm processing. Specifically, after obtaining the identification results of the primary and secondary channels, a Dynamic Time Warping (DTW) algorithm is used to refine the identification results of the primary and secondary channels. , ; , Character-level alignment is performed to eliminate character offsets caused by previous segmentation errors, with a maximum alignment offset of 2 pixels. If the positional deviation between the two channels is within 2 pixels, the DTW algorithm successfully aligns them, and subsequent weighted fusion continues. If the positional deviation exceeds 2 pixels, constraints and re-checks are performed, at which point weighted fusion is stopped, and a manual review flag is triggered.

[0070] This embodiment, under the premise of passing the above 2-pixel constraint check, calculates the dynamic weight of the main channel using a high confidence level calibrated by temperature scaling, specifically as follows: ; In the formula, The dynamic weighting coefficients of the main channel identification results. This represents the Sigmoid logical function. The character-level average confidence score of the main channel recognition results. The character-level average confidence level of the auxiliary channel recognition results.

[0071] Here, the auxiliary channel weight is In this embodiment, the triggering condition is also set as follows: when the absolute value of the confidence difference between the primary and secondary channels is greater than 0.2, that is... At that time, the manual review flag is triggered.

[0072] In this embodiment of the application, the final recognition result is calculated according to a weighted formula during the synthesis of the fusion results, specifically as follows: ; In the formula, For the final synthesized recognition result, The text to be recognized in the main channel. To assist in the recognition of text in the auxiliary channel, The dynamic weighting coefficients of the main channel identification results.

[0073] As a possible implementation method, this embodiment addresses each character position. Calculate the main channel weight and auxiliary channel weights Specifically: ; ; In the formula, This represents the Sigmoid logical function. The character-level average confidence score of the main channel recognition results, with 0.5 as the confidence baseline value; Therefore, for each character position Calculate the merged character value: ; In the formula, , To prevent the coefficient from being divided by zero, The merged character values, i.e., character-level merging. Main channel number The results of character recognition. Secondary channel The results of character recognition. Main channel weight, For auxiliary channel weights.

[0074] In this embodiment, the synthesized string is validated using a finite state machine (FSM) to ensure the result conforms to the meter reading rules. Furthermore, this embodiment can also perform character-level fusion of the result. The input Conditional Random Field (CRF) model is combined with finite state constraints on the meter readings to optimize the rationality of the fusion results. Specifically: ; In the formula, It is a readable string containing five types of elements: barcode, total meter reading, peak, peak, and valley / flat. It serves as the final result for identifying the meter's reading. This represents a conditional random field model; For character-level fusion results, For the finite state constraints of the meter reading (e.g., the number can only be 0-9, etc.).

[0075] In this embodiment, after obtaining a readable string containing five elements—barcode, total meter readings, peak, valley, and flat—the system performs range verification, logical verification, and spatiotemporal verification on the obtained data. After verifying all five types of meter data (total meter readings, peak, valley, flat, and barcode), the system first automatically fills the data into a structured report according to the power industry standard template, embedding the timestamp of the original image, illumination parameters, and quality score. Subsequently, it generates tamper-proof hash values ​​for key data using the SHA-256 algorithm, attaches a digital signature, and then transmits the report to the provincial metering platform's main database via HTTPS protocol. Specifically, the range verification requires the total meter readings to be greater than or equal to 0 and less than or equal to 999999.99; the logical verification requires the peak / valley / flat data to meet the following conditions: The timestamp interval between adjacent workstations in the spatiotemporal verification must meet the following requirements. The spatial coordinate continuity error is ≤3mm.

[0076] The meter bottom recognition method and system provided in this embodiment, after acquiring first and second image data of multiple densely packed meters in the verification chamber, combines the first and second image data through dual-channel splicing to form image fusion data. Multi-scale feature extraction and fusion processing are then performed on this image fusion data to generate an enhanced multi-scale feature map. Subsequently, deformation detection and adjustment are performed on this enhanced multi-scale feature map based on a deployed deformable convolutional layer to obtain a corrected multi-scale feature map adapted to the dense arrangement of multiple meters. Finally, the corrected multi-scale feature map is segmented based on the MobileNetV3 backbone network to obtain a... A multi-channel small-scale probability distribution map is generated, where each channel corresponds to a type of target element, including barcode, total degree, spike, peak, and valley. An adaptive upsampling algorithm is then used to optimize the image region edges corresponding to each target element in the obtained small-scale classification probability map, resulting in a high-resolution segmentation probability map. This high-resolution segmentation probability map is then converted into an initial binary mask. After morphological optimization of this initial binary mask, connected component filtering and orientation correction are performed to obtain region coordinate information containing the five target elements. Finally, a dual-channel OCR fusion algorithm is used to recognize the region coordinate information, obtaining a readable string corresponding to the target element. This application introduces deformable convolutional layers to perform deformation detection and spatial adaptive correction on multi-scale feature maps, which can accurately compensate for perspective distortion and local distortion caused by viewing angle differences in densely arranged meters, ensuring physical consistency of subsequent segmentation results. Secondly, MobileNetV3 is used as a lightweight backbone to output a small-scale distribution probability map containing multiple channels. Then, a high-definition segmentation probability map is reconstructed through an adaptive upsampling algorithm and transformed into regional coordinate information containing five target elements to directly drive dual-channel OCR to complete highly robust text recognition. Thus, a complete closed-loop system is constructed, consisting of dual-channel image fusion, deformable feature correction, MobileNetV3 multi-channel segmentation, adaptive upsampling, dynamic thresholding + geometric correction, and dual-channel OCR, achieving high-precision multi-element recognition for multi-meter verification under low computational complexity.

[0077] Therefore, by simultaneously acquiring color and grayscale dual-modal images, an improved SIFT feature registration algorithm is used to achieve adaptive geometric deformation correction. A fusion feature generation module is constructed to enhance character region perception, and an innovative cross-device edge comparison optimization mechanism is introduced to improve the model's generalization ability. Its technical features are: 1) Overcoming the LCD screen mirror effect and complex background interference through dual-spectral image spatiotemporal alignment technology; 2) Combining a lightweight MobileNetV3 network with bidirectional LSTM to construct a multi-scale recognition architecture, achieving efficient recognition processing in embedded devices; 3) Pioneering a production line-level dynamic edge feature library, using online clustering algorithms to achieve a 1% false recognition rate. Compared to traditional OCR solutions, this method can effectively improve the recognition accuracy under complex lighting conditions and improve the compatibility of different meter models through an adaptive geometric correction module, effectively solving industry pain points such as low efficiency of manual verification and difficulty in adapting fonts from multiple manufacturers.

[0078] This application combines dual-spectral imaging characteristics to simultaneously capture the colorimetric features and texture details of the meter screen: the color image preserves interface identifiers and status indicator information, while the grayscale image enhances character edge structures. After spatiotemporal alignment, the two types of data are dynamically weighted and fused using a channel attention mechanism, effectively solving the feature loss problem caused by reflections and shadows. Simultaneously, a production line-level edge feature comparison library is introduced to compare the screen contour features of the current meter with historical samples in real time. When local anomalies such as scratches or stains are detected, the system automatically calls upon the baseline contour of similar meters for compensation and correction, effectively reducing the false detection rate.

[0079] Furthermore, this application is based on an online learning framework, in which the system automatically incorporates daily processed data into the training set, optimizes model parameters through incremental training, and continuously reduces operation and maintenance costs.

[0080] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the method in any of the embodiments of this application. Specifically, a system or apparatus equipped with a storage medium may be provided, on which software program code implementing the functions of any of the above embodiments is stored, and the computer (or CPU (Central Processing Unit) or MPU (Microprocessor Unit) of the system or apparatus may read and execute the program code stored in the storage medium.

[0081] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), it performs the functions defined in the system of this application.

[0082] It should be noted that the computer-readable storage medium shown in this invention can be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. For example, a computer-readable storage medium can be an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. The transmitted data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable storage medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF (Radio Frequency), etc., or any suitable combination thereof.

[0083] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0084] The units described in the embodiments of the present invention can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0085] It should be noted that although several modules or units of the device for performing actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0086] In the several embodiments provided in this application, it should be understood that the disclosed systems, modules, and methods can be implemented in other ways. For example, the module embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between modules or units, and may be electrical, mechanical, or other forms.

[0087] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. This application is not limited to the exact structures described above and illustrated in the accompanying drawings, and it should not be considered that the specific implementation of this application is limited to these descriptions. For those skilled in the art, various changes and modifications made without departing from the concept of this application should be considered to fall within the protection scope of this application.

Claims

1. A method for identifying the readings of an electricity meter, characterized in that, include: Acquire first and second image data of multiple meters in the calibration chamber; The first image data and the second image data are combined through a dual-channel concatenation operation to form image fusion data. Multi-scale feature extraction and fusion processing are then performed on the image fusion data to generate an enhanced multi-scale feature map. The enhanced multi-scale feature map is deformed and adjusted based on the deployed deformable convolutional layer to obtain a corrected multi-scale feature map that adapts to the dense arrangement of the multiple meters. The corrected multi-scale feature map is segmented based on the MobileNetV3 backbone network to obtain a small-scale distribution probability map containing multiple channels; wherein, each channel corresponds to a type of target element, and the target element includes barcode, total degree, spike, peak, and valley. An adaptive upsampling algorithm is used to optimize the image region edges corresponding to each target element in the small-scale classification probability map to obtain a high-resolution segmentation probability map; The high-definition segmentation probability map is converted into an initial binary mask. After morphological optimization of the initial binary mask, connected component filtering and orientation correction are performed to obtain the region coordinate information containing the five target elements. A dual-channel OCR fusion algorithm is used to identify the coordinate information of the region to obtain a readable string corresponding to the target element.

2. The method for identifying the meter reading according to claim 1, characterized in that, The acquisition of first and second image data of multiple meters in the calibration chamber includes: Collect one color image and one black-and-white image of multiple meters in the calibration chamber; The color image is preprocessed to obtain the first image data; the black and white image is preprocessed using histogram equalization to obtain the second image data. The preprocessing of the color image to obtain the first image data includes: The color image is subjected to adaptive grayscale processing to obtain the first sub-image data; wherein, the adaptive grayscale weight function is defined as follows: ； ； In the formula, For the adaptive grayscale weight of the red channel, The standard deviation of the pixel values ​​in the red channel. The standard deviation of pixel values ​​in the green channel. The standard deviation of the pixel values ​​in the blue channel. For the adaptive grayscale weights of the green channel, For the adaptive grayscale weight of the blue channel, Coordinates in the final grayscale image The grayscale value of the pixel at that location. This is the red channel pixel value of that pixel. This is the green channel pixel value for that pixel. This is the blue channel pixel value for that pixel. The first image data is obtained by using a piecewise adaptive gamma correction model to enhance the contrast of the first sub-image data.

3. The meter reading identification method according to claim 2, characterized in that, The step of performing multi-scale feature extraction and fusion processing on the image fusion data to generate an enhanced multi-scale feature map includes: The image fusion data is input into a lightweight CSPDarknet-tiny backbone network for feature hierarchical abstraction, specifically: first through... The convolutional layer compresses the number of channels of the image fusion data to 64, and then sequentially processes it through layer-by-layer processing with downsampling rates of 2, 4, and 8, outputting three levels of original multi-scale feature maps. The first-level feature map has a size of 960×540×128 and is used to capture edge and texture features; the second-level feature map has a size of 480×270×256 and is used to extract character stroke structure features; the third-level feature map has a size of 240×135×512 and is used to focus on the overall outline and spatial layout features of the meter. Using the aforementioned three-level original multi-scale feature maps as input, the FPN feature enhancement algorithm is employed for bidirectional feature pyramid fusion processing to obtain three sets of enhanced feature maps; wherein, the constructed bidirectional feature pyramid multi-scale fusion expression is: ； In the formula, For the first The enhanced multi-scale feature map obtained after the first-level feature map is fused by the FPN feature enhancement algorithm includes... , , Three sets of enhanced feature maps; For deconvolution operation, For convolution operations, For the first Level 1 original multi-scale feature map, This is a feature fusion operation that adds elements one by one. For the first Level original multi-scale feature map.

4. The meter reading identification method according to claim 3, characterized in that, The deformable convolutional layer based on the deployment performs deformation detection and adjustment on the enhanced multi-scale feature map to obtain a corrected multi-scale feature map adapted to the dense arrangement of the multiple meters, including: Based on the deformable convolutional layer, the boundary spacing between adjacent meters is detected in the meter image region corresponding to the enhanced multi-scale feature map, and the detected value of the boundary spacing between adjacent meters is obtained; wherein, a deformable convolutional layer DCNv2 is deployed on the enhanced multi-scale feature map, and the kernel size of the deformable convolutional layer DCNv2 is... The initial offset range is ±5 pixels; The detection value of the boundary distance between adjacent meters is compared with a preset threshold. If the detection value of the boundary distance between adjacent meters is less than the preset threshold, a dynamic offset expansion mechanism is activated, an expansion coefficient is calculated, and the upper limit of the sampling point offset is expanded to ±8 pixels. If the detection value of the boundary distance between adjacent meters is greater than or equal to the preset threshold, the initial offset range is maintained. Based on the adjusted sampling point parameters, convolution operations are performed using a deformable convolution core formula to obtain a corrected multi-scale feature map adapted to the dense arrangement of the multiple electricity meters; specifically: ； In the formula, To output the position coordinates, For the first Each sampling point is offset from the center of the standard convolution kernel. For learning space offset, This represents the upper limit of the number of sampling points in the convolution kernel. For the first The convolution weight parameters corresponding to each sampling point The first in the convolution kernel The modulation scalar corresponding to each sampling point, where... The sigmoid function is used to constrain the interval to [0,1]. The input feature map is defined by the parameters within the parentheses, which represent the spatial locations of the sampling points within the input feature map. The output position coordinates in the output feature map are Pixel feature values; Deformation constraint functions are introduced during the training process: ； In the formula, The offset constraint cost function value. It is the set of offsets for all sampling points within the convolution kernel. This is the index of the sampling points within the convolution kernel. This represents the upper limit of the number of sampling points in the convolution kernel. For the first The L2 norm of the offset of each sampling point ,in, This represents the horizontal offset component. This represents the offset component in the vertical direction.

5. The method for identifying the meter reading according to any one of claims 1-4, characterized in that, During model training, the loss function is calculated as follows: ； ； ； In the formula, To jointly optimize the loss value, For bounding box regression loss, This is the ratio of the intersection area to the union area of ​​the predicted bounding box and the actual bounding box of the electricity meter. Predict the bounding box for the electricity meter. This is the actual annotation of the meter's bounding box. This is the balancing coefficient for the bounding box regression loss, used to balance the contributions of the center distance term and the aspect ratio term. The normalization factor for the center distance term, A parameter used to measure the consistency of the aspect ratio between the predicted bounding box and the ground truth bounding box. These are the weighting coefficients for the bounding box regression loss. To optimize the classification loss for class imbalance, For category weight coefficients, This represents the probability value predicted by the model that a sample belongs to the target category. Let be the modulation coefficient, where For focusing coefficient, ; Represents the natural logarithm operation. These are the weighting coefficients for the classification loss. The L2 norm of the offset of the deformable convolution sampling points. These are the weighting coefficients for the deformable convolution offset constraint loss.

6. The method for identifying the meter reading according to claim 5, characterized in that, The segmentation of the corrected multi-scale feature map based on the MobileNetV3 backbone network yields a small-scale probability distribution map containing multiple channels, including: The image pixel values ​​of the corrected multi-scale feature map are normalized to... The interval is then uniformly scaled using bicubic interpolation. Pixels, to obtain the preprocessed image; The preprocessed image is input into the MobileNetV3 backbone network and sequentially passes through three depthwise separable convolutional modules to extract local feature maps containing character strokes and barcode patterns. The local feature maps are then input into the MobileViT module, where they are first enlarged by 3×3 convolution and divided into Patch units. Then, a Transformer encoder is used to model global context features and output global context encoding features containing cross-regional correlation information. The CNN local features and the global context-encoded features are concatenated by channels to generate a fused feature map; Channel dimensionality reduction is performed on the fused feature map to obtain a simplified feature map; then, the spatial context features of the simplified feature map are extracted through a 3×3 convolutional layer, and a spatial attention weight map in the 0~1 interval is generated by passing it through a Sigmoid activation function. The spatial attention weight map and the simplified feature map are multiplied element-wise to obtain the effective feature map after spatial attention weighting. Based on the effective feature map, 3×3 and 5×5 depthwise separable convolutions and dilated convolutions with a dilation rate of 3 are used in parallel to capture features at different scales. After channel concatenation and shuffling, 1×1 convolutions are used to map the channels to 5 (each channel corresponds to a class of data elements (barcode, total degree, peak, trough, flat)) to obtain a small-scale classification probability map; specifically: ； ； In the formula, This is the feature map after multi-scale convolution fusion. This represents a depth-separable convolution operation of size 3×3. This represents a depth-separable convolution operation with a size of 5×5. This represents a depthwise separable dilated convolution operation with a dilation rate of 3. This indicates a channel splicing operation. The effective feature map after spatial attention weighting; This is a small-scale classification probability map. This represents the Softmax activation function. This represents a channel transform convolution operation with a 1×1 size convolution kernel. For the reason Multiscale fused feature map obtained through channel washing operation.

7. The method for identifying the meter reading according to claim 6, characterized in that, The adaptive upsampling algorithm is used to optimize the image region edges corresponding to each target element in the small-scale classification probability map to obtain a high-resolution segmentation probability map, including: An adaptive upsampling algorithm is used to reconstruct a high-resolution segmentation structure map based on the small-scale classification probability map and learnable weights; specifically: ； In the formula, The coordinates in the final generated high-definition segmentation structure diagram are Pixel values; The adaptive weighting coefficients are relative to the coordinates within the sampling window. The corresponding learnable weights are as follows: for key areas such as the edges of digit strokes, the weight values ​​are higher to strengthen edge features and avoid breakage, while for background areas, the weight values ​​are lower to suppress noise interference. For all within the adaptive sampling window Summation is performed on index points; This is a small-scale classification probability map; Let be the pixel value at the corresponding position in the small-scale probability map, where This is a floor function used to round down the coordinates of the high-resolution segmented structure image. Mapped to the corresponding baseline position in the small-scale probability map; and This indicates that the relative index of the sampling window is used in conjunction with the reference position. Determine the specific pixel positions in the small-scale probability map that participate in the weighted calculation.

8. The method for identifying the meter reading according to claim 7, characterized in that, The step of converting the high-definition segmentation probability map into an initial binary mask includes: For each channel, the local probability mean and standard deviation of each pixel's neighborhood are calculated based on a 25×25 window to generate a dynamic binarization threshold, and then the initial binary mask is obtained through binarization. After morphological optimization of the initial binary mask, connected component filtering and orientation correction are performed to obtain region coordinate information containing the five target elements, including: After performing closing and opening operations on the initial binary mask, all connected components are extracted, abnormal regions are eliminated, and candidate regions that conform to the form of electricity meter characters are determined. Calculate the minimum bounding rectangle of the candidate region; if the tilt angle of the minimum bounding rectangle is greater than a preset angle, use bilinear interpolation to perform rotation correction so that the character axis is aligned with the image coordinate axis, and obtain the region coordinate information containing the five target elements.

9. The method for identifying the meter reading according to claim 8, characterized in that, The method employs a dual-channel OCR fusion algorithm to identify the region coordinate information, obtaining a readable string corresponding to the target element, including: The region coordinate information is aligned with the original preprocessed image through coordinate mapping, and the element region image is obtained through cropping. Recognition is performed in the main channel based on the PP-OCRv3 backbone network, and adversarial training is conducted to obtain the main channel recognition result. Specifically, a direction-aware module is embedded in the PP-OCRv3 backbone network. First, the gradient histogram features of the element region image are calculated. Then, the gradient histogram features and CNN local features are concatenated channel-wise. Finally, direction-adaptive convolution kernel weights are calculated using convolution operations and the Softmax activation function to improve the recognition rate of tilted characters. ； In the formula, For orientation-adaptive convolutional kernel weights, This represents the Softmax activation function. This represents the convolution operation. The gradient histogram features of the image of the element region are... The CNN local features extracted by the PP-OCRv3 backbone network This indicates a channel splicing operation; In the auxiliary channel, a character region enhancement model based on ViT is used for recognition to obtain the recognition results for the auxiliary channel; specifically: ； In the formula, To assist in text recognition via the auxiliary channel, To assist in channel confidence, Indicates based on Character region enhancement model, It is a 12-layer Transformer structure with a patch size of 4×4; For the element region image, This is a spatial attention map. This indicates a feature channel splicing operation; The main channel recognition result and the auxiliary channel recognition result are aligned at the character level using a dynamic time warping algorithm to obtain the aligned main channel recognition result and auxiliary channel recognition result. The aligned main channel recognition result and auxiliary channel recognition result are dynamically weighted and fused to generate a readable string corresponding to the target element.

10. A meter reading identification system, used to implement the meter reading identification method according to any one of claims 1-9, characterized in that, include: The acquisition module is used to acquire the first and second image data of multiple meters in the calibration chamber; The fusion module is used to combine the first image data and the second image data through a dual-channel stitching operation to form image fusion data, and to perform multi-scale feature extraction and fusion processing on the image fusion data to generate an enhanced multi-scale feature map. An adaptation module is used to perform deformation detection and adjustment on the enhanced multi-scale feature map based on the deployed deformable convolutional layer to obtain a corrected multi-scale feature map that adapts to the dense arrangement of the multiple meters. The segmentation module is used to segment the corrected multi-scale feature map based on the MobileNetV3 backbone network to obtain a small-scale probability distribution map containing multiple channels; wherein, each channel corresponds to a type of target element, and the target element includes barcode, total degree, spike, peak, and valley. The optimization module is used to optimize the image region edges corresponding to each target element in the small-scale classification probability map using an adaptive upsampling algorithm to obtain a high-resolution segmentation probability map. The coordinate generation module is used to convert the high-definition segmentation probability map into an initial binary mask, and after morphological optimization of the initial binary mask, to perform connected component filtering and orientation correction to obtain region coordinate information containing the five target elements. The recognition module is used to recognize the coordinate information of the region using a dual-channel OCR fusion algorithm to obtain a readable string corresponding to the target element.

Images