A cross-channel attention fusion-based water meter reading region detection method

Abstract

The application discloses a kind of across passage attention fusion's water meter reading area detection method, including collecting and marking the picture containing water meter dial;Data processing and data enhancement are carried out to the picture;Water meter reading area detection network is constructed, and the water meter reading area detection network includes feature extraction network, cross-channel attention fusion network and two-stage feature refinement network;Loss is calculated and water meter reading area detection network is trained by back propagation;The water meter picture to be detected is input into the water meter reading area detection network after training and is tested, and the picture with detection frame is output, and the detection frame contains water meter dial reading and water meter pointer reading.The cross-channel fusion attention module and feature refinement module proposed in the application improve the feature extraction effect, can better adapt to water meter image detection demand, with small operation amount to achieve good detection performance, to improve the operation speed of algorithm in actual production process.

Description

Technical Field

[0001] This invention relates to the field of intelligent detection, and in particular to a method for detecting water meter reading areas through cross-channel attention fusion. Background Technology

[0002] The rational and effective use of water resources is of great significance to sustainable development, and obtaining water meter readings is a crucial part of water resource management. Currently, manual meter reading is not only labor-intensive and resource-intensive, but also inefficient. Remote smart meter reading technology, which uses equipment to periodically collect water meter images and uploads them to a remote server where algorithms detect and identify the meter readings, is gaining widespread attention.

[0003] Water meter reading detection requires high accuracy, but water meter images are captured from various angles, making traditional horizontal target detection methods perform poorly on water meter datasets. In water meter images, foreground targets are the reading frames containing the meter dial, divided into the dial wheel frame and the pointer frame. Abundant background information, such as text, can interfere with foreground target detection. Furthermore, water meter detection methods applied in real-world engineering projects demand both high detection speed and accuracy. Therefore, improving the speed of the detection algorithm and reducing the size of the detection model while maintaining good accuracy is a significant technical challenge.

[0004] In a broader sense, performing rotated / oriented object detection on water meter image datasets is a more valuable endeavor. Current algorithms for rotated object detection include improvements from general object detection methods and methods that construct new paradigms, such as FCOS and CornerNet. However, these methods fail to adequately distinguish between foreground and background objects in the backbone network and feature fusion stages, neglecting the necessary information and failing to fully utilize features from feature maps at different scales. This results in persistent issues where the detection bounding boxes do not align well with the actual target environment. Summary of the Invention

[0005] In order to overcome the above-mentioned shortcomings and deficiencies of the prior art, the purpose of this invention is to provide a method for detecting water meter reading areas through cross-channel attention fusion.

[0006] This method is based on the ATSS method, a single-stage object detection method. Compared with the two-stage method, it reduces the processing of the region proposal network, has a faster training and detection speed, and better solves the anchor alignment problem. It also better addresses the defect that the detection box in the water table image cannot adapt to the rotation of the real box.

[0007] The objective of this invention is achieved through the following technical solution:

[0008] A method for detecting water meter reading regions using cross-channel attention fusion includes:

[0009] Collect and label images containing water meter dials;

[0010] Perform data processing and data augmentation on images;

[0011] A water meter reading region detection network is constructed, which includes a feature extraction network, a cross-channel attention fusion network, and a two-stage feature refinement network.

[0012] Calculate the loss and train the water meter reading area detection network using backpropagation;

[0013] The image of the water meter to be tested is input into the trained water meter reading area detection network for testing. The output image contains a detection box, which includes the water meter wheel reading and the water meter pointer reading.

[0014] Furthermore, the annotation is done using a quadrilateral box.

[0015] Furthermore, data processing and data augmentation are performed on the images, specifically:

[0016] The original distribution of the image is randomized through scaling, random flipping, random cropping, normalization, and padding operations. The mean of each channel is subtracted from the data-enhanced image, divided by the variance of each channel, and then normalized and transformed into the image required by the feature extraction network.

[0017] Furthermore, the feature extraction network specifically includes:

[0018] Using a ResNet50 with a deformable convolutional module DCN, the input image is downsampled, and the final stage outputs a feature image in the feature tensor dimension.

[0019] Furthermore, the cross-channel attention fusion network is specifically as follows:

[0020] Input a tensor after feature extraction, wherein the tensor has dimensions C*H*W, where C represents the number of channels, H represents the height of the feature map, and W represents the width of the feature map;

[0021] The final output tensor is obtained using a branching structure, specifically:

[0022] The branch structure includes a first branch, a second branch, and a third branch;

[0023] The first branch is formed by exchanging the dimensions of channel C and width W through a permute operation, passing through a global average pooling layer, then through three 3*3 convolutional blocks, then through a sigmoid activation function multiplied with the input of the first branch, and finally through permute to convert it into a channel * width * height dimensional feature.

[0024] The second branch swaps the dimensions of channel C and height H through a permute operation, passes through a global average pooling layer, and then through three 3x3 convolutional modules. Each 3x3 convolutional module specifically: first, it passes through a 3x3 convolutional kernel, then through a ReLU function, then through batch normalization, then through a sigmoid activation function multiplied by the input, and finally through permute to convert it to the dimension of channel * width * height.

[0025] The third branch is formed by multiplying the input of this branch with the input after global average pooling, two 1*1 convolutional layers, sigmoid processing, and finally permute.view operation, resulting in C*H*W.

[0026] The outputs of the three branches are summed and averaged to obtain the final output tensor.

[0027] Furthermore, the two-stage feature refinement network is specifically as follows:

[0028] The first stage involves extracting the feature tensor from the feature extraction network, performing an upsampling operation corresponding to the downsampling operation during feature extraction, and then adding the upsampled and downsampled output features of the same scale to obtain new features.

[0029] Let a new feature at a certain scale be C*H*W, where C represents the number of channels, H represents the height of the feature map, and W represents the feature map. Process the following two branches:

[0030] One branch uses global average pooling to output a tensor of size C*1*1. After passing through a 1*1 convolutional layer, the output tensor size is C / 2*1*1. After passing through the ReLU function, the output tensor dimension is C / 2*1*1.

[0031] Another branch uses a 3*3 dilated convolutional layer, with an output tensor of C*(H-2)*(W-2), then passes through a 1*1 convolutional layer, with an output tensor size of C / 2*1*1, and after passing through the ReLU function, the output tensor dimension is C / 2*1*1.

[0032] Connect the two branches, perform a cat operation on each channel, and obtain the output feature, denoted as F1;

[0033] Second stage: The output features of the first stage are downsampled by the same factor as the first stage, and then upsampled by the same factor. The upsampled output features and the downsampled output features of the same scale are added together to obtain a new feature, denoted as F2.

[0034] Adding F1 and F2 yields the output of the two-stage feature refinement network.

[0035] Furthermore, the calculation of loss and backpropagation training of the water meter reading area detection network specifically involves: traversing pixels on the water meter image, generating anchor boxes of different scales at each location, and generating ground truth boxes based on the dataset annotations. The anchor boxes and ground truth boxes are encoded, and the IoU between the ground truth boxes and anchor boxes is calculated.

[0036] The assignment of positive and negative samples is as follows: For each ground truth box in the image, denoted as GT, in each feature pyramid layer, each GT is traversed, and the IoU between each GT and the anchor is calculated. The L2 distance between the center coordinates of each GT and the center coordinates of all anchors in that layer is calculated. Based on the L2 distance, the k candidate positive samples closest to the center coordinates of the GT are selected. There are a total of 4 feature layers, which will select a total of 4*k candidate boxes. The IoU value between each candidate box and the GT is calculated, and the mean and variance of this set of IoU values ​​are calculated. The sum of the mean and variance is used as the IoU threshold. Candidate boxes with values ​​greater than the threshold are selected as the final positive samples and are responsible for predicting the GT. If an anchor box is assigned to multiple GTs, the box with the highest IoU is selected, and the rest are negative samples. If the threshold is set too high, some positive samples may not be located inside the GT. These positive samples are filtered out and set as background samples.

[0037] The Focal strategy is used for sample balancing. The localization loss, classification loss and angle regression loss of positive and negative samples are calculated. The target of interest in the image is divided into two categories: water meter wheel reading (text) and water meter pointer reading (circle). The classification loss is Focal Loss and the bounding box regression loss is smooth-L1 Loss.

[0038] Based on the IoU value calculated between the generated anchor boxes and the ground truth boxes, the non-maximum suppression (NMS) method is used to filter out redundant candidate boxes. After calculating the loss, backpropagation is used to train the entire network model, and the final output is an image with candidate boxes.

[0039] Furthermore, the downsampling includes 1 / 4, 1 / 8, 1 / 16, and 1 / 32 magnification.

[0040] A system for detecting water meter reading zones includes:

[0041] Acquisition module: Used to acquire and annotate images of the water meter dial;

[0042] Processing module: Used for data processing and enhancement of the labeled images;

[0043] Construction and training module: used to construct the water meter reading area detection network and train the detection network;

[0044] Detection module: Input the image of the water meter to be detected into the model and output the image of the candidate box.

[0045] A storage medium storing computer instructions, which, when executed by a processor, implement the water meter reading area detection method.

[0046] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0047] This method uses rotated quadrilateral bounding boxes to label water meter datasets, which can detect rotated rectangular bounding boxes, making it more suitable for water meter reading detection tasks.

[0048] This method introduces an attention module and a feature refinement module, and is modified and customized according to the characteristics of the network and the task, achieving good performance with low computational cost and appropriate training speed. Attached Figure Description

[0049] Figure 1 This is a flowchart of the process of the present invention.

[0050] Figure 2 This is a schematic diagram of the cross-channel fusion interest module of the present invention.

[0051] Figure 3 This is a schematic diagram of the water meter detection results in Embodiment 1 of the present invention. Detailed Implementation

[0052] The present invention will be further described in detail below with reference to the embodiments, but the implementation of the present invention is not limited thereto.

[0053] like Figures 1-3 As shown, a method for detecting water meter reading regions using cross-channel attention fusion includes:

[0054] S1 collects and labels images containing water meter dials.

[0055] Image annotation uses a rotated quadrilateral frame. Specifically, the acquired meter image must contain the water meter dial and be relatively clear. The image is cropped, and an annotation tool is used, employing a rotated quadrilateral frame, to mark the pixel positions of the water meter reading frame and the water meter pointer frame. These positions are recorded in the file as the pixel coordinates of the four corners of the quadrilateral.

[0056] S2 performs data processing and data augmentation on the images, specifically as follows:

[0057] The data augmentation operation randomizes the original distribution of the image through color transformation, scaling, random flipping, random cropping, and padding. It converts the image to a 1024*1024 pixel size. If the original image size is smaller than 1024*1024 pixels, it is padded with grayscale values; if the original image size is larger than 1024*1024 pixels, it is randomly cropped to a uniform size for subsequent processing.

[0058] S3 constructs a water meter reading region detection network, which includes a feature extraction network, a cross-channel attention fusion network, and a two-stage feature refinement network.

[0059] The feature extraction network uses a ResNet50 with a deformable convolutional module DCN. The input image is downsampled, and the final stage outputs a feature image with a feature tensor dimension.

[0060] The feature extraction process in this embodiment is as follows:

[0061] Using ResNet50, the input image tensor dimension is N*3*1024*1024, where N represents the number of images. After downsampling by 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32, the final output feature tensor dimension is N*2048*32*32.

[0062] Because the water meter images in the dataset are arbitrarily oriented, and the shooting angle cannot be guaranteed to be parallel to the horizontal direction, and the ResNet50 network lacks rotation invariance, some features will be lost during the feature extraction process of convolution operations. Therefore, a deformable convolutional network (DCN) is introduced into the backbone network. This module has rotation invariance and can preserve the features of rotated images more completely. Specifically, ResNet50 is divided into four stages: stage 1, stage 2, stage 3, and stage 4, with DCN processing added in stages 2, 3, and 4.

[0063] The cross-channel attention fusion network is specifically as follows:

[0064] Since the foreground and background weights are not consistent in water meter images, this network can increase the network's attention weight for foreground target boxes. Specifically, the input is a tensor after feature extraction, and a branching structure is used, such as... Figure 2 As shown. The tensor dimension is C*H*W, where C represents the number of channels, H represents the height of the feature map, and W represents the width of the feature map.

[0065] The branching structure includes three branches, as follows:

[0066] The first branch: Channel C and width W are swapped through a permute operation, then passed through a global average pooling layer, followed by three 3x3 convolutional blocks, outputting a feature map with a dimension of 1*C*W. This is then multiplied by the input from the first branch using a sigmoid activation function, and finally permuteed to transform it into a feature map with dimensions C*H*W. Each 3x3 convolutional module specifically: first, it passes through a 3x3 convolutional kernel, then through a ReLU function, and finally through batch normalization.

[0067] The second branch: Channel C and height H exchange dimensions through a permute operation, pass through a global average pooling layer, and then through three 3x3 convolutional modules, the same as in the first branch, outputting a feature dimension of 1*H*C. The output is then multiplied by the second branch input through a sigmoid activation function, and finally permuteed, outputting a tensor with a dimension of C*H*W.

[0068] The third branch: After global average pooling, it goes through two 1x1 convolutional layers, then the sigmoid function, multiplied by the input of this branch, and finally passed through permute.view, resulting in an output tensor with dimensions C*H*W. Adding the outputs of the three branches and averaging them yields the final output tensor, with dimensions (C*H*W), i.e.:

[0069]

[0070] The outputs of the three branches are summed and averaged to obtain the final output tensor.

[0071] The two-stage feature refinement network specifically includes two stages;

[0072] The first phase includes:

[0073] The feature extraction network performs downsampling at 1 / 4, 1 / 8, 1 / 16, and 1 / 32 times, outputting feature tensors at four different scales. The output tensor dimensions are 256*256*256, 512*128*128, 1024*64*64, and 2048*32*32, respectively, with the first number representing the number of channels. Subsequently, upsampling is performed at 4x, 8x, 16x, and 32x. The upsampled and downsampled features at the same scale are then added together (add) to obtain new features, resulting in four new features at different scales.

[0074] Let the size of the new feature map obtained in the previous step at a certain scale be C*H*W, where C represents the number of channels, H represents the height of the feature map, and W represents the width of the feature map. Then, after the following processing, two branches are used: First, global average pooling is applied, resulting in an output tensor of size C*1*1. This is followed by a 1*1 convolutional layer, resulting in an output tensor of size C / 2*1*1, and then ReLU, resulting in an output tensor of size C / 2*1*1. Second, a 3*3 dilated convolutional layer is applied, followed by another 1*1 convolutional layer, resulting in an output tensor with C / 2 channels, and then ReLU processing. The two branches are then connected, and a cat operation is performed on each channel. At this point, the output tensor dimension is the same as before entering S3. This is denoted as F1.

[0075] Furthermore, similar to the above operation, the feature map output from the first stage is downsampled again by 1 / 4, 1 / 8, 1 / 16, and 1 / 32 times. Each of the four stages outputs a feature tensor at a different scale, i.e., the output tensor dimensions remain 256*256*256, 512*128*128, 1024*64*64, and 2048*32*32 respectively. Then, it is upsampled by 2, 4, 8, 16, and 32 times. The upsampled and downsampled features at the same scale are added together (add) to obtain a new feature, denoted as F2. F1 and F2 are then added together (add) to obtain the final output.

[0076] This method uses a two-stage training approach, fusing the features extracted in the first stage and the features extracted in the second stage in three ways, so that the features extracted in the second stage can guide the training in the first stage and pay attention to the features ignored during backpropagation.

[0077] S4 calculates the loss and backpropagates to train the water meter reading area detection network;

[0078] Specifically:

[0079] Generate anchor boxes with pre-defined parameters on the water meter image, encode candidate boxes and ground truth boxes, and calculate the IoU between the ground truth boxes and the anchors.

[0080] Positive and negative samples are assigned. Specifically, for each ground truth box in the image, denoted as GT, in each feature pyramid layer, each GT is traversed, and the IoU between each GT and the anchor is calculated. The L2 distance between the center coordinates of each GT and the center coordinates of all anchors in that layer is calculated. Based on the L2 distance, the k candidate positive samples closest to the center coordinates of the GT are selected. There are a total of 4 feature layers, which will select a total of 4*k candidate boxes. The IoU value between each candidate box and the GT is calculated, and the mean and variance of this set of IoU values ​​are calculated. The sum of the mean and variance is used as the IoU threshold. Candidate boxes with IoU values ​​greater than the threshold are selected as the final positive samples and are responsible for predicting the GT. If an anchor box is assigned to multiple GTs, the box with the highest IoU is selected, and the rest are negative samples. If the threshold is set too high, some positive samples may not be located within the GT. These positive samples are filtered out and set as background samples.

[0081] Sample balancing is performed using a Focal strategy. Since the water meter images are oriented differently, an angle variable is introduced. Localization loss, classification loss, and angle regression loss are calculated for both positive and negative samples. The objects of interest in the images are divided into two categories: water meter wheel reading (text) and water meter pointer reading (circle). The classification loss is Focal Loss, and the bounding box regression loss is smooth-L1 Loss. The loss function is:

[0082] L = L cls +L reg =L Focal +L smooth-L1 +L angle

[0083] Based on the IoU value calculated between the generated anchor boxes and the ground truth boxes, non-maximum suppression (NMS) is used to filter out redundant candidate boxes. After calculating the loss, backpropagation is used to train the entire network model. The final output is an image with candidate boxes.

[0084] This method introduces an angle prediction loss variable, enabling it to detect rotated rectangular boxes, a capability previously unavailable. A cross-channel focus module is also designed to better meet the needs of water meter reading detection. Furthermore, this invention adds a feature refinement module, enhancing the perception of multi-scale features and achieving excellent results.

[0085] S5 inputs the image of the water meter to be detected into the trained water meter reading region detection network and outputs an image with encoded candidate boxes. The candidate boxes include the water meter wheel reading (text class) and the water meter pointer reading (circle class).

[0086] Example 2

[0087] A system for detecting water meter reading zones includes:

[0088] Acquisition module: Used to acquire and annotate images of the water meter dial;

[0089] Processing module: Used for data processing and enhancement of the labeled images;

[0090] Construction and training module: used to construct the water meter reading area detection network and train the detection network;

[0091] Detection module: Input the image of the water meter to be detected into the model and output the image of the candidate box.

[0092] Example 3

[0093] A storage medium storing computer instructions, which, when executed by a processor, implement the detection method described in Embodiment 1.

[0094] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the embodiments described above. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for cross-attention fusion-based water meter reading region detection, characterized in that, The utility model relates to a kind of water meter reading method, comprising: Collect and mark the picture containing water meter dial; Data processing and data enhancement are carried out on the picture; A water meter reading area detection network is constructed, which includes a feature extraction network, a cross-channel attention fusion network and a two-stage feature refinement network; Loss is calculated, and the water meter reading area detection network is trained using backpropagation; The trained water meter reading area detection network is input into the water meter reading area detection network for testing, and the picture with detection frame is output, which contains the water meter dial reading and the water meter pointer reading; The feature extraction network specifically includes: ResNet50 with deformable convolution module (DCN) is used to input the picture, downsample it, and output the feature tensor of the last stage; The cross-channel attention fusion network specifically includes: The tensor after feature extraction is input, with dimensions C*H*W, where C represents the number of channels, H represents the height of the feature map, and W represents the width of the feature map; The final output tensor is obtained using a branch structure, specifically: The branch structure includes a first branch, a second branch and a third branch; The first branch exchanges the dimensions of channel C and width W through permute operation, passes through a global average pooling layer, then passes through three 3*3 convolution blocks, multiplies the output of the first branch by the sigmoid activation function, and finally converts the dimensions to channel*width*height through permute operation; The second branch exchanges the dimensions of channel C and height H through permute operation, passes through a global average pooling layer, then passes through three 3*3 convolution blocks, each of which specifically includes: first passing through a 3*3 convolution kernel, then passing through a ReLU function, then passing through batch Normalization, then multiplying the input by the sigmoid activation function, and finally converting the dimensions to channel*width*height through permute operation; The third branch passes through a global average pooling layer, then passes through two 1*1 convolution layers, processes the output using sigmoid, multiplies the output by the input of the third branch, and finally converts the dimensions to C*H*W through permute.view operation; The outputs of the three branches are added and averaged to obtain the final output tensor; The two-stage feature refinement network specifically includes: In the first stage, the feature tensor output by the feature extraction network is upsampled, the upsample rate corresponds to the downsample rate in the feature extraction process, the upsampled output feature and the downsampled output feature of the same scale are added to obtain a new feature; Let a certain scale new feature be C*H*W, C represents the number of channels, H represents the height of the feature map, and W represents the feature map; One branch uses global average pooling, outputs a tensor of C*1*1, passes through a 1*1 convolution layer, outputs a tensor of C / 2*1*1, and then passes through a ReLU function to output a tensor of C / 2*1*1. Another branch, using 3*3 dilated convolution layer, the output tensor is C*(H-2)*(W-2), then through 1*1 convolution layer, the output tensor size is C / 2*1*1, through the ReLU function, the output tensor dimension is C / 2*1*1; The two branches are connected, and the cat operation is performed on the channels to obtain the output feature, denoted as F1. The second stage: the output feature of the first stage is subjected to down-sampling with the same rate as the first stage, and then subjected to up-sampling with the same rate, and the output features of the same scale are added to obtain new features, denoted as F2. F1 and F2 are added to obtain the output of the two-stage feature refinement network.

2. The water meter reading region detection method according to claim 1, characterized by, The annotation is a quadrilateral box annotation.

3. The water meter reading region detection method according to claim 1, characterized by The data processing and data enhancement of the picture are as follows: Through scaling, random flipping, random cropping, normalization and padding operations, the original distribution of the image is randomized, the data enhanced picture is subtracted by the mean value of each channel, divided by the variance of each channel, then normalized, and converted into an image required by the feature extraction network.

4. The water meter reading region detection method according to claim 1, characterized by, The loss is calculated and back-propagation training is performed on the water meter reading region detection network, specifically: traversing the pixel points on the water meter image, generating anchor boxes of different scales at each position, and obtaining real boxes according to the data set annotation, encoding the anchor boxes and the real boxes, and calculating the IoU between the real boxes and the anchor boxes; Assigning positive and negative samples, specifically: for each ground truth box GT on the image, traverse each GT on each layer of the feature pyramid, calculate the IoU between each GT and the anchor, calculate the L2 distance between the center coordinates of each GT and the center coordinates of all anchors on the layer, select the k candidate positive samples closest to the center coordinates of the GT according to the L2 distance, a total of 4*k candidate boxes will be selected, calculate the IoU value of each candidate box and the GT, and calculate the mean and variance of the IoU value set, and the sum of the mean and variance is taken as the IoU threshold, and the candidate boxes greater than the threshold are selected as the final positive samples responsible for predicting the GT; if an anchor box is assigned to multiple GTs, the one with the highest IoU is selected, and the rest are negative samples; if the threshold is set too large, some positive samples may not be inside the GT, which are filtered out and set as background samples; Using the Focal strategy for sample balancing, calculating the positioning loss, classification loss and angle regression loss of positive and negative samples, the detection target of interest in the image is divided into two categories: water meter text reading and water meter pointer reading, the classification loss is Focal Loss, and the boundary box regression loss is smooth-L1 Loss; According to the IoU value calculated from the generated anchor box and the real box, the non-maximum suppression NMS method is used to filter redundant candidate boxes, and after the loss is calculated, back-propagation is used to train the entire network model, and finally an image with unique detection is output.

5. The water meter reading region detection method according to claim 1, characterized by, The downsampling includes 1 / 4, 1 / 8, 1 / 16, 1 / 32 times.

6. A system for water meter reading zone detection based on the method of any of claims 1-5, characterized by, Comprise: The acquisition module is used for acquiring the meter dial image and marking; The processing module is used for data processing and enhancement of the marked image; The construction and training module is used for constructing the meter reading area detection network and training the detection network; The detection module inputs the image of the meter to be detected into the model and outputs the image containing the detection frame.

7. A storage medium having stored thereon computer instructions, characterized in that The computer instructions are executed by the processor to realize the meter reading area detection method in any one of claims 1-5.

Images