A visual detection method and system for the state of in-box equipment of an electric energy metering box

Abstract

The application provides a kind of visual detection method and system for the state of equipment in electric energy metering box, the method includes collecting the image in electric energy metering box, and initial feature tensor is generated by normalization and size scaling preprocessing;The initial feature tensor is sent into the backbone network to extract multi-scale initial feature map, and weighted feature fusion is carried out;The fusion feature map is input into the detection head containing classification branch and regression branch, and the channel dimension and spatial dimension are bidirectionally calibrated through the intermediate features in the two branches;The shape prediction vector output by the regression branch is used to solve the reference scale and the reference width-height ratio with the reference size matrix, and the prediction box is obtained by combining the regression offset amount decoding;After screening by non-maximum suppression, the visual detection results of visible state of equipment in box and foreign matter are output.The application can alleviate the problem of mismatch between classification and positioning under complex box background, and improve the accuracy and reliability of visible state detection of equipment in metering box and foreign matter.

Description

Technical Field

[0001] This application belongs to the field of electricity metering boxes, and in particular relates to a visual inspection method and system for the status of equipment inside an electricity metering box. Background Technology

[0002] As a crucial terminal device in the power grid, the internal environment and visible condition of electricity metering boxes directly impact the reliability of user electricity supply and the safety of grid operation. In practical applications, the inside of metering boxes may be invaded by foreign objects such as flying insects, bird nests, honeycomb, or leftover construction tools. Visible abnormalities that can be identified through image processing may also occur, such as meter obstruction, abnormally exposed wiring terminals, or abnormal switch positions. These abnormalities can affect the insulation performance of the equipment, operational safety, and the efficiency of subsequent inspections and maintenance. In severe cases, they may even lead to safety hazards such as short circuits, overheating, or fires.

[0003] In recent years, image detection technologies based on machine vision and deep learning have been gradually applied to power equipment inspection scenarios. Conventional visual inspection methods typically rely on convolutional neural networks to extract features from images acquired on-site and construct multi-level feature pyramids to handle targets of different sizes. However, the interior of metering boxes often presents interference factors such as insufficient lighting, crisscrossing cables, dense components, and complex background textures. Existing multi-scale feature fusion strategies mostly employ element-wise addition or channel concatenation, which struggles to adequately distinguish the importance of semantic and detail information at different network depths. This can easily lead to insufficient fusion between deep semantics and shallow texture, thus affecting the recognition performance of target regions in complex backgrounds and increasing the risk of missed or false detections.

[0004] In addition, in existing object detection network architectures, the detection head is typically divided into two relatively independent computational paths: a classification branch and a regression branch. The two branches lack sufficient information interaction during feature extraction and prediction, easily leading to inconsistencies between classification and localization results. For example, some candidate boxes may have high class confidence, but their bounding box positions may still show significant deviations. In the target bounding box decoding stage, traditional algorithms usually rely on manually preset anchor box sizes or a single regression benchmark for offset calculation. Since the objects detected inside the metering box include various shapes of foreign objects, as well as visible areas of equipment such as meters, terminals, and switches, different targets vary in size, outline, and aspect ratio. A single benchmark setting is insufficient to fully adapt to diverse target shapes, thus affecting the accuracy of bounding box regression. Therefore, it is necessary to propose a visual detection method suitable for scenarios inside electricity metering boxes to improve the accuracy and stability of detection results in feature fusion, task branch interaction, and predicted box decoding. Summary of the Invention

[0005] To address the problems in existing technologies, such as insufficient fusion of multi-scale features in complex backgrounds inside metering boxes, inadequate interaction between classification and positioning branches, and difficulty in adapting fixed anchor frames to diverse target shapes, which lead to insufficient accuracy in detecting the visible status of equipment and foreign objects, this invention proposes a visual detection method and system for the status of equipment inside an energy metering box.

[0006] According to one aspect of the present invention, a visual inspection method for the status of equipment inside an electricity metering box is provided, comprising the following steps: An image of the inside of the electricity metering box is acquired. The image is preprocessed based on normalization and size scaling to obtain an initial feature tensor. The initial feature tensor is input into the backbone network, and multi-scale initial feature maps containing semantic information at different levels are extracted according to different network depths. Weighted feature fusion is performed based on the multi-scale initial feature map to obtain a fused feature map. The fused feature map is then input into a detection head containing a classification branch and a regression branch. Intermediate layer features of the classification branch and the regression branch are extracted respectively. The channel dimension of the intermediate layer features of the regression branch is calibrated based on the intermediate layer features of the classification branch, and the spatial dimension of the intermediate layer features of the classification branch is calibrated based on the intermediate layer features of the regression branch. The reference scale and reference aspect ratio are obtained by calculating the shape prediction vector output by the regression branch at each spatial location and the preset reference size matrix. The reference scale and reference aspect ratio are used as the reference for decoding the prediction box size. The prediction box is obtained by decoding the regression offset output by the regression branch. The prediction probability and the prediction box are post-processed based on nonmaximum suppression to obtain the final visual detection result of the equipment status inside the box.

[0007] Optionally, the step of acquiring an image of the interior of the electricity metering box, and preprocessing the image based on normalization and size scaling to obtain an initial feature tensor, includes: The multi-channel pixel matrix of the target area inside the power metering box is obtained as the original image using an image acquisition device; The original image is scaled to a square input size with equal side lengths; Subtract the mean of the corresponding channel from the pixel values ​​of all training images in the dataset, and divide the difference by the standard deviation of the corresponding channel to convert it into the final initial feature tensor after standardization adjustment of the numerical distribution.

[0008] Optionally, the step of inputting the initial feature tensor into the backbone network and extracting multi-scale initial feature maps containing semantic information at different levels according to different network depths includes: The initial feature tensor is input into a backbone network containing residual blocks for downsampling feature extraction. As the network depth increases, feature maps are extracted at the exit points of the three set downsampling stages. Parallel hollow spatial pyramid pooling operation is applied to the extracted feature maps at each stage to extract features with different receptive fields, resulting in multi-scale initial feature maps containing bottom-level boundary texture features, mid-level device localization features, and deep-level global semantic features.

[0009] Optionally, the step of performing weighted feature fusion based on the multi-scale initial feature map to obtain a fused feature map includes: In the top-down fusion path, high-level semantic features are input into a network structure that includes global average pooling and multilayer perceptron, and are mapped to generate content-driven convolutional kernel parameters. The convolution kernel is used to perform convolution operations on low-level detail features in the downlink branches of adjacent network layers, and a spatial weight map is generated by mapping through a normalized activation function. The spatial weight map is multiplied element-wise with the low-level detail features, and the result of the multiplication is added element-wise with the high-level semantic features that have been upsampled and aligned by channel dimension convolution to obtain the fused feature map.

[0010] Optionally, the step of labeling the channel dimension of the intermediate layer features of the regression branch based on the intermediate layer features of the classification branch includes: Global spatial average pooling is performed on the intermediate layer features of the classification branch to obtain the global response value of each channel; The global response value is input into a fully connected network containing a non-linear activation function for feature activation, and after being mapped by the Sigmoid function, an importance gain score in the channel dimension is generated. The importance gain score is multiplied element-wise with the intermediate layer features of the regression branch along the channel dimension to calibrate the channel dimension importance of the intermediate layer features of the regression branch.

[0011] Optionally, the step of calculating the reference scale and reference aspect ratio by operating the shape prediction vector output by the regression branch at each spatial location and a preset reference size matrix, using the reference scale and reference aspect ratio as the decoding reference for the prediction box size, and decoding the prediction box by combining the regression offset output by the regression branch, includes: The intermediate layer features after the regression branch is calibrated are subjected to feature mapping processing to obtain the shape prediction vectors output at each spatial location; The reference size matrix is ​​obtained from the parameter library, which represents the reference scale of the object and the standard proportion of the appearance boundary. The shape prediction vectors output at each spatial location are multiplied by the reference size matrix, and the results of the matrix multiplication are positively mapped to obtain the reference scale and reference aspect ratio corresponding to the current spatial location. Based on the reference scale and the reference aspect ratio, the reference absolute width and reference absolute height are calculated. The grid center coordinates corresponding to each spatial position are used as the center position reference, and the reference absolute width and reference absolute height are used as the size reference. Combined with the center offset and size scaling offset in the regression offset, the position coordinates and size information of the prediction box are decoded.

[0012] Optionally, the post-processing of the predicted probability and the predicted bounding box based on nonmaximum suppression to obtain the final visual detection result of the equipment status inside the box includes: Candidate prediction boxes with predicted probabilities lower than the set confidence threshold are filtered out, and a category determination list is established according to the highest probability category of each candidate prediction box. For each category determination list, the remaining candidate prediction boxes are added to the corresponding initial retention set. The candidate prediction boxes in the initial retention set are sorted in descending order according to the prediction probability, and the candidate prediction box with the highest score is taken and moved into the final retention library. Calculate the intersection-union ratio (IUU) of the remaining candidate predicted boxes in the same category judgment list with the candidate predicted box with the highest score, and remove candidate predicted boxes whose IUU is greater than a set overlap threshold; The operation of retrieving the highest-scoring predicted bounding box and removing overlapping predicted bounding boxes is performed repeatedly until the initial retention set is empty. The detection category and bounding box position information in the final retention library are then output as the final visual detection result of the in-box device status.

[0013] According to another aspect of the present invention, a visual inspection system for the status of equipment inside an electricity metering box is provided, comprising the following modules: The extraction module is used to acquire images inside the electricity metering box, preprocess the images based on normalization and size scaling to obtain an initial feature tensor, input the initial feature tensor into the backbone network, and extract multi-scale initial feature maps containing semantic information at different levels according to different network depths. The calibration module is used to perform weighted feature fusion based on the multi-scale initial feature map to obtain a fused feature map, input the fused feature map into a detection head containing a classification branch and a regression branch, extract the intermediate layer features of the classification branch and the intermediate layer features of the regression branch respectively, perform channel dimension calibration on the intermediate layer features of the regression branch based on the intermediate layer features of the classification branch, and perform spatial dimension calibration on the intermediate layer features of the classification branch based on the intermediate layer features of the regression branch. The output module is used to calculate the reference scale and reference aspect ratio by performing calculations on the shape prediction vector output by the regression branch at each spatial location and the preset reference size matrix. The reference scale and reference aspect ratio are used as the reference for decoding the prediction box size. The prediction box is obtained by decoding the regression offset output by the regression branch. The prediction probability and the prediction box are post-processed based on nonmaximum suppression to obtain the final visual inspection result of the equipment status inside the box.

[0014] Preferably, the step of acquiring an image of the inside of the electricity metering box, and preprocessing the image based on normalization and size scaling to obtain an initial feature tensor, includes: The multi-channel pixel matrix of the target area inside the power metering box is obtained as the original image using an image acquisition device; The original image is scaled to a square input size with equal side lengths; Subtract the mean of the corresponding channel from the pixel values ​​of all training images in the dataset, and divide the difference by the standard deviation of the corresponding channel to convert it into the final initial feature tensor after standardization adjustment of the numerical distribution.

[0015] Preferably, the step of inputting the initial feature tensor into the backbone network and extracting multi-scale initial feature maps containing semantic information at different levels according to different network depths includes: The initial feature tensor is input into a backbone network containing residual blocks for downsampling feature extraction. As the network depth increases, feature maps are extracted at the exit points of the three set downsampling stages. Parallel hollow spatial pyramid pooling operation is applied to the extracted feature maps at each stage to extract features with different receptive fields, resulting in multi-scale initial feature maps containing bottom-level boundary texture features, mid-level device localization features, and deep-level global semantic features.

[0016] Preferably, the step of performing weighted feature fusion based on the multi-scale initial feature map to obtain a fused feature map includes: In the top-down fusion path, high-level semantic features are input into a network structure that includes global average pooling and multilayer perceptron, and are mapped to generate content-driven convolutional kernel parameters. The convolution kernel is used to perform convolution operations on low-level detail features in the downlink branches of adjacent network layers, and a spatial weight map is generated by mapping through a normalized activation function. The spatial weight map is multiplied element-wise with the low-level detail features, and the result of the multiplication is added element-wise with the high-level semantic features that have been upsampled and aligned by channel dimension convolution to obtain the fused feature map.

[0017] Preferably, the step of labeling the channel dimension of the intermediate layer features of the regression branch based on the intermediate layer features of the classification branch includes: Global spatial average pooling is performed on the intermediate layer features of the classification branch to obtain the global response value of each channel; The global response value is input into a fully connected network containing a non-linear activation function for feature activation, and after being mapped by the Sigmoid function, an importance gain score in the channel dimension is generated. The importance gain score is multiplied element-wise with the intermediate layer features of the regression branch along the channel dimension to calibrate the channel dimension importance of the intermediate layer features of the regression branch.

[0018] Preferably, the step of calculating the reference scale and reference aspect ratio by operating the shape prediction vector output by the regression branch at each spatial location and a preset reference size matrix, using the reference scale and reference aspect ratio as the decoding reference for the prediction box size, and decoding the prediction box by combining the regression offset output by the regression branch, includes: The intermediate layer features after the regression branch is calibrated are subjected to feature mapping processing to obtain the shape prediction vectors output at each spatial location; The reference size matrix is ​​obtained from the parameter library, which represents the reference scale of the object and the standard proportion of the appearance boundary. The shape prediction vectors output at each spatial location are multiplied by the reference size matrix, and the results of the matrix multiplication are positively mapped to obtain the reference scale and reference aspect ratio corresponding to the current spatial location. Based on the reference scale and the reference aspect ratio, the reference absolute width and reference absolute height are calculated. The grid center coordinates corresponding to each spatial position are used as the center position reference, and the reference absolute width and reference absolute height are used as the size reference. Combined with the center offset and size scaling offset in the regression offset, the position coordinates and size information of the prediction box are decoded.

[0019] Preferably, the post-processing of the predicted probability and the predicted bounding box based on nonmaximum suppression to obtain the final visual detection result of the equipment status inside the box includes: Candidate prediction boxes with predicted probabilities lower than the set confidence threshold are filtered out, and a category determination list is established according to the highest probability category of each candidate prediction box. For each category determination list, the remaining candidate prediction boxes are added to the corresponding initial retention set. The candidate prediction boxes in the initial retention set are sorted in descending order according to the prediction probability, and the candidate prediction box with the highest score is taken and moved into the final retention library. Calculate the intersection-union ratio (IUU) of the remaining candidate predicted boxes in the same category judgment list with the candidate predicted box with the highest score, and remove candidate predicted boxes whose IUU is greater than a set overlap threshold; The operation of retrieving the highest-scoring predicted bounding box and removing overlapping predicted bounding boxes is performed repeatedly until the initial retention set is empty. The detection category and bounding box position information in the final retention library are then output as the final visual detection result of the in-box device status.

[0020] The present invention has the following advantages: By extracting multi-level features from images inside the box and fusing semantic and detail information at different scales using a weighted mechanism, the ability to perceive visible areas of foreign objects and equipment in complex box backgrounds can be improved. During detection, bidirectional interactive calibration between classification and regression branches is used, allowing classification features to participate in the channel calibration of regression features and vice versa, thus mitigating the mismatch between classification judgment and bounding box localization. Furthermore, a baseline scale and aspect ratio adapted to the current target shape are obtained through shape prediction vector and baseline size matrix operations, and this is combined with regression offsets to complete the predicted bounding box decoding, reducing the limitations of fixed anchor frames on diverse target shapes. Therefore, the accuracy and reliability of detecting visible areas of foreign objects and equipment inside power metering boxes can be improved. Attached Figure Description

[0021] Figure 1 A flowchart of the first embodiment; Figure 2 This is a schematic diagram of the dilation rate parameter for dilated convolution branches. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] In the first embodiment, the present invention proposes a visual detection method for the status of equipment inside an electricity metering box, such as... Figure 1 It includes the following steps: S1. Acquire an image of the inside of the electricity metering box. Preprocess the image based on normalization and size scaling to obtain an initial feature tensor. Input the initial feature tensor into the backbone network and extract multi-scale initial feature maps containing semantic information at different levels according to different network depths.

[0024] Color images are captured by a high-definition industrial camera installed inside the electricity metering box to serve as the original images of the box's interior. A bilinear interpolation algorithm is used to adjust the spatial resolution of the original images to a preset standard input size, such as 640 pixels in both width and height. The horizontal and vertical scaling ratios between the original image width, original image height, standard input width, and standard input height are recorded.

[0025] The pixel values ​​of the red, green, and blue channels of each pixel in the resized image are divided by 255 to perform preliminary normalization and map them to the range of 0 to 1. Then, the mean obtained from the large image dataset is subtracted and divided by the standard deviation to complete the standardization process, resulting in the preprocessed three-dimensional initial feature tensor.

[0026] The initial feature tensor is input into a convolutional neural network backbone consisting of alternating convolutional layers, batch normalization layers, and activation function layers. As the initial feature tensor propagates forward through the backbone, the spatial size of the feature map is halved at each downsampling module, while the number of channels increases proportionally. During the forward propagation of the network, feature truncation is performed at three preset downsampling rate output nodes, extracting feature maps with spatial resolutions of 1 / 8, 1 / 16, and 1 / 32 of the input size, respectively. These three feature maps correspond to high-resolution local detail texture features extracted from the shallow layers, medium-resolution transition features extracted from the middle layers, and low-resolution global contextual semantic features extracted from the deep layers, respectively. These three feature maps together constitute a multi-scale initial feature map containing semantic information at different levels.

[0027] In an optional embodiment, the step of acquiring an image of the interior of the electricity metering box and preprocessing the image based on normalization and size scaling to obtain an initial feature tensor includes: The multi-channel pixel matrix of the target area inside the power metering box is obtained as the original image using an image acquisition device; The original image is scaled to a square input size with equal side lengths; Subtract the mean of the corresponding channel from the pixel values ​​of all training images in the dataset, and divide the difference by the standard deviation of the corresponding channel to convert it into the final initial feature tensor after standardization adjustment of the numerical distribution.

[0028] A 1920×1080 resolution image of the interior of a three-channel (red, green, and blue) energy metering box is acquired. This image is then scaled using bilinear interpolation to a square pixel matrix with a height and width of 640 pixels, resulting in a tensor input of shape 640×640×3. During this scaling process, the horizontal scaling ratio is recorded. =1920 / 640, vertical scaling to restore ratio =1080 / 640.

[0029] The values ​​of each channel in the pixel matrix, ranging from 0 to 255, are divided by 255.0 to map to a floating-point range of 0 to 1. The mean vector μ of the statistically obtained dataset is set to 0.485, 0.456, and 0.406 for the red, green, and blue channels, respectively, and the standard deviation vector σ is set to 0.229, 0.224, and 0.225, respectively. For the channel pixel value at each spatial location in the feature tensor at the i-th row and j-th column... Standardize, For example, when the initial normalized value of the red channel of a certain pixel is 0.6, the final characteristic value of that channel is calculated to be approximately 0.502, which is the difference between 0.6 and 0.485 divided by 0.229.

[0030] By performing element-wise broadcasting operations on the entire pixel matrix, a floating-point initial feature tensor with standardized numerical distribution is generated, reducing the impact of absolute pixel distribution deviation caused by image illumination differences on subsequent feature extraction.

[0031] In an optional embodiment, the step of inputting the initial feature tensor into the backbone network and extracting multi-scale initial feature maps containing semantic information at different levels according to different network depths includes: The initial feature tensor is input into a backbone network containing residual blocks for downsampling feature extraction. As the network depth increases, feature maps are extracted at the exit points of the three set downsampling stages. Parallel hollow spatial pyramid pooling operation is applied to the extracted feature maps at each stage to extract features with different receptive fields, resulting in multi-scale initial feature maps containing bottom-level boundary texture features, mid-level device localization features, and deep-level global semantic features.

[0032] The backbone network is a residual network, comprising an initial convolutional layer, a max-pooling layer, and four feature extraction stages consisting of stacked residual bottleneck blocks. The first stage contains three residual blocks, the second stage contains four residual blocks, the third stage contains six residual blocks, and the fourth stage contains three residual blocks. The input to this backbone network is a preprocessed initial feature tensor, and the output is feature maps at multiple scales containing semantic information at different levels.

[0033] An initial feature tensor of 640×640×3 is input into the residual network. As the features propagate forward in the residual bottleneck layer, feature maps are extracted sequentially at the exits of the second, third, and fourth stages with downsampling rates of 8, 16, and 32, respectively, resulting in a bottom-level feature map of 80×80×512, a middle-level feature map of 40×40×1024, and a deep-level feature map of 20×20×2048.

[0034] The feature maps of the three dimensions mentioned above are independently fed into a parallel dilated spatial pyramid pooling module. The network structure of the dilated spatial pyramid pooling module contains five parallel branches: a 1×1 standard convolution branch, three 3×3 dilated convolution branches with different dilation rates, and a global average pooling branch. The outputs of each branch are concatenated along the channel dimension and then subjected to a 1×1 convolution layer for dimensionality reduction. The input of this module is a feature map of a single scale, and the output is a feature map after the fusion of multi-scale receptive fields.

[0035] Taking a 20×20×2048 deep feature map as an example, the dilated spatial pyramid pooling module is configured with five parallel branches, including a 1×1 standard convolution branch, three 3×3 dilated convolution branches with dilation rates set to 6, 12, and 18 respectively, and a global average pooling branch. The output feature maps of these five branches are concatenated along the channel dimension and then fused into a 20×20×256 deep multi-scale initial feature map through 1×1 convolution dimensionality reduction. The same processing flow is used to process the middle and bottom layer features, uniformly outputting a set of multi-scale initial feature maps with unchanged spatial resolution but a uniform number of channels of 256. The dilation rate parameters corresponding to the three dilated convolution branches are as follows... Figure 2 As shown.

[0036] S2, perform weighted feature fusion based on the multi-scale initial feature map to obtain a fused feature map, input the fused feature map into a detection head containing a classification branch and a regression branch, extract the intermediate layer features of the classification branch and the intermediate layer features of the regression branch respectively, perform channel dimension calibration on the intermediate layer features of the regression branch based on the intermediate layer features of the classification branch, and perform spatial dimension calibration on the intermediate layer features of the classification branch based on the intermediate layer features of the regression branch.

[0037] In one implementation, to achieve weighted feature fusion, a learnable weight scalar parameter is assigned and initialized for each feature map in the multi-scale initial feature maps. These weight parameters are continuously updated and optimized during model training using backpropagation. Bicubic interpolation is used to upsample the deeper feature maps with smaller spatial dimensions, while a 2D convolution operation with a stride of 2 is used to downsample the shallower feature maps with larger spatial dimensions, ensuring that all the initial feature maps to be fused are adjusted to a uniform intermediate spatial resolution. Before element-wise addition, a 1×1 convolution is performed on each feature map to unify the number of channels. Each feature map is multiplied by its corresponding learnable weight parameter processed by a normalized exponential function, and then element-wise feature aggregation is performed along the channel dimension to complete the fusion of multi-scale features, outputting a fused feature map rich in multi-scale contextual information.

[0038] The fused feature map is input into the detection head, which employs a dual-branch decoupled network structure, containing parallel classification and regression branches. Inside the detection head, the fused feature map is input into both the classification and regression branches. In each branch, the fused feature map sequentially passes through three consecutive 3×3 convolutional layers, a batch normalization layer, and a non-linear activation function to obtain classification and regression feature maps that have not yet entered the final output layer. The classification feature map is used as the intermediate layer feature of the classification branch, and the regression feature map is used as the intermediate layer feature of the regression branch.

[0039] The intermediate layer features of the classification branch are compressed into a one-dimensional channel description vector through global average pooling. This vector is then subjected to two fully connected layers for dimensionality reduction and redimensionality increase, and a sigmoid activation function is used to generate a one-dimensional channel attention weight vector. This channel attention weight vector is then broadcast along the channel dimension with the intermediate layer features of the regression branch and an element-wise multiplication operation is performed to enhance the channel feature response in the regression branch that is closely related to the target boundary localization.

[0040] The intermediate layer features of the regression branch are subjected to max pooling and average pooling in the channel dimension to obtain two two-dimensional spatial statistical feature maps. These two spatial statistical feature maps are then concatenated in the channel dimension and passed through a 7×7 convolutional layer and a sigmoid activation function to generate a two-dimensional spatial attention weight matrix. This spatial attention weight matrix is ​​then element-wise multiplied with the intermediate layer features of the classification branch in the spatial dimension, causing the feature extraction of the classification branch to focus on target regions with higher discriminative power.

[0041] In an optional embodiment, the weighted feature fusion based on the multi-scale initial feature map to obtain a fused feature map includes: In the top-down fusion path, high-level semantic features are input into a network structure that includes global average pooling and multilayer perceptron, and are mapped to generate content-driven convolutional kernel parameters. The convolution kernel is used to perform convolution operations on low-level detail features in the downlink branches of adjacent network layers, and a spatial weight map is generated by mapping through a normalized activation function. The spatial weight map is multiplied element-wise with the low-level detail features, and the result of the multiplication is added element-wise with the high-level semantic features that have been upsampled and aligned by channel dimension convolution to obtain the fused feature map.

[0042] The network structure of the multilayer perceptron includes an input layer, a hidden layer, and an output layer connected in sequence. Information is transmitted between layers through a fully connected manner. The input is a one-dimensional feature vector after global spatial compression, and the output is the convolution parameter vector before reshaping.

[0043] Taking a high-level semantic feature of size 20×20×256 and an adjacent low-level detail feature of size 40×40×256 as an example, a global average pooling operation with a kernel size of 20×20 is performed on the high-level semantic feature to compress the spatial dimension into a global feature vector of size 1×1×256, which is then fed into a multilayer perceptron containing two fully connected layers. The output of the multilayer perceptron generates a convolutional kernel weight tensor and a bias vector through a reshaping operation. The spatial scale of the convolutional kernel is set to 3×3, and the number of input and output channels is set to 256. The generated convolutional kernel parameters are applied to the 40×40×256 low-level detail feature. After convolution, the result is input into the sigmoid function, which constrains the response value of the output feature to the range of 0 to 1, generating a spatial weight map of size 40×40×256.

[0044] For example, a weight value of 0.85 at a certain coordinate location indicates that the detailed features at that location are more likely to correspond to the local boundary and texture change features of an area with abnormal visible status or foreign objects within the equipment. Next, the Hadamard product of the low-level detailed features and the spatial weight map is calculated to reduce the influence of irrelevant background features. Simultaneously, the original high-level semantic features are upsampled to 40×40 using bilinear interpolation, and after channel feature alignment using 1×1 convolution, element-wise addition is performed on the result matrix obtained from the previous element-wise multiplication, outputting an enhanced fused feature map with a size of 40×40×256.

[0045] In an optional embodiment, the step of labeling the channel dimension of the intermediate layer features of the regression branch based on the intermediate layer features of the classification branch includes: Global spatial average pooling is performed on the intermediate layer features of the classification branch to obtain the global response value of each channel; The global response value is input into a fully connected network containing a non-linear activation function for feature activation, and after being mapped by the Sigmoid function, an importance gain score in the channel dimension is generated. The importance gain score is multiplied element-wise with the intermediate layer features of the regression branch along the channel dimension to calibrate the channel dimension importance of the intermediate layer features of the regression branch.

[0046] The fully connected network constitutes the feature activation module. The network structure includes a dimension-reducing fully connected layer, a non-linear activation function layer, and a dimension-upgrading fully connected layer connected in sequence. The input of the feature activation module is a one-dimensional vector composed of the global response values ​​of each channel, and the output is a feature activation vector mapped to the original channel dimension.

[0047] Let the intermediate layer feature tensor of the classification branch extracted from the decoupled detection head be... The feature tensor of the intermediate layer in the regression branch is Both have dimensions H×W×C, for example, 40×40×256. Along the spatial dimensions H and W... Perform global spatial average pooling to calculate the average activation value of all pixels in each channel, resulting in a global response vector with dimensions 1×1×256. .

[0048] Will The input is fed into a feature activation module with dimensionality reduction and expansion structures. It sequentially passes through a first fully connected network layer to compress the channel dimension from 256 to 16 dimensions (a reduction ratio of 16). Then, a non-linear mapping characteristic is introduced through a linear rectified activation function, and the input is fed into a second fully connected network layer to restore the original 256 dimensions. Finally, a 1×1×256-dimensional importance gain score vector is calculated using the sigmoid activation function. .

[0049] Will and Each spatial location is multiplied element-wise along the channel direction to enhance the regression feature channel response that helps locate the bounding box and suppress redundant channel responses, thus completing the channel dimension calibration.

[0050] In this embodiment, the predefined detection categories corresponding to the detection head include equipment visible state categories and foreign object categories inside the enclosure. The equipment visible state categories include meter obstruction, abnormal exposed wiring terminals, abnormal switch position status, or other visible state categories that can be directly identified from the enclosure image or identified based on a preset normal state comparison. The foreign object categories inside the enclosure include flying insects, bird nests, honeycomb, leftover construction tools, or other abnormal objects that have entered the enclosure. Meter obstruction refers to a visible obstruction state formed by the meter display area or the meter body area being covered by foreign objects, cables, or other obstructions. Abnormal exposed wiring terminals refer to a visible state in the enclosure image where wiring terminals, exposed conductors, or wiring fixing parts appear to exceed the preset visible area or are not covered by an insulating protective structure. Abnormal switch position status refers to a detected position status of the switch lever, switch indicator, or switch operating part that is inconsistent with the pre-configured normal switch status.

[0051] The final output layer of the classification branch performs category prediction for each spatial location and outputs the category probability of each spatial location corresponding to the predefined detection category. The final output layer of the regression branch performs bounding box parameter prediction for each spatial location and outputs the corresponding shape prediction vector and regression offset. The regression offset includes the relative offset of the center coordinate and the size scaling offset.

[0052] S3. The reference scale and reference aspect ratio are obtained by calculating the shape prediction vector output by the regression branch at each spatial position and the preset reference size matrix. The reference scale and reference aspect ratio are used as the reference for decoding the prediction box size. The prediction box is obtained by decoding the regression offset output by the regression branch. The prediction probability and the prediction box are post-processed based on nonmaximum suppression to obtain the final visual detection result of the equipment status inside the box.

[0053] The final output of the regression branch after dimensionality reduction by the terminal convolutional layer includes the shape prediction vector and regression offset for each spatial grid location on the feature map. Based on the downsampling ratio of the currently processed feature map, a reference size matrix corresponding to the feature map scale is retrieved from the parameter library. The shape prediction vectors output by the regression branch at each spatial location are then subjected to matrix operations with the preset reference size matrix, and the results are positively mapped to calculate the reference scale and reference aspect ratio suitable for each specific grid location feature.

[0054] The prediction box coordinate decoding operation is performed. The absolute coordinates of the grid center points corresponding to each spatial location are used as the center position reference. The horizontal and vertical relative offsets of the center points output by the regression branch are scaled and superimposed according to the reference absolute width and reference absolute height, respectively, to calculate the horizontal and vertical coordinates of the center point of the prediction box at the network input scale. The reference absolute width at the network input scale is obtained by multiplying the calculated reference scale by the square root of the reference aspect ratio. The reference absolute height at the network input scale is obtained by dividing the reference scale by the square root of the reference aspect ratio. The width and height scaling offsets output by the regression branch are further combined to obtain the final width and final height of the prediction box at the network input scale. Subsequently, based on the horizontal and vertical scaling restoration ratios recorded in S1, the center coordinates, final width, and final height of the prediction box at the network input scale are restored to the original image scale to determine the complete spatial boundary of the prediction box.

[0055] After decoding the predicted bounding boxes corresponding to each spatial location based on the output of the regression branch, the category probability output by the classification branch at the corresponding spatial location is associated with the predicted bounding box to obtain the predicted probability of each candidate predicted bounding box. Then, a non-maximum suppression algorithm is performed to remove duplicates based on the candidate predicted bounding boxes and their predicted probabilities.

[0056] Before performing non-maximum suppression, a class determination list is established according to the highest probability category of each candidate predicted bounding box. Within the same class determination list, sorting and overlapping box suppression operations are performed. For each class determination list, invalid candidate predicted bounding boxes with predicted probabilities lower than a preset classification confidence threshold (e.g., 0.5) are first removed. Then, the remaining candidate predicted bounding boxes are sorted in descending order of predicted probability. The candidate predicted bounding box with the highest predicted probability in the current class determination list is selected as the baseline box, and the intersection-union ratio (IUR) between the baseline box and the remaining candidate predicted bounding boxes in the current class determination list is calculated one by one. If the IUR of a candidate predicted bounding box with the baseline box is greater than a preset IUR threshold (e.g., 0.45), the two boxes are determined to be highly overlapping and predicting the same target category. The overlapping candidate predicted bounding box is then removed from the list. The baseline box is moved to the final retention library for storage. The process of selecting the highest probability candidate predicted bounding box from the remaining candidate predicted bounding boxes in the current class determination list and performing the above IUR calculation and overlapping box removal operations is repeated until all candidate predicted bounding boxes in the current class determination list have been traversed.

[0057] The set of predicted box boundary coordinates and the corresponding highest probability category labels retained after the above threshold filtering and deduplication iteration constitute the visual detection result of the equipment status inside the power metering box. The highest probability category labels include the visible status category label of the equipment inside the box or the foreign object category label inside the box.

[0058] In an optional embodiment, the step of calculating the reference scale and reference aspect ratio by operating the shape prediction vector output by the regression branch at each spatial location and a preset reference size matrix, using the reference scale and reference aspect ratio as the decoding reference for the prediction box size, and decoding the prediction box by combining the regression offset output by the regression branch, includes: The intermediate layer features after the regression branch is calibrated are subjected to feature mapping processing to obtain the shape prediction vectors output at each spatial location; The reference size matrix is ​​obtained from the parameter library, which represents the reference scale of the object and the standard proportion of the appearance boundary. The shape prediction vectors output at each spatial location are multiplied by the reference size matrix, and the results of the matrix multiplication are positively mapped to obtain the reference scale and reference aspect ratio corresponding to the current spatial location. Based on the reference scale and the reference aspect ratio, the reference absolute width and reference absolute height are calculated. The grid center coordinates corresponding to each spatial position are used as the center position reference, and the reference absolute width and reference absolute height are used as the size reference. Combined with the center offset and size scaling offset in the regression offset, the position coordinates and size information of the prediction box are decoded.

[0059] For the calibrated regression features with dimensions of 40×40×256, a 3×3 convolution layer is used for spatial feature mapping, outputting a prediction tensor containing shape prior information. Each grid center point outputs a shape prediction vector of dimension k. Assume k=4 and the eigenvector at a certain point contains 0.5, 0.2, 0.8, and -0.1. Retrieve a baseline dimension parameter matrix of size 4 rows and 2 columns from the model parameter library. This serves as the reference dimension matrix. The reference dimension parameter matrix... This matrix is ​​used to map the shape prediction vector to a reference scale and a reference aspect ratio. It serves as a trainable parameter in the object detection network during model training, updating along with the backbone network and detection head parameters via backpropagation based on the detection loss function. Once the model training is complete,... The matrix parameters are fixed and stored in the model parameter library, and are called as fixed parameters during the inference phase.

[0060] For each grid point, the vector With matrix Perform standard matrix multiplication, i.e., multiply a 1x4 matrix by a 4x2 matrix, and apply positive value mapping to the result to calculate the reference parameter vector for the current grid point, including the reference scale. and the reference aspect ratio .

[0061] If the output calculation result is a benchmark scale Equal to 64 and the base aspect ratio If the value is equal to 1.5, then the reference absolute width of the current mesh is calculated using the analytical formula. ≈78.4 pixels, and baseline absolute height ≈52.3 pixels.

[0062] The relative offsets dx and dy of the network output center coordinates and the scaling offsets dw and dh are extracted from the regular mapping layer of the regression branch. These are then combined with the absolute coordinates cx and cy of the grid center after mapping the current feature map back to the network input image. For example, if the center coordinates of a certain grid are cx=160 and cy=160 after mapping back to the network input image, these are substituted into the bounding box decoding formula to calculate the predicted box coordinates, including the center x-coordinate at the network input scale. The center ordinate at the network input scale Width of the predicted bounding box at the network input scale Predicted bounding box height at network input scale Then restore the aspect ratio based on the horizontal scaling. and vertical scaling restoration ratio ,Will and Multiply by respectively ,Will and Multiply by respectively The center coordinates, width, and height of the predicted bounding box at the original image scale are obtained, and a bounding box is constructed based on this to locate visible targets or foreign objects within the box.

[0063] In an optional embodiment, the post-processing of the predicted probability and the predicted bounding box based on nonmaximum suppression to obtain the final visual detection result of the equipment status inside the box includes: Candidate prediction boxes with predicted probabilities lower than the set confidence threshold are filtered out, and a category determination list is established according to the highest probability category of each candidate prediction box. For each category determination list, the remaining candidate prediction boxes are added to the corresponding initial retention set. The candidate prediction boxes in the initial retention set are sorted in descending order according to the prediction probability, and the candidate prediction box with the highest score is taken and moved into the final retention library. Calculate the intersection-union ratio (IUU) of the remaining candidate predicted boxes in the same category judgment list with the candidate predicted box with the highest score, and remove candidate predicted boxes whose IUU is greater than a set overlap threshold; The operation of retrieving the highest-scoring predicted bounding box and removing overlapping predicted bounding boxes is performed repeatedly until the initial retention set is empty. The detection category and bounding box position information in the final retention library are then output as the final visual detection result of the in-box device status.

[0064] During the inference phase, candidate bounding boxes are extracted from the network output, and the highest class classification probability of each candidate box is used as the final prediction probability score. A preset classification confidence threshold of 0.5 is set, and a traversal operation is performed on thousands of candidate boxes in the entire image to filter out candidate bounding boxes with prediction probability scores lower than 0.5.

[0065] Suppose that after filtering a certain category's decision list, 100 candidate bounding boxes remain. These 100 candidate bounding boxes are added to the initial retention set corresponding to that category. Then, using a quicksort algorithm, these 100 bounding boxes are sorted in descending order of their scores, for example, from a high of 0.98 to a low of 0.52. Finally, the candidate bounding box with the highest score in the current category's decision list is removed from the initial retention set and stored in the final retention set.

[0066] For the remaining 99 predicted bounding boxes in the current category decision list, calculate their intersection-union ratio (IU) with the target candidate box. The IU is calculated by dividing the area of ​​the intersection region of the two bounding boxes by the area of ​​their union region. The preset overlap suppression threshold is 0.45. If the calculation detects an IU of 0.60 between a candidate box and the target candidate box, it indicates a high degree of spatial overlap and a high probability that they are detecting the same target; in this case, the redundant candidate box is removed from the initial retention set. After a single round of suppression, extract the remaining candidate box with the highest score from the current category decision list, for example, the candidate box with a score of 0.85, and move it to the final retention set. Repeat the IU calculation and removal steps until the initial retention set is empty.

[0067] The detection categories of each retained prediction box in the final retention library, as well as the coordinates of the upper left and lower right bounding boxes obtained by converting the center coordinates, width, and height of the prediction box, are packaged and output as the final visual inspection results of the equipment status inside the power metering box.

[0068] During the model training phase, a training sample set is constructed, containing images of the inside of the electricity metering box and their annotation information. The annotation information includes labels for the visible state of the equipment, labels for foreign objects inside the box, and the corresponding bounding box locations. The training samples are input into the object detection network. The classification loss is calculated using the class probabilities output by the classification branch and the class labels. The bounding box regression loss is calculated using the predicted bounding boxes decoded by the regression branch and the labeled bounding boxes. A detection loss function is constructed based on the classification loss and the bounding box regression loss. Backpropagation is performed based on the detection loss function to update the backbone network parameters, multi-scale feature fusion weight parameters, detection head parameters, and the baseline size parameter matrix. After model training is complete, the updated network parameters and the baseline size parameter matrix are fixedly saved to the model parameter library. During the inference phase, the fixed model parameters are called to perform visual detection of the equipment status inside the box.

[0069] In the second embodiment, the present invention also proposes a visual inspection system for the status of equipment inside an energy metering box, comprising the following modules: The extraction module is used to acquire images inside the electricity metering box, preprocess the images based on normalization and size scaling to obtain an initial feature tensor, input the initial feature tensor into the backbone network, and extract multi-scale initial feature maps containing semantic information at different levels according to different network depths. The calibration module is used to perform weighted feature fusion based on the multi-scale initial feature map to obtain a fused feature map, input the fused feature map into a detection head containing a classification branch and a regression branch, extract the intermediate layer features of the classification branch and the intermediate layer features of the regression branch respectively, perform channel dimension calibration on the intermediate layer features of the regression branch based on the intermediate layer features of the classification branch, and perform spatial dimension calibration on the intermediate layer features of the classification branch based on the intermediate layer features of the regression branch. The output module is used to calculate the reference scale and reference aspect ratio by performing calculations on the shape prediction vector output by the regression branch at each spatial location and the preset reference size matrix. The reference scale and reference aspect ratio are used as the reference for decoding the prediction box size. The prediction box is obtained by decoding the regression offset output by the regression branch. The prediction probability and the prediction box are post-processed based on nonmaximum suppression to obtain the final visual inspection result of the equipment status inside the box.

[0070] The above description represents the preferred embodiments of the present invention. It should be noted that, for those skilled in the art, various improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A visual inspection method for the status of equipment inside an electricity metering box, characterized in that, include: An image of the inside of the electricity metering box is acquired. The image is preprocessed based on normalization and size scaling to obtain an initial feature tensor. The initial feature tensor is input into the backbone network, and multi-scale initial feature maps containing semantic information at different levels are extracted according to different network depths. Weighted feature fusion is performed based on the multi-scale initial feature map to obtain a fused feature map. The fused feature map is then input into a detection head containing a classification branch and a regression branch. Intermediate layer features of the classification branch and the regression branch are extracted respectively. The channel dimension of the intermediate layer features of the regression branch is calibrated based on the intermediate layer features of the classification branch, and the spatial dimension of the intermediate layer features of the classification branch is calibrated based on the intermediate layer features of the regression branch. The reference scale and reference aspect ratio are obtained by calculating the shape prediction vector output by the regression branch at each spatial location and the preset reference size matrix. The reference scale and reference aspect ratio are used as the reference for decoding the prediction box size. The prediction box is obtained by decoding the regression offset output by the regression branch. The prediction probability and the prediction box are post-processed based on nonmaximum suppression to obtain the final visual detection result of the equipment status inside the box.

2. The method according to claim 1, characterized in that, The process of acquiring an image of the interior of the electricity metering box, and preprocessing the image based on normalization and scaling to obtain an initial feature tensor, includes: The multi-channel pixel matrix of the target area inside the power metering box is obtained as the original image using an image acquisition device; The original image is scaled to a square input size with equal side lengths; Subtract the mean of the corresponding channel from the pixel values ​​of all training images in the dataset, and divide the difference by the standard deviation of the corresponding channel to convert it into the final initial feature tensor after standardization adjustment of the numerical distribution.

3. The method according to claim 2, characterized in that, The step of inputting the initial feature tensor into the backbone network and extracting multi-scale initial feature maps containing semantic information at different levels according to different network depths includes: The initial feature tensor is input into a backbone network containing residual blocks for downsampling feature extraction. As the network depth increases, feature maps are extracted at the exit points of the three set downsampling stages. Parallel hollow spatial pyramid pooling operation is applied to the extracted feature maps at each stage to extract features with different receptive fields, resulting in multi-scale initial feature maps containing bottom-level boundary texture features, mid-level device localization features, and deep-level global semantic features.

4. The method according to claim 1, characterized in that, The weighted feature fusion based on the multi-scale initial feature map to obtain the fused feature map includes: In the top-down fusion path, high-level semantic features are input into a network structure that includes global average pooling and multilayer perceptron, and are mapped to generate content-driven convolutional kernel parameters. The convolution kernel is used to perform convolution operations on low-level detail features in the downlink branches of adjacent network layers, and a spatial weight map is generated by mapping through a normalized activation function. The spatial weight map is multiplied element-wise with the low-level detail features, and the result of the multiplication is added element-wise with the high-level semantic features that have been upsampled and aligned by channel dimension convolution to obtain the fused feature map.

5. The method according to claim 1, characterized in that, The step of labeling the channel dimension of the intermediate layer features of the regression branch based on the intermediate layer features of the classification branch includes: Global spatial average pooling is performed on the intermediate layer features of the classification branch to obtain the global response value of each channel; The global response value is input into a fully connected network containing a non-linear activation function for feature activation, and after being mapped by the Sigmoid function, an importance gain score in the channel dimension is generated. The importance gain score is multiplied element-wise with the intermediate layer features of the regression branch along the channel dimension to calibrate the channel dimension importance of the intermediate layer features of the regression branch.

6. The method according to claim 1, characterized in that, The step of calculating the reference scale and reference aspect ratio by performing calculations on the shape prediction vector output by the regression branch at each spatial location and a preset reference size matrix, using the reference scale and reference aspect ratio as the decoding reference for the prediction box size, and decoding the prediction box by combining the regression offset output by the regression branch, includes: The intermediate layer features after the regression branch is calibrated are subjected to feature mapping processing to obtain the shape prediction vectors output at each spatial location; The reference size matrix is ​​obtained from the parameter library, which represents the reference scale of the object and the standard proportion of the appearance boundary. The shape prediction vectors output at each spatial location are multiplied by the reference size matrix, and the results of the matrix multiplication are positively mapped to obtain the reference scale and reference aspect ratio corresponding to the current spatial location. Based on the reference scale and the reference aspect ratio, the reference absolute width and reference absolute height are calculated. The grid center coordinates corresponding to each spatial position are used as the center position reference, and the reference absolute width and reference absolute height are used as the size reference. Combined with the center offset and size scaling offset in the regression offset, the position coordinates and size information of the prediction box are decoded.

7. The method according to claim 1, characterized in that, The post-processing of the predicted probability and the predicted bounding box based on nonmaximum suppression yields the final visual detection result of the equipment status inside the box, including: Candidate prediction boxes with predicted probabilities lower than the set confidence threshold are filtered out, and a category determination list is established according to the highest probability category of each candidate prediction box. For each category determination list, the remaining candidate prediction boxes are added to the corresponding initial retention set. The candidate prediction boxes in the initial retention set are sorted in descending order according to the prediction probability, and the candidate prediction box with the highest score is taken and moved into the final retention library. Calculate the intersection-union ratio (IUU) of the remaining candidate predicted boxes in the same category judgment list with the candidate predicted box with the highest score, and remove candidate predicted boxes whose IUU is greater than a set overlap threshold; The operation of retrieving the highest-scoring predicted bounding box and removing overlapping predicted bounding boxes is performed repeatedly until the initial retention set is empty. The detection category and bounding box position information in the final retention library are then output as the final visual detection result of the in-box device status.

8. A visual inspection system for the status of equipment inside an electricity metering box, characterized in that, include: The extraction module is used to acquire images inside the electricity metering box, preprocess the images based on normalization and size scaling to obtain an initial feature tensor, input the initial feature tensor into the backbone network, and extract multi-scale initial feature maps containing semantic information at different levels according to different network depths. The calibration module is used to perform weighted feature fusion based on the multi-scale initial feature map to obtain a fused feature map, input the fused feature map into a detection head containing a classification branch and a regression branch, extract the intermediate layer features of the classification branch and the intermediate layer features of the regression branch respectively, perform channel dimension calibration on the intermediate layer features of the regression branch based on the intermediate layer features of the classification branch, and perform spatial dimension calibration on the intermediate layer features of the classification branch based on the intermediate layer features of the regression branch. The output module is used to calculate the reference scale and reference aspect ratio by performing calculations on the shape prediction vector output by the regression branch at each spatial location and the preset reference size matrix. The reference scale and reference aspect ratio are used as the reference for decoding the prediction box size. The prediction box is obtained by decoding the regression offset output by the regression branch. The prediction probability and the prediction box are post-processed based on nonmaximum suppression to obtain the final visual inspection result of the equipment status inside the box.

9. The system according to claim 8, characterized in that, The process of acquiring an image of the interior of the electricity metering box, and preprocessing the image based on normalization and scaling to obtain an initial feature tensor, includes: The multi-channel pixel matrix of the target area inside the power metering box is obtained as the original image using an image acquisition device; The original image is scaled to a square input size with equal side lengths; Subtract the mean of the corresponding channel from the pixel values ​​of all training images in the dataset, and divide the difference by the standard deviation of the corresponding channel to convert it into the final initial feature tensor after standardization adjustment of the numerical distribution.

10. The system according to claim 8, characterized in that, The step of inputting the initial feature tensor into the backbone network and extracting multi-scale initial feature maps containing semantic information at different levels according to different network depths includes: The initial feature tensor is input into a backbone network containing residual blocks for downsampling feature extraction. As the network depth increases, feature maps are extracted at the exit points of the three set downsampling stages. Parallel hollow spatial pyramid pooling operation is applied to the extracted feature maps at each stage to extract features with different receptive fields, resulting in multi-scale initial feature maps containing bottom-level boundary texture features, mid-level device localization features, and deep-level global semantic features.

Images