A substation intelligent operation and maintenance method based on an adaptive path aggregation network

Abstract

The application discloses a transformer substation intelligent operation and maintenance method based on an adaptive path aggregation network, relates to the technical field of target detection, and is used for realizing transformer substation operation and maintenance multi-target detection. The method comprises the following steps: a backbone network is used for extracting feature maps of different scales from a transformer substation operation and maintenance target detection image; in a neck network, an adaptive path aggregation pyramid network fuses multi-scale features extracted by the backbone network, a channel attention fusion module for relieving semantic information conflicts caused by directly fusing different scale features, and a cross-level path aggregation module for distributing additional feature levels to obtain an optimal instance gradient back propagation path; and a head network adopts a decoupling head to predict the multi-scale features fused by the adaptive path aggregation pyramid network. The method can improve the robustness of the network to scales and plays a key role in coping with large-scale scale changes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of power grid operation and maintenance target detection, and particularly relates to a substation intelligent operation and maintenance method based on an adaptive path aggregation network. BACKGROUND

[0002] Power grid operation and maintenance is a crucial link for ensuring safe, reliable, economic and efficient operation of the power grid. The traditional manual inspection method not only has a high labor intensity, but also has a serious dependence on the quality of the inspection personnel. The phenomenon of "missing inspection", "delayed inspection" and "incompetent inspection" often occurs, and the detection data cannot be accurately and timely input into the management information system. In high-voltage and some harsh weather conditions, manual inspection also has a great safety hazard, which can easily lead to accidents. With the continuous expansion of the power grid, the contradiction between heavy power grid inspection tasks and the shortage of operation and maintenance personnel is increasingly prominent. Therefore, it is urgent to develop intelligent operation and maintenance technology to replace manual inspection.

[0003] In recent years, thanks to the continuous improvement of computing power and storage capacity, deep learning technology has developed rapidly and achieved great success in image segmentation, image fusion, image classification, object recognition, and object detection. Inspired by these works, researchers have begun to explore the application of deep learning technology to solve substation operation and maintenance target detection problems, including the application of deep learning to identify and locate circuit breakers and insulators, fault analysis of insulators, detection of key points on substation instrument panels, status recognition of disconnectors, and status recognition of circuit breaker switches. For example, the document "A Segmentation-Based Multitask Learning Approach for Isolating Switch State Recognition in High-Speed Railway Traction Substation" (X. Lu et al., in IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 9, pp. 15922-15939, Sept. 2022, doi: 10.1109 / TITS.2022.3146338.) proposes an automatic isolating switch segmentation and state recognition framework named ISSSR-Net to accurately locate and recognize the state of the isolating switch. The framework uses multi-task learning and is divided into two stages: first, an isolating switch segmentation network is proposed to accurately segment the isolating switch at the pixel level. Second, to improve the accuracy of state recognition, the segmentation map extracted by the ISR-Net and the feature map extracted by the shared backbone network are fed into the ISR-Net isolating switch recognition network. In addition, a global context block is integrated into the ISR-Net to further improve the accuracy of state recognition.

[0004] However, the above method is for single target detection, and there is a lack of multi-target detection method for substation operation and maintenance detection. In addition, in substation operation and maintenance detection, there is a large scale difference between small targets such as foreign matter nests and closed switches and large targets such as abnormal box door closure. Therefore, there is an urgent need for an intelligent operation and maintenance method that can adaptively recognize multi-dimensional target detection in substations to better solve the problem of multiple target types and large scale differences in substation operation and maintenance. SUMMARY

[0005] The purpose of the present application is to provide a substation intelligent operation and maintenance method based on an adaptive path aggregation network to solve the problem of lack of multi-target detection in substation operation and maintenance detection in the prior art.

[0006] To solve the above technical problems, the present application specifically provides the following technical solutions:

[0007] The substation intelligent operation and maintenance method based on the adaptive path aggregation network is used to realize substation operation and maintenance multi-objective detection, and has the characteristics that,

[0008] Step (1), the backbone network is used to extract feature maps of different scales from the substation operation and maintenance target detection image;

[0009] Step (2), in the neck network, the adaptive path aggregation pyramid network fuses the multi-scale features extracted by the backbone network in the following manner: the feature maps of different levels extracted by the backbone network are input into the adaptive path aggregation pyramid network, the adaptive path aggregation pyramid network transmits high-level semantic information used for classification from top to bottom, the adaptive path aggregation pyramid network transmits detailed information used for regression from bottom to top, and the adaptive path aggregation pyramid network fuses parameters of different levels by adaptation to alleviate semantic conflicts and extract features;

[0010] The adaptive path aggregation pyramid network includes a channel attention fusion module and a cross-level path aggregation module, the channel attention fusion module is used to alleviate the semantic information conflict caused by directly fusing different scale features, and the cross-level path aggregation module derives the optimal instance gradient backpropagation path by allocating additional feature levels;

[0011] Step (3), the head network adopts a decoupled head to predict the multi-scale features fused by the adaptive path aggregation pyramid network.

[0012] Further, in step (1), the backbone network uses CSPDarkNet and SwinTransformer-Tiny based on CNN as the backbone network.

[0013] Further, in step (2), in the adaptive path aggregation pyramid network, different backbone networks are used to extract feature maps of different scales from the input image, and the number of channels of each level of feature map is increased by 2 times of the previous level.

[0014] Further, in step (2), the channel attention fusion module includes a complex feature fusion module for filtering conflict information and redundant information to alleviate the semantic conflict caused by direct fusion; the complex feature fusion module is composed of channel splicing and channel purification, the channel splicing can obtain fused features without losing information, and the channel purification is used to generate adaptive weights in the channel dimension.

[0015] Further, the cross-level path aggregation module comprises a feature path aggregation module, which realizes the change of the propagation path by extruding, scaling, integrating and activating the fused features output by the complex feature fusion module.

[0016] Further, the specific manner in which the feature path aggregation module extrudes, scales, integrates and activates the fused features output by the complex feature fusion module is as follows: extrusion: input the first features generated by the bottom layer, the intermediate layer and the top layer in the fused features, compress the three levels of first features into the same number of channels to obtain corresponding second features; scaling: adjusting all features in the second features to the same resolution; integration: adding an additional weight to each input when fusing multi-scale features, and using weight normalization to limit the range value of each weight to obtain integrated third features; activation: using an activation function on the third features and introducing a nonlinear transformation to obtain fourth features.

[0017] Further, the feature path aggregation module changes the forward and backward paths by aggregating features of different levels, and the levels can adaptively adjust the parameters of other levels when backpropagation.

[0018] Further, in the first features, the number of feature channels generated by the top layer is greater than that generated by the bottom layer, and the resolution of the features generated by the top layer is less than that generated by the bottom layer.

[0019] Compared with the prior art, the present application has the following beneficial effects:

[0020] (1) The adaptive path aggregation target detection network formed by the three steps of the backbone network, the neck network and the head network realizes multi-target detection in the intelligent operation and maintenance of the substation.

[0021] (2) The adaptive path aggregation pyramid network can alleviate the semantic conflict problem caused by directly fusing features of different scales. Through the complex feature fusion module, the fusion of multi-scale features is guided, and the feature channels with conflicting information are filtered. Secondly, the adaptive path aggregation pyramid network performs cross-level path aggregation between the neck network and the head network. By changing the backpropagation path, the levels of the cross-level path aggregation module can adaptively update the parameters of other levels when backpropagation, so that other levels can learn the feature representation of the appropriate scale range, and the robustness of the network to scale is improved. In the adaptive path aggregation pyramid network, the channel attention enhanced fusion module with adaptive fusion and the cross-level path aggregation module with adaptive selection of the backpropagation path play a key role in dealing with large-scale scale changes. BRIEF DESCRIPTION OF DRAWINGS

[0022] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of the overall architecture of the intelligent operation and maintenance method for substations based on adaptive path aggregation network provided by the present invention.

[0024] Figure 2 This is a schematic diagram of the neck network provided by the present invention.

[0025] Figure 3 A schematic diagram of the channel attention fusion module structure provided by the present invention.

[0026] Figure 4 The schematic diagram shows the structure of the feature path aggregation module provided by this invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] like Figure 1 As shown, this invention provides a substation intelligent operation and maintenance method based on an adaptive path aggregation network:

[0029] The backbone network is used to extract feature maps of different scales and levels from substation operation and maintenance target detection images;

[0030] In the neck network, the adaptive path aggregation pyramid network fuses the multi-scale features extracted by the backbone network in the following way: the feature maps at all levels extracted by the backbone network are input into the adaptive path aggregation pyramid network. The adaptive path aggregation pyramid network passes high-level semantic information for classification from top to bottom and detailed information for regression from bottom to top. The adaptive path aggregation pyramid network mitigates semantic conflicts and extracts features by adaptively fusing parameters at different levels.

[0031] The head network uses a decoupled head to predict the multi-scale features fused by the adaptive path aggregation pyramid network.

[0032] In the adaptive path aggregation pyramid network, different backbone networks are used to extract feature maps of different levels from the substation operation and maintenance target detection image, and the number of channels of each level of feature map is increased by 2 times of the upper level in turn.

[0033] The substation intelligent operation and maintenance method based on the adaptive path aggregation network can flexibly extract features using different backbone networks. In the present application, CSPDarkNet and Swin Transformer-Tiny (Swin-T) based on CNN are used as backbone networks to extract feature maps of different levels {C1, C2, C3, C4, C5} from the input image, and the resolutions of the five feature maps are times of the original picture in turn. The number of channels is increased by 2 times of the upper level in turn. Swin-T can achieve better detection performance than CSPDarkNet.

[0034] The output feature maps of the first layer of the top-down and bottom-up pyramid are represented as P l , and l = 3, 4, 5 are set. For an input image of size W x H, the resolution of P l Before being input into the head network, the feature paths of other levels are aggregated by the cross-level path aggregation module at the 3rd, 4th and 5th layers of the pyramid to obtain the aggregated feature maps {P t3 , P t4 , P t5}, so as to ensure that instances of different scales are selected to the best back propagation path.

[0035] The head network uses a decoupled head to predict the features fused by the adaptive path aggregation pyramid network.

[0036] Exemplarily, the detection head uses a decoupled head of YOLOX with minor changes.

[0037] Considering that the adaptive path aggregation pyramid network outputs features compressed by 1 x 1 convolution, the first 1 x 1 convolution of the YOLOX decoupled head is removed, and all subsequent convolution layers in the two parallel branches for classification and regression are retained. For the output feature P ti ​(i = 3, 4, 5), two parallel branches are used, each branch has two 3x3 convolutional layers, one branch is a classification branch (Cls.) for judging the target class, the other branch contains an IoU branch (Obj.) for judging whether the feature point has a target and a regression branch (Reg.) for judging the regression parameters of the prediction box. In addition, a 3x3 convolutional layer is added on each branch for final prediction. In the three branches, different loss functions are used to supervise the training of the network, the classification branch and the IoU branch use the cross-entropy loss function, and the regression branch uses the L1 loss function.

[0038] The adaptive path aggregation pyramid network comprises the following modules:

[0039] The channel attention fusion module is used to alleviate the semantic information conflict caused by direct fusion of different scale features.

[0040] The cross-level path aggregation module obtains the optimal instance gradient backpropagation path by assigning additional feature levels.

[0041] The adaptive path aggregation pyramid network mainly includes two sub-tasks, namely alleviating semantic conflict and correcting the backpropagation path, which are realized by the channel attention fusion module and the cross-level path aggregation module respectively.

[0042] The channel attention fusion module comprises:

[0043] The complex feature fusion module is used to filter conflict information and redundant information to alleviate the semantic conflict caused by direct fusion.

[0044] The complex fusion module is composed of channel splicing and channel purification, the channel splicing can obtain the fused features without losing information, and the channel purification is used to generate adaptive weights in the channel dimension.

[0045] In order to alleviate the semantic conflict caused by direct fusion, the channel attention fusion module structure is as follows Figure 3As shown, for filtering conflict information and redundant information, preventing instance features from being overwhelmed by conflict information. The channel attention fusion module is composed of two parts: channel splicing and channel purification. Channel splicing splices multi-scale feature maps U and D together in the channel dimension to obtain the fused feature map X without losing information. Channel purification is used to generate adaptive weights in the channel dimension, adjust the weights of different channels, adaptively select the main feature channels, filter the feature channels with conflict information, and guide the feature map to learn in a more critical direction. Specifically, first, the fused feature map X is mapped into a one-dimensional feature vector with a size of 1x1x2C (2C is the number of channels of the feature map X) through global average pooling, and then a convolution kernel with a size of 1xk is used to capture local cross-channel interaction information and activated through a sigmoid function, thereby obtaining the attention of each channel of the feature map X. Finally, multiply each channel's attention by the original feature X according to the broadcast mechanism. The working principle of the channel attention fusion module can be summarized as follows:

[0046] X = concat[U, D]

[0047]

[0048] s = σ(Conv 1×k (z))

[0049]

[0050] where X c (i, j) represents the cth channel X c of the fused feature map at the ith row and jth column, and s represents the sigmoid operation. k is the kernel size of the 1D convolution, which is used to determine the coverage of the captured local cross-channel interaction. A larger k value can obtain more global cross-channel interaction information, but it will also introduce redundant information. In order to balance the two, experiments with different k values are set. Considering that the coverage of local cross-channel interaction is generally proportional to the number of channels, the size of k is determined as follows:

[0051] k = τ·log2 C

[0052] where C is the number of channels of the input feature, and τ is a proportional adjustment factor that can be used to determine the range of local cross-channel interaction by adjusting the size of τ.

[0053] In the cross-level path aggregation module of the neck network, the adaptive fusion layer obtained by the channel attention fusion module guides the fusion of multi-scale features. The channel attention fusion module focuses on the relationship between different channels, enabling the model to automatically learn the importance of each channel of the fused features to suppress the introduction of conflicting information, and only a small number of parameters need to be added to improve performance. The fine features of different scales fused by the adaptive path aggregation pyramid network improve the robustness to scale variance without compromising the performance of large-scale instance detection.

[0054] The cross-level path aggregation module comprises:

[0055] The feature path aggregation module: by squeezing, scaling, integrating, and activating the output features of the complex feature fusion module, the direction of the propagation path is changed.

[0056] Instance objects of the same category may have different scales. During backpropagation, the feature selection based on instance scale selects the layer number of the multi-scale feature fusion network to update, resulting in sparse distribution of feature representations of the same instance in each level of the pyramid network. To concentrate the feature representations of the same category as much as possible, the feature path aggregation module is added before the output features of the channel attention fusion module are sent to the detection head, including squeezing, scaling, integrating, and activating the output features. By changing the backpropagation path, the level with the cross-level path aggregation module can adaptively update the parameters of other levels during backpropagation, allowing other levels to learn feature representations in the appropriate scale range and improving the detection performance of other levels for instance targets of the corresponding scale. The path aggregation process is described in detail below through the structure of the cross-level path aggregation module.

[0057] As shown in Figure 4 , the specific way in which the feature path aggregation module squeezes, scales, integrates, and activates the output features of the complex feature fusion module is as follows:

[0058] Squeezing: input the first features generated by the bottom layer, the middle layer, and the top layer in the fused features, respectively, and compress the three levels of first features to the same number of channels to obtain the corresponding second features.

[0059] In this step, the input features P3, P4, and P5 are generated by the bottom layer, the middle layer, and the top layer of the multi-scale feature fusion, respectively. They represent features of different levels, and the resolution and the number of channels are inconsistent. From high to low, the number of channels of the top layer is large and the resolution is small, and the number of channels of the bottom layer is small and the resolution is large. Therefore, before path aggregation, the number of channels of the three levels of features is unified to the same number of channels, for example, the number of channels of P4, to obtain the corresponding features P c3 , Pc4 and P c5 Channel compression is achieved by a 1x1 convolutional layer. It is noted that the cross-level path aggregation module does not necessarily process all layer features for multi-scale feature fusion, but can only use the layer features and adjacent layer features as input features.

[0060] Scaling: adjusting all features in the second features to the same resolution.

[0061] The same as the previous step, since the resolutions of the three layer features P3, P4 and P5 output by the multi-scale feature fusion are inconsistent, the features P c3 , P c4 and P c5 obtained in step 1 need to be adjusted to the same resolution. For example, the resolution of the intermediate resolution, i.e. P c4 , so as to obtain the scaled features P s3 , P s4 and P s5 . In this case, P s4 =P c4 holds. The up-sampling uses the CARAFE up-sampling method. CARAFE can use adaptive and optimized reorganization kernels at different positions to achieve better performance than mainstream up-sampling operations such as interpolation and deconvolution. The down-sampling is achieved by a 3x3 convolution with a step of 2.

[0062] Integration: an additional weight is added to each input during multi-scale feature fusion, and weight normalization is used to limit the range of each weight value to obtain the integrated third features.

[0063] Traditional feature fusion methods include element-wise addition and Concat tensor splicing, which treat all input features equally. In order to enable the hierarchical level with redundant semantic information to adaptively adjust the parameters of other hierarchical levels during gradient backpropagation, adaptive path aggregation is achieved. An additional weight is added to each input during feature fusion, and the network is allowed to learn the importance of each input feature. However, the range of weight values is unbounded, which can cause unstable training. Therefore, weight normalization is used to limit the range of each weight value, and the integrated features P g can be written as:

[0064]

[0065]

[0066] wherein the weights w3, w4 and w5 are learnable weights, and a relu is added after each w i to ensure that w i≥ 0, e = 1e-6 is a minimum value to avoid denominator being zero.

[0067] Activation: the fourth feature is obtained after using an activation function and introducing a nonlinear transformation on the third feature.

[0068] Considering that the foregoing operations are linear, in order to improve the representation ability of the model, P g The Swish activation function is used to introduce a nonlinear transformation to obtain the feature P t , and the specific operation is as shown below:

[0069] P ti = P gi · σ(P gi )

[0070] Wherein P ti represents the output P i of the FPA-P t used in the i-th (i = 3, 4, 5) layer, and sigma represents a sigmoid operation.

[0071] The cross-layer path aggregation module changes the forward and backward paths by aggregating features of different layers, so that the layer to which the cross-layer path aggregation module is applied can adaptively adjust the parameters of other layers during back propagation, so that the feature representation of instances of the same category is relatively concentrated. However, during the forward propagation process, the path aggregation will introduce redundant information of other layers, which reduces the detection performance of the layer, so the cross-layer path aggregation module cannot be widely applied to all layers. Therefore, by designing an overall structure of four different layers and different aggregation modes, the influence of the concentrated distribution of feature representation in each layer and the introduction of redundant information on the detection performance is balanced.

[0072] In the first feature, the number of feature channels generated by the top layer is greater than the number of feature channels generated by the bottom layer, and the resolution of the feature generated by the top layer is less than the resolution of the feature generated by the bottom layer.

[0073] Exemplarily, the adaptive path aggregation pyramid network establishes a pyramid structure among the three feature layers {C3, C4, C5} extracted by the backbone network through a 1x1 convolution, an up-sampling block, a CSP Block, a 3x3 convolution for down-sampling, and a channel attention fusion module and a cross-level path aggregation module, etc. The pyramid structure first realizes top-down path enhancement. For the low-resolution feature map, the channel number is first reduced by a 1x1 convolution, and then the spatial resolution is up-sampled by 2 times (using nearest neighbor up-sampling). Then the up-sampled feature is fused with the higher resolution feature connected horizontally through the channel attention fusion module, and the channel number is reduced through the channel attention fusion module, so as to reduce the calculation amount and memory occupation of the model. Repeat this process to get refined features. In order to improve the learning ability of CNN and alleviate the problem of repeated calculation of gradient information, the CSP Block structure is composed of two parts: one part is calculated by a 1x1 convolution for reducing the channel number and a Conv Block with a bottleneck structure (structure as shown in the upper right corner) Figure 2 In addition, the Conv Block with a bottleneck structure uses a 1x1 convolution to reduce the channel number and uses a 3x3 convolution to expand the channel number. At the same time, in the pyramid structure, an up-down expansion path is also constructed, starting from the high-resolution feature map of the lowest layer, gradually approaching the low-resolution feature map. For the high-resolution feature map, first pass through the 3x3 convolution layer, step 2 down-sampling to reduce the spatial resolution. Then, the down-sampled feature is fused with the lower resolution feature connected horizontally through the channel attention fusion module, and the output feature {P3, P4, P5} of the pyramid is obtained through the CSP Block. Finally, the features of each level are adaptively aggregated through the cross-level path aggregation module to obtain the final output {P t3 , P t4 , P t5} of the adaptive path aggregation pyramid network, and send them to the head network for prediction.

[0074] Specifically, the performance of the substation intelligent operation and maintenance method based on the adaptive path aggregation network of the present application and other multi-object detection methods on the SOM dataset and PASCAL VOC2012 is compared. At the same time, in order to make a fair comparison, the models are all trained and tested in a single scale, and the multi-scale feature fusion modules of all models are based on the path aggregation network (PANet) and its variants. As shown in Tables 1 and 2, two groups of experiments are compared on each dataset: comparative experiments based on CNN backbone network and comparative experiments based on Transformer backbone network, each group of experiments covering two-stage detectors and one-stage detector methods, and the comparative experiments based on Transformer backbone network are experiments in which the backbone networks of all comparative methods are replaced with Swin Transformer-Tiny (Swin-T).

[0075] (1) Comparative experiments based on CNN backbone network

[0076] As can be seen from Tables 1 and 2, on the SOM dataset, YOLOX achieves the best detection accuracy among all comparative methods, with an mAP value of 71.7%. On the PASCAL VOC2012 dataset, PAA achieves the best detection accuracy, with an mAP value of 68.6%. These results are 3.6% and 1.8% lower than the results obtained by the APA-Detector proposed by us, respectively. In terms of parameter quantity and computational quantity, the adaptive path aggregation object detection network (APA-Detector) is only 1.09M and 1.7G higher than YOLOX. Compared with PAA, although the APA-Detector is 1.97M higher in parameter quantity, it is much lower than PAA in computational quantity, with a reduction of 34.29G. Therefore, it is shown that the APA-Detector can greatly improve the detection accuracy at a lower cost of parameter quantity and computational quantity under the CNN-based backbone network. In addition, on the SOM dataset, the APA-Detector achieves the best AP s and AP l on the SOM dataset. The difference is that on the PASCAL VOC2012 dataset, the APA-Detector achieves the best AP m on the SOM dataset, and the AP on AP s and AP l is only 0.3% and 0.7% lower than the best AP. This is because the AF Module in the APA-Detector can effectively filter out conflicting information from similar targets in complex backgrounds, so the APA-Detector performs better on the SOM image dataset in complex scenes.

[0077] (2) Comparative experiments based on Transformer backbone network

[0078] As can be seen from Table 1 and Table 2, among all the comparative methods, Dynamic RCNN achieves the best detection accuracy on SOM and PASCAL VOC2012 datasets, with mAP values of 75.3% and 72.4%, respectively. In comparison, the APA-Detector proposed by us achieves better detection accuracy and is much lower in parameter quantity and computational quantity than Dynamic RCNN. On the SOM dataset, the mAP value of the APA-Detector reaches 78.0%, which is 2.7% higher than that of Dynamic RCNN, and the parameter quantity and computational quantity are 43.38M and 59.90G, respectively, which are 4.98M and 42.07G lower than those of Dynamic RCNN. On the PASCAL VOC2012 dataset, the mAP value of the APA-Detector reaches 72.9%, which is 0.5% higher than that of Dynamic RCNN, and the parameter quantity and computational quantity are 43.38M and 59.91G, respectively, which are 5.0M and 42.08G lower than those of Dynamic RCNN. The APA-Detector based on the Transformer-based backbone network can greatly improve the detection accuracy and performance. In addition, on the SOM dataset, the APA-Detector achieves the best AP in AP s and AP l On the PASCAL VOC2012 dataset, the APA-Detector achieves the best AP in AP s and AP m , and the AP in AP l is only 0.8% lower than the best AP. Therefore, it is shown that the APA-Detector can well detect small targets and large targets, thereby verifying the effectiveness of the FPA Module proposed by us.

[0079] As can be seen from the above two groups of experiments, for the same detector, the Transformer-based backbone network is not only 3% to 5% higher than the CNN-based backbone network in mAP, but also much lower than the CNN-based backbone network in parameter quantity and FLOPs, for example, in the SOM dataset, the two-stage detector Dynamic RCNN uses the ResNet101 backbone network, with mAP of 71.3%, parameter quantity and computational quantity of 63.73M and 131.42G, respectively. When the Swin-T backbone network is used, the mAP is increased by 4.0%, and the parameter quantity and computational quantity are reduced by 15.37M and 29.45G, respectively. Therefore, the best structure proposed by us is to use the Swin Transformer-Tiny as the backbone network to extract multi-scale features of the APA-Detector.

[0080] Table 1: Comparison with other advanced methods on the SOM dataset

[0081]

[0082]

[0083] Table 2: Comparison with other advanced methods on the PASCAL VOC 2012 dataset

[0084]

[0085]

[0086] The adaptive path aggregation target detection network formed by the three steps of the backbone network, the neck network and the head network realizes multi-target detection in the intelligent operation and maintenance of the transformer substation. At the same time, a new adaptive path aggregation pyramid network is designed, which can relieve semantic conflict and correct the back propagation path, and combines the attention enhanced fusion module and the feature path aggregation module with the top-down and bottom-up path aggregation network. The adaptive path aggregation pyramid network can relieve the semantic conflict problem caused by directly fusing different scale features. Through the complex feature fusion module, the fusion of multi-scale features is guided, and the feature channels with conflicting information are filtered. Secondly, the adaptive path aggregation pyramid network performs cross-level path aggregation between the neck network and the head network. By changing the back propagation path, the level of the cross-level path aggregation module can adaptively update the parameters of other levels during back propagation, so that other levels can learn the feature representation of the appropriate scale range, and the robustness of the network to scale is improved. In the adaptive path aggregation pyramid network, the channel attention enhanced fusion module with adaptive fusion and the cross-level path aggregation module with adaptive selection of the back propagation path play a key role in dealing with large-scale scale changes.

[0087] The application can be implemented in other specific forms without departing from the spirit or essential characteristics thereof. The described embodiments are considered merely illustrative in all respects, not restrictive, for example:

[0088] 1) The model structure used in the backbone network proposed by the application is not limited to that used in the embodiments;

[0089] 2) The number of channels of the adaptive path aggregation pyramid network in the neck network proposed by the application is not limited to that used in the embodiments;

[0090] 3) The head network proposed by the application is not limited to the decoupled head of YOLOX used in the embodiments.

[0091] Therefore, the scope of the application is indicated by the appended claims rather than the description above. All changes that come within the meaning and range of equivalency of the claims are to be embraced within their scope.

Claims

1. A substation intelligent operation and maintenance method based on adaptive path aggregation network, used to realize multi-target detection in substation operation and maintenance, characterized in that, Step (1): The backbone network is used to extract feature maps of different scales from the substation operation and maintenance target detection images. Step (2), in the neck network, the adaptive path aggregation pyramid network fuses the multi-scale features extracted by the backbone network in the following way: the feature maps at all levels extracted by the backbone network are input into the adaptive path aggregation pyramid network. The adaptive path aggregation pyramid network transmits high-level semantic information for classification from top to bottom and detailed information for regression from bottom to top. The adaptive path aggregation pyramid network alleviates semantic conflicts and extracts features by adaptively fusing parameters at different levels. The adaptive path aggregation pyramid network includes a channel attention fusion module and a cross-level path aggregation module. The channel attention fusion module is used to alleviate the semantic information conflict caused by directly fusing features of different scales. The cross-level path aggregation module obtains the optimal instance gradient backpropagation path by allocating additional feature levels. The channel attention fusion module includes: a complex feature fusion module, used to filter conflicting and redundant information to alleviate semantic conflicts caused by direct fusion; the complex feature fusion module consists of channel concatenation and channel purification, the channel concatenation can obtain fused features without losing information, and the channel purification is used to generate adaptive weights in the channel dimension; the cross-level path aggregation module includes: a feature path aggregation module, which squeezes, scales, integrates, and activates the fused features output by the complex feature fusion module to change the direction of propagation path; The specific methods by which the feature path aggregation module squeezes, scales, integrates, and activates the fused features output by the complex feature fusion module are as follows: Squeezing: Inputting the first features generated from the bottom layer, intermediate layer, and top layer of the fused features, the first features at the three levels are unified to the same number of channels to obtain the corresponding second features and compressed; Scaling: Adjusting all features in the second features to the same resolution; Integration: Adding an extra weight to each input during multi-scale feature fusion, and using weight normalization to limit the range of each weight value to obtain the integrated third feature; Activation: Applying an activation function to the third feature and introducing a nonlinear transformation to obtain the fourth feature; Step (3): The head network uses a decoupled head to predict the multi-scale features fused by the adaptive path aggregation pyramid network.

2. The intelligent operation and maintenance method for substations based on adaptive path aggregation network according to claim 1, characterized in that, In step (1), the backbone network uses CNN-based CSPDarkNet and Swin Transformer-Tiny as the backbone network.

3. The intelligent operation and maintenance method for substations based on adaptive path aggregation network according to claim 1, characterized in that, In step (2), in the adaptive path aggregation pyramid network, different levels of feature maps at different scales are extracted from the input image by using different backbone networks, and the number of channels of each level of feature map is increased to twice that of the previous level.

4. The intelligent operation and maintenance method for substations based on adaptive path aggregation network according to claim 1, characterized in that, The feature path aggregation module aggregates features from different levels to change the forward and backward paths. During backpropagation, the level can adaptively adjust the parameters of other levels.

5. The intelligent operation and maintenance method for substations based on adaptive path aggregation network according to claim 1, characterized in that, In the first feature, the number of feature channels generated by the top layer is greater than the number of feature channels generated by the bottom layer, and the feature resolution generated by the top layer is less than the feature resolution generated by the bottom layer.

Images