Method for detecting key components and defects of overhead transmission line
By adopting a meta-learning method with a regional perception fusion module and a many-help-few training strategy in overhead transmission line detection, the problems of weak feature representation of support images and query images and data offset are solved, and the detection accuracy is improved, especially the recognition ability of a small number of sample categories.
Patent Information
- Application Number
- CN202311155236.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-08
- Publication Date
- 2025-10-17
- Estimated Expiration
- 2043-09-08
AI Technical Summary
In the meta-learning-based detection method for key components and defects of overhead transmission lines, there are problems such as weak feature representation ability of the fusion of support images and query images, large differences between training images and test images, resulting in insufficient detection accuracy.
A meta-learning-based detection model is constructed by adopting a region-aware fusion module and a two-stage fine-tuning training strategy of using more to help less. Resnet101 is used as the backbone network, and the spatial relationship between support features and query features is captured through the mutual attention mechanism. The data offset problem is alleviated through two-stage training.
The model's detection accuracy for a small number of sample categories is improved, memory usage is reduced, the representation ability of fused features is enhanced, and higher detection accuracy is achieved.
Smart Images
Figure CN117292176B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of image analysis, and particularly relates to a method for detecting key components and defects of overhead power transmission lines based on meta-learning. BACKGROUND
[0002] The stable operation of power transmission lines requires the normal operation of key components such as insulators and fittings. Routine inspection of power lines helps to ensure the continuity and stability of power supply, and can find faults in key components and repair them in time. However, the traditional inspection method relying on human resources cannot cover the extensive power transmission lines in complex terrain. Target detection technology based on deep vision can identify faults in key components in time by using visual images of power lines taken by unmanned aerial vehicles or helicopters, which significantly overcomes the limitations of manual inspection.
[0003] Target detection technology based on deep learning has achieved remarkable results in different fields, including image classification, bounding box regression, instance segmentation and key point detection, for example, on various large public datasets such as ImageNet, COCO and PASCAL VOC, which benefits from the continuous development of computer vision technology. In addition, researchers currently emphasize the application of deep learning-based technology in detecting important components and defects in power transmission lines. Compared with traditional image processing technology, this method has strong robustness and generalization, but it needs a large number of high-quality labeled images, and involves a time-consuming and labor-intensive annotation process. Therefore, it is essential to overcome the problem of insufficient number of samples for the detection of power transmission line components and their defects.
[0004] In recent years, meta-learning-based target detection technology has made great breakthroughs, and meta-learning-based target detection technology for key components and defects of overhead power transmission lines has become a major means to solve the problem of insufficient number of samples.
[0005] However, there are still several difficulties in using meta-learning methods for the detection of key components and defects in power transmission lines:
[0006] 1) The support image has weak fusion feature representation ability with the query image. The components of the overhead transmission line all have similar irregular geometric overall profiles, which leads to weak representation ability of the single dimension feature generated by the fusion of the support image and the query image. 2) There is a significant difference between the training images on the public dataset and the inspection images. Although the public dataset can provide the model with images with rich categories and large sample size of single class, the scenes in the transmission line are mostly high mountains, green land and sky, which are significantly different from the daily life scenes from all over the world in the public dataset. In addition, the key objects in the transmission line, such as towers, insulators and various shaped fittings, have significant differences in appearance with people, animals and household appliances in the public dataset. SUMMARY
[0007] Regarding the above problems, the inventors found that, in the process of detecting key components and defects of overhead transmission lines based on meta-learning, capturing the spatial relationship between the regional significant features of key components and defects is one of the effective ways to improve the representation ability of the fusion features. In the process of detecting key components and defects of overhead transmission lines based on meta-learning, constructing images with small differences in sample categories as training data and selecting a training strategy to avoid overfitting are key factors to improve the performance of the detection model. Therefore, the present application uses a region perception fusion module and a two-stage fine-tuning training strategy with more help and less to enhance the fusion ability between the support features and the query features extracted by the model, and to alleviate the data drift problem between the training images and the test images, in order to solve the problem of detecting key components and defects of overhead transmission lines based on meta-learning. The purpose of the present application is to provide a meta-learning detection method for key components and defects of overhead transmission lines based on a region perception fusion module and a two-stage fine-tuning training strategy with more help and less, to solve the problems of weak fusion feature representation ability and large difference between training images and test images in the meta-learning task of transmission line detection, to design a target detection framework of meta-learning Cascade RCNN based on Resnet101 as the backbone network, to design a region perception fusion module to improve the representation ability of the fusion features, and to design a two-stage fine-tuning training strategy with more help and less to further improve the positioning accuracy of the model.
[0008] To achieve the above purpose, the present application provides the following scheme:
[0009] A detection method for key components and defects of overhead transmission lines based on meta-learning, characterized in that it comprises:
[0010] constructing a detection model for key components and defects of overhead transmission lines based on meta-learning;
[0011] The detection model is trained by using a two-stage fine-tuning training strategy of more helps and less, to obtain a target detection model;
[0012] Support images containing a small number of sample categories and test images are input into the target detection network, and objects corresponding to the support image categories are marked in the test images in the form of a bounding box;
[0013] The target detection model includes four modules: a feature extraction module, a region perception fusion module, a region proposal frame module, and a cascade detection head module.
[0014] The feature extraction module is a network with Resnet101 as shared parameters, which realizes feature extraction of query images and support images, and generates query feature maps and support feature maps with spatial information;
[0015] The region perception fusion module performs mutual attention mechanism operation on the query feature maps and support feature maps with spatial information to generate fusion features for guiding network target detection;
[0016] The region proposal frame module uses the fusion features generated by the region perception fusion module to suppress background information irrelevant to the categories contained in the support images, and generates proposal frames for classification and regression of the cascade detection head module;
[0017] The cascade detection head module uses a cascade of multiple detection heads to gradually improve the accuracy of the region proposal features and obtain more accurate class scores and prediction boxes.
[0018] The two-stage fine-tuning training strategy of more helps and less specifically includes:
[0019] Divide the image data of key components and defects of power transmission lines, including base class image data for first-stage model training containing sample sufficient categories, and new class image data for second-stage model fine-tuning containing sample few categories;
[0020] Construct a meta-learning task containing support images and query images from the base class image data to train the target detection model, to obtain a target detection model containing pre-training weights;
[0021] Construct a meta-learning task containing support images and query images from the new class image data to fine-tune the target detection model containing pre-training weights, to obtain a final target detection model containing weights.
[0022] The application discloses the following technical effects: the application provides a method for detecting key components and defects of an overhead power transmission line based on meta learning, specifically, the method is based on a regional perception fusion module and a meta learning method for detecting key components and defects of an overhead power transmission line based on a two-stage training strategy of more helping less, a Cascade RCNN is selected as a framework, and Resnet101 is used as a backbone network with shared parameters, so that the memory occupation of the model is reduced; based on the idea of the mutual attention mechanism, a fusion module of regional perception is designed, the spatial relationship between support features and query features is captured, fusion features used for guiding the network to detect the categories corresponding to the support image from the query image are obtained, and the representation ability of the fusion features is improved; a two-stage fine-tuning training strategy of more helping less is adopted, the categories of sufficient samples are trained in the first stage, pre-training weights with prior knowledge of the power transmission line are obtained, and in the second stage, the fine-tuning of the frozen backbone network is performed on the balanced and small sample categories, the data offset problem between the sufficient sample categories and the small sample categories in the data set is alleviated, and the detection accuracy of the model for the small sample categories is further improved. BRIEF DESCRIPTION OF DRAWINGS
[0023] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description only constitute some embodiments of the application, and for those skilled in the art, other drawings can also be obtained based on these drawings without creative labor.
[0024] Figure 1 is a flowchart of a method for detecting key components and defects of an overhead power transmission line based on meta learning according to an embodiment of the application;
[0025] Figure 2 is a schematic diagram of a two-stage fine-tuning training strategy of more helping less according to an embodiment of the application;
[0026] Figure 3 is a schematic diagram of a model structure according to an embodiment of the application;
[0027] Figure 4 is a schematic diagram of a network structure of a regional perception fusion module according to an embodiment of the application;
[0028] Figure 5 is a schematic diagram of a cascade detection head module structure based on a regional perception fusion module according to an embodiment of the application;
[0029] Figure 6 is a detection effect diagram of a key component of a power transmission line according to an embodiment of the application. DETAILED DESCRIPTION
[0030] With reference to the accompanying drawings, the technical solutions in the embodiments of the present application will be clearly and completely described below, obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the protection scope of the present application.
[0031] The purpose of the present application is to provide a meta-learning overhead transmission line key component and defect detection method based on regional perception fusion module and multi-assistance and few-stage fine-tuning strategy, to solve the problems of data offset between training and testing data, insufficient support feature and query feature fusion, and to realize the purpose of improving the object detection accuracy of a small number of sample categories under the premise of meeting real-time detection.
[0032] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.
[0033] As shown in Figure 1 The present application provides a meta-learning overhead transmission line key component and defect detection method, specifically a meta-learning overhead transmission line key component and defect detection method based on regional perception fusion module and multi-assistance and few-stage fine-tuning training strategy, which includes:
[0034] First, a meta-learning overhead transmission line key component and defect target detection model is constructed;
[0035] Next, the preset target detection model is trained using a multi-assistance and few-stage fine-tuning training strategy;
[0036] Finally, the support image and test image containing a small number of sample categories are input into the preset detection network, and the objects corresponding to the support image categories are marked in the test image in the form of a bounding box.
[0037] Among them, the model selects Cascade RCNN as the framework, uses Resnet101 as the backbone network, and designs a regional perception fusion module based on the idea of mutual attention mechanism to capture the spatial relationship between support features and query features, obtain fusion features for guiding the network to detect the corresponding categories from the query image. The training strategy of multi-assistance and few-stage fine-tuning is adopted, the first stage is to train the categories with sufficient samples, and the second stage is to fine-tune the backbone network on the balanced and small sample categories, to alleviate the data offset problem between the sufficient sample categories and the small sample categories in the data set, and to improve the detection accuracy of the model for small sample categories.
[0038] Considering that a deep learning model needs a large number of image samples as data support during training, however, due to the special purpose of key components and the rare occurrence of defects in the power transmission line, it is difficult to collect samples of some categories. In order to realize the detection of key components and defects of the power transmission line in the sample small category, it is necessary to screen and label the samples containing key components and defects. Therefore, an image dataset of key components and defects of the power transmission line containing sample sufficient categories (base class) and sample small categories (new class) is constructed, specifically including:
[0039] The images of key components and defects of the power transmission line taken in the inspection are collected, cleaned, and the images with clear images and containing key components and defects of the power transmission line are selected, and the json annotation file is generated according to the method of the COCO dataset; wherein, the categories with more than 600 samples are divided into base classes, including but not limited to the following categories: hanging plate, hanging ring, composite insulator, bird nest, adjustment plate, glass insulator, grading ring, anti-vibration hammer, spacer, suspension clamp, link plate, shielding ring and weight; the categories with less than 200 samples are divided into new classes, including but not limited to the following categories: ground wire suspension insulator, pre-twisted suspension clamp, and groove clamp, grading ring damage, and insulator shed damage.
[0040] Considering that there are obvious scene differences and object appearance differences between the image data of the base class on the public dataset and the image data of the sample small category of the power transmission line, the present application performs the first stage training on the categories with more samples in the power transmission line image, and obtains the prior knowledge closely related to the power transmission line. Therefore, a two-stage fine-tuning training strategy of more and less is adopted, as shown in Figure 2 , specifically including:
[0041] Firstly, the meta-learning task containing support images and query images is constructed from the base class image data, the target detection model is trained, and the target detection model containing pre-training weights is obtained; wherein, the support images and the query images are input of the meta-learning task; the object corresponding to the category contained in the support image is found from each query image as the goal of the meta-learning task; N categories are randomly selected from the image dataset, each category contains K samples, as support images in the meta-learning task; Q samples are randomly selected from the samples of the N categories except the support images as query images.
[0042] Then, a meta-learning task containing the support images and the query images is constructed from the new class image data, a target detection model containing pre-training weights is fine-tuned to obtain a final target detection model containing weights. The fine-tuning method specifically refers to keeping the parameters of the model feature extraction module unchanged on the basis of the pre-training model, only fine-tuning the parameters of the region perception fusion module, the region proposal frame module and the cascade detection head module, so as to adapt to the meta-learning task containing the support set images and the query images constructed from the new class image data, reduce the demand for computing resources and accelerate the training process.
[0043] As shown in Figure 3 The target detection model for key components and defects of overhead transmission lines based on meta-learning provided by the application includes four modules: a feature extraction module (Backbone), a region perception fusion module (Region-Aware Fusion, RAF), a region proposal frame module (Region Proposal Network) and a cascade detection head module (Cascade RAF Heads). The feature extraction module is a network with Resnet101 as shared parameters, which realizes feature extraction of the query image and the support image and generates query feature maps and support feature maps with spatial dimensions; the region perception fusion module performs mutual attention mechanism operation on the query feature maps and the support feature maps with spatial dimensions to generate fusion features for guiding network target detection; the region proposal frame module uses the fusion features generated by the region perception fusion module to suppress background information irrelevant to the categories contained in the support image and generate proposal frames for classification and regression of the cascade detection head module; and the cascade detection head module adopts multiple detection cascades to gradually improve the accuracy of the region proposal features and obtain more accurate category scores and prediction frames.
[0044] Considering the weak fusion feature representation ability of the transmission line support image and the query image in the meta-learning task, the application designs a region perception fusion module to capture the spatial relationship between the support features and the query features and enhance the representation ability of the fusion features, as shown in Figure 4 Specifically, the region perception fusion module includes:
[0045] The input of the region perception fusion module contains the query feature F of the query image q obtained from the feature extraction module and the support feature F of the support image s obtained from the feature extraction module. The support feature and the query feature have the same number of channels C. The output of the region perception fusion module is the fusion feature F which has the same spatial shape as the query feature and twice the number of channels of the query feature.
[0046] First, the region perception fusion module performs spatial attention operation on the support feature F s and the query feature Fq Reshape into new features and Then, F s and F q is divided into N subspaces along the channel dimension, and the support feature and query feature of the nth subspace are and where
[0047] In the nth subspace, the reshaped support feature F s n through the learnable parameter matrix
[0048] The calculation method of embedding is:
[0049]
[0050]
[0051]
[0052] In the nth subspace, is the query embedding, is the key embedding, is the value embedding. Then, the similarity between the query embedding vector and the key embedding vector is measured by the similarity score, and the specific calculation formula is:
[0053]
[0054] where q i is the ith query vector of Q n , k j is the jth key vector of K n , and is the similarity score between q i and k j calculated by the scaling cosine function. τ and B i,j are learnable scalars. The shape of the similarity score measure S n is H q W q × H v W v , which shows that we get multiple support attention maps (HsWs) at each spatial position of the query feature map. Finally, the region-aware attention feature map of the N subspaces is obtained by:
[0055] A = concat (V 1 S 1 , V 2 S 2...V N S N )
[0056] wherein, represents a region perception feature map formed by connecting region perception vectors from N subspaces. Finally, we concatenate the outputs A and F and change their shapes to form the final fusion feature fusion feature F a has the same spatial shape as the query feature map F q , but its channel number is twice that of F q .
[0057] In order to further improve the accuracy of object detection, a multi-stage cascade detection head module based on a region perception fusion module is designed, as shown in Figure 5 , which specifically includes:
[0058] The cascade detection head module is formed by cascading three identical detection heads; the input of the detection head of each stage is the boundary proposal box generated by the previous stage and the feature obtained from the backbone network; first, the interest region pooling layer performs a pooling operation on the feature obtained from the backbone network according to the boundary proposal box to generate a pooled feature map; then, the region perception module performs a fusion operation on the pooled feature map and the feature of the support image to generate a fused feature map; finally, the class score and the boundary box of this stage are generated according to the fused feature map.
[0059] The detection effect diagram of the detection method of the present application is shown in Figure 6 . The present application uses Resnet101 as a shared parameter backbone network based on Cascade RCNN as a basic architecture, reducing the memory occupancy of the model; in view of the weak fusion feature representation ability of the support image and the query image in the meta-learning task, a region perception fusion module is designed to capture the spatial relationship between the support feature and the query feature, and to enhance the representation ability of the fusion feature. The present application effectively improves the detection performance of the detection network on a small amount of classes, and realizes the accurate identification of a small amount of classes in the power transmission line.
[0060] The principles and implementation modes of the present application are described in this paper by applying specific examples; the above examples are only used to help understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed. In view of the above, the content of the specification should not be understood as a limitation of the present application.
Claims
1. A method for detecting key components and defects of overhead transmission lines based on meta-learning, characterized in that: include: Build a meta-learning-based detection model for key components and defects of overhead transmission lines; The detection model is trained using a two-stage fine-tuning training strategy of using more to help less to obtain an object detection model; Inputting a support image containing a small number of categories of samples and a test image into the object detection network, marking objects corresponding to the categories in the support image in the form of bounding boxes in the test image; The target detection model consists of four modules: feature extraction module, region perception fusion module, region proposal box module and cascade detection head module; The feature extraction module is a network with Resnet101 as shared parameters, which realizes feature extraction of query image and support image, and generates query feature map and support feature map with spatial information; The region-aware fusion module performs a mutual attention mechanism on the query feature map and the support feature map with spatial information to generate fusion features for guiding network target detection; The region proposal module uses the fusion features generated by the region-aware fusion module to suppress background information irrelevant to the categories contained in the support image and generates proposal boxes for classification and regression of the cascade detection head module; The cascade detection head module uses multiple detection heads in cascade to gradually improve the accuracy of region proposal features and obtain more accurate category scores and prediction boxes; The two-stage fine-tuning training strategy of using more to help less specifically includes: Image data of key components and defects of power transmission lines are divided into base class image data with sufficient samples for first-stage model training, and new class image data with few samples for second-stage model fine-tuning; Construct a meta-learning task containing support images and query images from the base class image data, train the object detection model, and obtain an object detection model with pre-trained weights; Construct a meta-learning task containing support images and query images from new image data, fine-tune the object detection model with pre-trained weights, and obtain the final object detection model with weights.
2. The method for detecting key components and defects of overhead transmission lines based on meta-learning according to claim 1, characterized in that: The regional perception fusion module specifically includes: The input of the region-aware fusion module includes: the query features of the query image q obtained from the feature extraction module and the support features of the support image s obtained from the feature extraction module The support feature and the query feature have the same number of channels C, and the output of the region-aware fusion module is the fusion feature It has the same spatial shape as the query feature, but has twice as many channels as the query feature. First, the region-aware fusion module will support feature F s and query feature F q Reshape into new features and Then, F s and F q It is divided into N subspaces along the channel dimension, and the support features and query features of the nth subspace are and in In the nth subspace, the reshaped support feature F s n Through the learnable parameter matrix and is converted into a key embedding K n Sum value embedding V n , and the reshaped query feature F q n Through the parameter matrix is converted into a query embedding Q n , query, key and value embeddings are calculated as: In the nth subspace, is the query embedding, is the key embedding, is the value embedding. Then, the similarity between the query embedding vector and the key embedding vector is measured by the similarity score between them. The specific calculation formula is: Among them, q i It's Q n The i-th query vector, k j It's K n The jth key vector of is q calculated by scaling the cosine function i and k j The similarity score between τ and B i,j is a learnable scalar; the similarity score metric S n The shape is H q W q ×H v W v , which indicates that multiple support attention maps (HsWs) are obtained at each spatial position of the query feature map. Finally, the region-aware attention feature map of N subspaces is obtained by: A=concat(V 1 S 1 ,V 2 S 2 ...V N S N ) (3) in, Represents the region-aware feature map formed by connecting the region-aware vectors from N subspaces; finally, the output A and F are concatenated and their shapes are changed to form the final fusion feature Fusion feature F a and query feature graph F q has the same spatial shape, but its number of channels is F q twice as much.
3. The method for detecting key components and defects of overhead transmission lines based on meta-learning according to claim 1, characterized in that: The region suggestion frame module specifically includes: First, based on the support features and query features extracted by the feature extraction module, the region-aware fusion module generates fusion features with spatial information. Then, the fused features with a channel number of 2C are passed through a 3x3 convolutional layer network to generate a feature map with a channel number of C; Finally, the feature map with the number of channels C is passed through a 1x1 convolutional layer network to generate the category score and coordinate offset for each anchor point and box on the feature map.
4. The method for detecting key components and defects of overhead transmission lines based on meta-learning according to claim 1, characterized in that: The cascade detection head module specifically includes: The cascade detection head module is formed by cascading three identical detection heads; the input of the detection head in each stage is the bounding box proposal generated by the previous stage and the features obtained from the backbone network; first, the region of interest pooling layer pools the features obtained from the backbone network according to the bounding box proposal to generate a pooled feature map; then, the region perception module fuses the pooled feature map with the features of the support image to generate a fused feature map; finally, based on the fused feature map, the category score and bounding box of this stage are generated as the input of the detection head in the next stage.
5. The method for detecting key components and defects of overhead transmission lines based on meta-learning according to claim 1, characterized in that: The meta-learning tasks specifically include: Randomly select N categories from the image dataset, each category contains K samples, as support images in the meta-learning task; randomly extract Q samples from the samples of N categories excluding support images as query images; support images and query images are used as inputs of the meta-learning task; find objects corresponding to the categories contained in the support images from each query image as the purpose of the meta-learning task.
6. The method for detecting key components and defects of overhead transmission lines based on meta-learning according to claim 1, characterized in that: The fine-tuning specifically includes: Based on the pre-trained object detection model, the parameters of the model feature extraction module remain unchanged, and only the parameters of the region-aware fusion module, the region proposal box module, and the cascade detection head module are fine-tuned to adapt to the meta-learning task constructed from new image data containing support set images and query images, reducing the demand for computing resources and accelerating the training process.