A method and system for detecting foreign matter on a distribution line based on small sample target detection

Abstract

The application discloses a kind of based on small sample target detection's distribution network line foreign matter detection method and system, the present application includes for foreign matter image, foreign matter image training foreign matter image generation network model and increase the sample quantity of foreign matter image;All category images are unified format and are marked and handled and form foreign matter data set;The length-width ratio of anchor point frame of all category images in foreign matter data set is counted, and anchor point frame template is generated;Anchorage point frame template is used, and feature extraction network is trained based on base class training set;Improved small sample target detection network model is constructed, and the training of improved small sample target detection network model is completed using equal amount of training examples, and the optimal weight parameter and hyperparameter in improved small sample target detection network model are saved.The present application aims to reduce artificial participation and ensure detection real-time, in a small amount of foreign matter sample, high detection accuracy is realized, and good detection effect can be obtained.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of small sample target detection, and particularly relates to a distribution network line foreign matter detection method and system based on small sample target detection. BACKGROUND

[0002] Normal and safe operation of a power system is of great significance to people's life. In the process of power transmission, balloons, kites and other foreign matters often wrap around the transmission line due to human activities, causing circuit damage, circuit tripping, fire and a series of safety accidents, which paralyzes the power system, affects people's daily life and causes significant economic losses. In actual life, the foreign matters of the transmission line mainly include kites, balloons and bird nests, and the conventional line foreign matter detection methods mainly include two types: a traditional manual inspection method and an unmanned aerial vehicle inspection method. The traditional manual inspection method has low efficiency and high labor cost, and cannot guarantee the safety of the inspection personnel in extreme conditions such as adverse geographical conditions and adverse weather conditions, which has great safety hazards. The unmanned aerial vehicle inspection method mainly collects line foreign matter images in real time by means of an unmanned aerial vehicle, and then inputs the foreign matter images into a target detection algorithm based on deep learning to detect the distribution network line foreign matter. This method can guarantee the safety of the personnel, has high efficiency and real-time performance, and can effectively reduce the probability of safety accidents caused by transmission line foreign matters. In addition, the method can save labor costs and improve the efficiency of distribution network line foreign matter detection. In recent years, with the continuous development of deep learning, the target detection algorithm has been continuously improved. The traditional target detection method needs to be trained on a large amount of data, but the frequency of occurrence of transmission line foreign matters is low, and the amount of available data for collection is limited, so the traditional target detection method cannot be used to train the network. Therefore, a small sample target detection method needs to be used. SUMMARY

[0003] The present application aims to solve the above-mentioned problems in the prior art, and provides a distribution network line foreign matter detection method and system based on small sample target detection, which can complete high-precision distribution network line foreign matter detection with a small amount of foreign matter data sets, improve the model robustness and the distribution network line foreign matter detection precision, and facilitate the use in real industrial scenarios.

[0004] To solve the above-mentioned technical problems, the technical scheme adopted by the present application is as follows:

[0005] A distribution network line foreign matter detection method based on small sample target detection, comprising:

[0006] S101, collecting base class images corresponding to basic targets of the distribution network line and foreign matter images corresponding to foreign matters; training a foreign matter image generation network model by using the collected foreign matter images, and generating new foreign matter images by using the foreign matter image generation network model to increase the sample quantity;

[0007] S102, unify formats and perform annotation processing on the base class images and the foreign matter images, divide them into training sets, verification sets and test sets in proportion, and form a foreign matter detection data set containing the base class training sets;

[0008] S103, count the aspect ratios of anchor boxes of all class images in the foreign matter detection data set, and generate an anchor box template;

[0009] S104, training a feature extraction network based on the base class training sets using the anchor box template;

[0010] S105, constructing a target detection network model, the target detection network model being an improved FSCE network model obtained by improving a basic FSCE network model, the improvement including adding a channel attention mechanism module ASE to each horizontal branch of a feature pyramid module of the basic model, and replacing a convolution layer on each output branch of the feature pyramid module with an inner convolution layer;

[0011] S106, freezing parameters of the feature extraction network, migrating the feature extraction network to the target detection network model, training the target detection network model using equal amounts of instances in the foreign matter data set, and saving optimal weight parameters and hyperparameters obtained through the training.

[0012] Optionally, the basic targets of the wiring route include tension clamps and sleeves, and the foreign matter is a bird nest.

[0013] Optionally, before the annotation processing in step S102, the method further includes generating new images to increase the number of samples by performing part or all of the operations of Gaussian noise, horizontal and vertical flipping and brightness adjustment on the base class images and the foreign matter images.

[0014] Optionally, the foreign matter image generation network model in step S101 is a ConSinGAN network model, and the feature extraction network in step S104 is a ResNet101 network model.

[0015] Optionally, the improved FSCE network model in step S105 includes:

[0016] a feature pyramid module, configured to fuse semantic features of upper and lower layers of an input feature map through a feature pyramid structure to obtain a fused multi-layer feature map;

[0017] a channel attention mechanism module ASE, configured to assign attention weights to the multi-layer feature map to output a feature map with channel attention weights;

[0018] an inner convolution module, configured to perform inner convolution processing on the feature map with channel attention weights;

[0019] a region generation network configured to generate an anchor box based on the feature map processed by the inner convolution;

[0020] a region of interest pooling module configured to project the anchor box generated by the region generation network onto the feature map to generate a region of interest R i ;

[0021] a region of interest processing network configured to process the region of interest R i extract region of interest features through the region of interest feature extractor, respectively through a classification branch to predict the type of the target, through a position branch to predict the position of the anchor box, respectively calculate the classification loss and the position loss, and sort the classification loss through an online difficult sample sampler according to the size of the classification loss to select difficult samples

[0022] a difficult sample processing network configured to process the difficult samples extract region of interest features through the region of interest feature extractor, respectively through a classification branch to predict the type of the target, through a position branch to predict the position of the anchor box, and through a comparison head to compare, respectively calculate the classification loss, the position loss and the comparison loss for training and improving the FSCE network model.

[0023] Optionally, the channel attention mechanism module ASE includes:

[0024] an adaptive average pooling layer AdaptiveAvgPool2d configured to perform spatial feature compression on the input feature map with a size of HxWx256 to obtain a feature map with a size of 1x1x256, wherein H and W are the height and width of the feature map, respectively;

[0025] a one-dimensional convolution layer with a convolution kernel size of 5x5 and a padding size of 2, configured to perform channel feature weight learning on the compressed 1x1x256 feature map to obtain a feature map with a size of 1x1x256;

[0026] a fusion module configured to multiply the feature map with a size of 1x1x256 output by the one-dimensional convolution layer with the input feature map with a size of HxWx256 after normalized processing by a Sigmoid function, and finally output a feature map with channel attention weights.

[0027] Optionally, the inner convolution module is composed of a kernel generation module and a multiply-add module, and the kernel generation module is configured to generate an inner convolution kernel based on the input feature map through a kernel function, wherein the kernel function is represented as:

[0028]

[0029] In the above formula, H i,j The generated involution kernel is φ, where φ is the kernel function. Let Ψ be the set of pixels at position (i,j). i,j Input feature mapping, Ψ i,j Let Ψ be the set of pixels at position (i,j). i,j W0 and W1 are two linear transformations, and σ represents the batch normalization layer and the nonlinear activation function layer; the multiply-accumulate module is used to generate an involution kernel H. i,j Using a single pixel at position (i,j) as a condition, the channels are rearranged in space before multiplication and addition operations are performed. The product operation in the multiplication and addition operation involves multiplying the tensors of the C channels with the inner convolution kernel H. i,j Multiplication, the addition operation in multiplication-addition operations is to multiply the involution kernel H. i,j Elements within the range are added to the inner convolution kernel H. i,j The core of it.

[0030] Optionally, after step S106, the method further includes testing the object detection network model using the saved optimal weight parameters and hyperparameters, and visualizing the test results.

[0031] In addition, the present invention also provides a foreign object detection system for distribution network lines based on small sample target detection, including a microprocessor and a memory connected to each other, wherein the microprocessor is programmed or configured to execute the foreign object detection method for distribution network lines based on small sample target detection.

[0032] Furthermore, the present invention also provides a computer-readable storage medium storing a computer program, the computer program being programmed or configured by a microprocessor to execute the foreign object detection method for distribution network lines based on small sample target detection.

[0033] Compared with the prior art, the present invention has the following main advantages:

[0034] 1. The target detection network model of this invention is an improved FSCE network model obtained by modifying the FSCE network model as the base model. The improvements include adding a channel attention mechanism module (ASE) to each horizontal branch of the feature pyramid module of the base model, and replacing the convolutional layers on each output branch of the feature pyramid module with involutional layers. By improving the FSCE network model, the robustness of the model is improved while increasing the accuracy of foreign object detection in power distribution lines, even when foreign object category sample data is extremely scarce, making it suitable for use in real-world industrial scenarios. By adopting the channel attention mechanism module (ASE), the network's attention to foreign objects is increased even in cases of extreme sample imbalance, effectively reducing the low accuracy of foreign object category detection caused by extreme sample imbalance. By replacing convolutional layers with involutional layers, the drawbacks of traditional convolutions are effectively addressed while ensuring detection accuracy, thus significantly improving foreign object detection accuracy.

[0035] 2. This invention includes statistically analyzing the aspect ratio of anchor boxes in all foreign object images in a foreign object dataset, generating anchor box templates, and using statistical methods to generate anchor box templates adapted to the foreign object detection dataset, which can improve the accuracy of foreign object detection. This method has certain reference value for anchor box template generation in other small sample target detection fields. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of the basic process of the method in an embodiment of the present invention.

[0037] Figure 2 This is a schematic diagram of the network structure of the improved FSCE network model in an embodiment of the present invention.

[0038] Figure 3 This is a schematic diagram of the network structure of the channel attention mechanism module ASE in an embodiment of the present invention.

[0039] Figure 4 This is a statistical result of the aspect ratio of the anchor point frame in the embodiment of the present invention.

[0040] Figure 5 This is a comparative diagram of the generated anchor point frame template (b) and the default anchor point frame (a) in an embodiment of the present invention.

[0041] Figure 6 This is a schematic diagram of the feature pyramid module in an embodiment of the present invention.

[0042] Figure 7 This is a schematic diagram of the inner convolution module in an embodiment of the present invention.

[0043] Figure 8 These are some examples of foreign object detection data in embodiments of the present invention. Detailed Implementation

[0044] like Figure 1 As shown, the foreign object detection method for distribution network lines based on small sample target detection in this embodiment includes:

[0045] S101, collect the base class image corresponding to the basic target of the distribution network line and the foreign object image corresponding to the foreign object; train the foreign object image generation network model with the collected foreign object images, and use the foreign object image generation network model to generate new foreign object images to increase the number of samples.

[0046] S102, for base class images and foreign object images, a unified format is adopted and labeled, and the images are divided into training set, validation set and test set according to the proportion to form a foreign object detection dataset containing the base class training set.

[0047] S103, Calculate the aspect ratio of the anchor boxes for all categories of images in the foreign object detection dataset and generate anchor box templates;

[0048] S104 uses anchor box templates and trains a feature extraction network based on the base class training set;

[0049] S105, Construct an object detection network model. The object detection network model is an improved FSCE network model obtained by improving the FSCE network model as the base model. The improvement includes adding a channel attention mechanism module ASE to each horizontal branch of the feature pyramid module of the base model, and replacing the convolutional layer on each output branch of the feature pyramid module with an inner convolutional layer.

[0050] S106. Freeze the parameters of the feature extraction network, transfer the feature extraction network to the object detection network model, and train the object detection network model using a small number of instances in the foreign object dataset (small sample training is sufficient with a small number of instances). Save the optimal weight parameters and hyperparameters obtained during training to obtain the trained object detection network model. Input the image of the power distribution line into the object detection network model to obtain the foreign object detection results.

[0051] This embodiment of the foreign object detection method for distribution network lines based on small sample target detection can achieve high-precision foreign object detection in distribution network lines with a small amount of foreign object dataset. It improves the robustness of the model and the accuracy of foreign object detection in distribution network lines, which is conducive to its use in real industrial scenarios. It reduces manual intervention and ensures real-time detection. It achieves high detection accuracy and good detection results in a very small number of foreign object samples.

[0052] In this embodiment, the basic targets of the distribution network line include tension clamps and bushings, and the foreign object is a bird's nest. Alternatively, the type of foreign object and the basic target can be selected as needed.

[0053] In this embodiment, the foreign object image generation network model in step S101 is the ConSinGAN network model. Other image generation network models can also be used as needed. The acquired foreign object images are used to train the ConSinGAN network model, which can then be used to generate new foreign object images to increase the number of foreign object image samples. The new foreign object image generation method is as follows: the acquired foreign object images are used as training samples for the ConSinGAN network model, the training mode (train_mode) of the ConSinGAN network model is adjusted to generation mode (“generate”), and the ConSinGAN network model is trained to generate new foreign object image samples to expand the foreign object image sample set in the foreign object detection dataset.

[0054] In this embodiment, step S102, unifying the format of all category images, means converting all acquired and newly generated category images into .jpg format. The annotation process in step S102 involves using the labelimg annotation tool to annotate each category image, generating its corresponding XML format label file. The label file contains the two diagonal coordinates of the target rectangle and the true category. Before the annotation process in step S102, this embodiment also includes generating new category images for the base class image and the foreign object image through some or all of the operations of Gaussian noise, horizontal and vertical flipping, and brightness adjustment to increase the number of category image samples.

[0055] In step S102, when unifying the format and annotating all category images, this includes annotating the collected images of tension clamps, sleeves, and newly generated foreign object images. A foreign object detection dataset is constructed for all annotated category images, and each category image in the dataset is divided into a training set, a validation set, and a test set according to a set ratio. Specifically, in this embodiment, the training set, validation set, and test set are divided for both tension clamp and sleeve images in an 8:1:1 ratio, and the training set, validation set, and test set are divided for foreign object images in a 6:2:2 ratio.

[0056] In this embodiment, the feature extraction network in step S104 is a ResNet101 network model. Other feature extraction network models can also be used as needed. The base class training set consists of training sets for tension clamps and sleeves. The tension clamps and sleeves are used as the base classes for training the FSCE network model, and the ResNet101 network model is trained using the base class training sets of tension clamp and sleeve images.

[0057] It should be noted that the target detection network model constructed in step S105 can be any target detection network model that is required. For example, as an optional implementation, the target detection network model constructed in step S105 of this embodiment is an improved FSCE network model based on the FSCE network model.

[0058] like Figure 2 As shown, the improved FSCE network model includes:

[0059] The feature pyramid module is used to fuse the semantic features of the upper and lower layers of the input feature map through the feature pyramid structure to obtain a fused multi-layer feature map.

[0060] The Channel Attention Mechanism (ASE) module is used to assign attention weights to multi-layer feature maps and output feature maps with channel attention weights.

[0061] The inner convolution module is used to perform inner convolution processing on feature maps with channel attention weights;

[0062] A region generation network is used to generate anchor boxes based on the feature maps processed by involution.

[0063] The region of interest pooling module projects the anchor boxes generated by the region generation network onto the feature map to generate the region of interest R. i ;

[0064] Region of Interest (ROI) processing networks, such as Figure 2 As shown in (A), it is used to define the region of interest R. i Regions of Interest (ROI) features are extracted using a Region of Interest (ROI) feature extractor. The target type is predicted via a classification branch, and the anchor box position is predicted via a location branch. Classification and location losses are calculated separately. The classification losses are then sorted in descending order of magnitude using an online difficulty sampler to select the most difficult samples.

[0065] Difficult sample processing networks, such as Figure 2 As shown in (B), this is used to classify difficulty samples. Features of the region of interest (ROI) are extracted using a ROI feature extractor. The target type is predicted by a classification branch, the anchor box position is predicted by a location branch, and the comparison is performed by a contrastive head. The classification loss, location loss, and contrastive loss are calculated to train and improve the FSCE network model.

[0066] The improved FSCE network model construction method is as follows: The FSCE network model is used as the base model; a channel attention mechanism module (ASE) is added to each lateral branch of the feature pyramid of the FSCE network model; and the convolutional layers on each output branch of the feature pyramid of the FSCE network model are replaced with involutional layers. In this embodiment, the input category image is scaled proportionally according to its aspect ratio to one of 11 sizes closest to the original ratio: (1333, 480), (1333, 512), (1333, 544), (1333, 576), (1333, 608), (1333, 640), (1333, 672), (1333, 704), (1333, 736), (1333, 768), and (1333, 800). The scaled category image is then input into the feature extraction network ResN. The et101 extracts semantic features from categorical images. When the number of input channels first reaches 256, 512, 1024, and 2048, corresponding to the outputs of layers 0, 1, 2, and 3 of the ResNet101 feature extraction network, the feature maps of these four layers are extracted and output. The corresponding sizes of the output feature maps are (168, 224, 256), (84, 112, 512), (42, 56, 1024), and (21, 28, 2048), respectively, used for classification and regression of semantic features of small, medium, and large sizes. The output feature maps are input into a feature pyramid (an improved FPN module) structure, where semantic features from upper and lower layers are fused. The fused five-layer feature map is then fed into a region generation network. The region generation network generates candidate boxes based on anchor box templates and projects these candidate boxes onto the fused feature maps to obtain the corresponding feature matrix. The feature matrix is ​​scaled by region-of-interest pooling, and the semantic feature vector of the feature matrix is ​​extracted by the region-of-interest feature extractor. The extracted semantic feature vector of the feature matrix is ​​input into the network classification and regression branches for classification regression and coordinate regression, respectively. Finally, the target category and coordinates of the input category image are output.

[0067] In this embodiment, the channel attention mechanism module ASE is a lightweight module. In the case of extreme sample imbalance, it can improve the network's attention to foreign objects while effectively reducing the low detection accuracy of foreign object categories caused by extreme sample imbalance.

[0068] like Figure 3 As shown, the channel attention mechanism module ASE in this embodiment includes:

[0069] The Adaptive AvgPool2d layer is used to perform spatial feature compression on the input feature map of size H×W×256 (adaptive global average pooling GAP is used in the spatial dimension) to obtain a feature map of size 1×1×256, where H and W are the height and width of the feature map, respectively.

[0070] A one-dimensional convolutional layer with a kernel size of 5×5 and a padding size of 2 is used to learn channel feature weights on the compressed 1×1×256 feature map to obtain a feature map of size 1×1×256.

[0071] The fusion module is used to normalize the 1×1×256 feature map output by the one-dimensional convolutional layer using the Sigmoid function, and then multiply it with the corresponding channel of the input feature map of size H×W×256, finally outputting a feature map with channel attention weights.

[0072] The Channel Attention Mechanism (ASE) module effectively focuses on new classes while paying attention to important channels of semantic feature information in the feature map of new classes, weakening the interference of relatively unimportant channels, which can improve the fusion effect of semantic features and improve the accuracy of foreign object detection.

[0073] In this embodiment, step 103 includes statistically analyzing the aspect ratio of the anchor boxes of all foreign object images in the foreign object dataset, fixing the anchor box areas to 32, 64, 128, 256, and 512, and generating an anchor box template suitable for this foreign object detection dataset based on the anchor box aspect ratio. That is, the area of ​​the anchor box template is a fixed size, and the aspect ratio is obtained statistically. Compared with using the default unified anchor box template, the network trained in this way has better detection performance on foreign object images.

[0074] The convolutional structure used in this embodiment effectively addresses the drawbacks of traditional convolution while ensuring detection accuracy, thus significantly improving foreign object detection accuracy. Traditional convolutions exhibit spatial invariance and channel heterogeneity. Spatial invariance deprives the convolutional kernel of its ability to capture multiple semantic features at different spatial locations in the image, thereby ignoring contextual information. Channel heterogeneity easily leads to feature map channel redundancy, causing the same feature to be extracted multiple times, affecting network efficiency. Based on the problems of traditional convolution, a module is proposed to address these shortcomings, improving detection accuracy while reducing the number of parameters and ensuring network efficiency. In contrast to traditional convolution, convolution exhibits spatial heterogeneity and channel consistency, meaning it differs in spatial range but shares channels. In this embodiment, the convolutional module consists of a kernel generation module and a multiply-accumulate module. The kernel generation module generates the convolutional kernel based on the input feature map using a kernel function, where the kernel function is expressed as:

[0075]

[0076] In the above formula, H i,j The generated involution kernel is φ, where φ is the kernel function. Let Ψ be the set of pixels at position (i,j). i,j Input feature mapping, Ψ i,jLet Ψ be the set of pixels at position (i,j). i,j W0 and W1 are two linear transformations, and σ represents the batch normalization layer and the nonlinear activation function layer; the multiply-accumulate module is used to generate an involution kernel H. i,j Using a single pixel at position (i,j) as a condition, the channels are rearranged in space before multiplication and addition operations are performed. The product operation in the multiplication and addition operation involves multiplying the tensors of the C channels with the inner convolution kernel H. i,j Multiplication, the addition operation in multiplication-addition operations is to multiply the involution kernel H. i,j Elements within the range are added to the inner convolution kernel H. i,j The core. For example... Figure 7 As shown, the input feature map size is C×W×H, where C is the number of channels in the input feature map, H is the height of the input feature map, and W is the width of the input feature map. Figure 7 As shown in the left cube, a set of shared weight pixel feature maps X are extracted from the input feature map, and X is processed by a kernel function φ to generate a 1×1×K matrix. 2 A vector of size X is rearranged in terms of channels and space to generate a 1×1×K convolution kernel H. This kernel is multiplied by each channel of the feature map X to obtain a 1×1×C pixel feature map Y. The resulting 1×1×C pixel feature map Y is then added to the portion of the input feature map excluding the shared weight pixel feature map X to obtain... Figure 7 The right cube shows the output feature map of size C×W×H. The involution kernel H is generated by the kernel function φ, which takes a single pixel at position (i,j) as a condition, and then performs a multiplication-addition operation after rearranging the channels to space. The multiplication operation is to multiply the tensors of the C channels by the involution kernel H respectively, and the addition operation is to add the elements within the range of the involution kernel to the kernel core.

[0077] After constructing the target detection network model in step S105, the parameters of the feature extraction network can be frozen in step S106. The target detection network model is then transferred to a new class (target detection network model). Instances from an equal number of tension clamps, sleeves, and foreign object images are used to train the detection head of the improved FSCE network model. The improved FSCE network model is then validated on a validation set, and the weights and hyperparameters with the highest detection accuracy on the validation set are saved. Finally, the optimal weights and hyperparameters in the target detection network model are saved, resulting in the trained target detection network model. As described above, the training process can be divided into two stages:

[0078] Phase 1: Training phase of the feature extraction network ResNet101.

[0079] Tension clamps and sleeves are used as the base classes for training the FSCE network model. The FSCE network is trained using training sets of images of tension clamps and sleeves. The ResNet101 feature extraction network extracts semantic features from the images of tension clamps and sleeves to form feature maps. When the number of input channels of the feature maps first reaches 256, 512, 1024, and 2048, that is, when the outputs of layers 0, 1, 2, and 3 of the ResNet101 feature extraction network are reached, the feature maps of these four layers are extracted and output. The corresponding sizes of the output feature maps are (168, 224, 256), (84, 112, 512), and (42, 56, 1...). (024), (21,28,2048), are used for the classification and regression of semantic features of small, medium, and large sizes, respectively. The output feature map is input into the feature pyramid structure, and the semantic features of the upper and lower layers are fused through the feature pyramid structure. The fused 5-layer feature map is then input into the region generation network. The region generation network generates anchor boxes according to the anchor box template. The number of anchor boxes is no more than 2000. For the generated anchor boxes, 512 are randomly sampled at a positive to negative sample ratio of 1:3. When the number of positive samples is insufficient, negative samples are used to fill the gaps. The 512 sampled anchor boxes are projected onto the fused feature map to obtain the corresponding feature matrix. Specifically, the anchor box generation method is as follows: anchor box templates suitable for this foreign object detection dataset are generated by combining the fixed anchor box area of ​​32, 64, 128, 256, 512 with the length and width ratio of the anchor box. When training the feature extraction network ResNet101, the length and width ratio of the anchor boxes of each category of the foreign object detection dataset label file is statistically analyzed. The statistical results are as follows. Figure 4 As shown, the horizontal axis represents the aspect ratio, and the vertical axis represents the number of elements. Anchor frame templates suitable for this foreign object detection dataset are generated using fixed anchor frame areas of 32, 64, 128, 256, and 512, combined with the aspect ratios of the anchor points. Figure 5 As shown in (b) above. Relative to, for example... Figure 5 As shown in (a) of the training method using the default uniform anchor box template, such as Figure 5 The network trained using the method shown in (b) exhibits superior detection performance on foreign object images. The feature matrix is ​​scaled to 7×7 size by the region of interest pooling module. In the region of interest processing network, the semantic features of the feature matrix are extracted by the region of interest feature extractor, forming a 1×1024 feature vector. This feature vector is then input to the network's classification and regression branches for classification regression and coordinate regression, respectively. Finally, the target category and coordinates of the input category image are output. Validation is performed on a validation set, and the weights and hyperparameters with the highest detection accuracy on the validation set are saved.

[0080] Phase Two: Transfer Training Phase.

[0081] Based on the FSCE network model; such asFigure 6 As shown, a channel attention mechanism module ASE is added to each lateral branch of the feature pyramid of the FSCE network model. Figure 6 In the diagram, Input represents the input image. After passing through multiple convolutional blocks (Conv1 block, Conv2 block, Conv3 block, and Conv4 block) stacked in a pyramid shape on the left, four horizontal branches are formed. Each horizontal branch has a 2D convolutional layer with a kernel size of 1×1 (1x1Conv2d) and a channel attention mechanism module ASE. The four horizontal branches then converge to form feature maps M4-M1. Feature maps M4-M1 are then convolved by a convolutional layer to obtain the output feature maps P4-P1. The output of the last convolutional block, Conv4 block, is then processed by a max pooling module to obtain the output feature map P5.

[0082] Because only tension clamps and sleeves were used as base classes for the ResNet101 feature extraction network during the first training stage, and the network parameters were frozen during the second training stage to prevent overfitting, the network model was prone to bias towards the base classes during training, neglecting the new foreign object image class. This caused class imbalance and resulted in suboptimal foreign object detection. Therefore, this invention proposes a channel attention mechanism (ASE module) to enhance the network's attention to the new class. Considering that subsequent classification regression and coordinate regression are based on the feature pyramid output feature map, when the new class is added for transfer training in the second stage, the ASE module is added to each output branch of the feature pyramid of the FSCE network model to enhance the network's attention to the new class and prevent suboptimal foreign object detection due to sample imbalance.

[0083] The second stage of training is as follows: the two categories of tension clamps and sleeves are used as the base classes for training the improved FSCE network model, and the category of foreign objects is used as the new class for training the improved FSCE network model. Instances from an equal number of tension clamp, sleeve, and foreign object images are used to train the improved FSCE network model.

[0084] During the second stage of transfer training, the network randomly samples an equal number of instances from each category's image training set for training. For example, when training with 5 instances (targets in the images), the improved FSCE network model randomly selects 5 images from the training sets for tension clamps, sleeves, and foreign objects. The network iterates through the selected images. If a sample image contains only one target instance, the network iterates through all 5 sample images and selects 5 instances for network model transfer training. If a sample image contains multiple target instances (e.g., an image contains two sleeve instances), the network stops iterating through the remaining sample images after selecting 5 instances. The ResNet101 feature extraction network extracts the semantic features of the extracted tension clamp, sleeve, and foreign object category sample images to form feature maps. When the number of input channels of the feature map first reaches 256, 512, 1024, and 2048, i.e., when the outputs of layers 0, 1, 2, and 3 of the ResNet101 feature extraction network are reached, the feature maps of these four layers are extracted and output. The corresponding feature maps are (168, 224, 256), (84, 112, 512), (42, 56, 1024), and (21, 28, 2048), respectively, used for the classification and regression of semantic features of small, medium, and large sizes. The output feature maps are input into a feature pyramid structure, and the semantic features of the upper and lower layers are fused through the feature pyramid structure. The fused 5-layer feature map is then fed into a Region Generation Network (RPN). The RPN generates anchor boxes based on the anchor box template, with |R| = 2000 anchor boxes. For each generated anchor box R... i Sampling was performed using the Online Difficulty Sample Selection (ODSS) method, with 512 samples taken. The resulting 512 anchor boxes... Project the image onto the fused feature map to obtain the corresponding feature matrix;

[0085] like Figure 2 As shown, when sampling the generated anchor boxes using the online difficulty sample selection method, the generated anchor boxes |R|=2000 are projected onto the feature map to generate the region of interest R. i Region of interest R i Enter Figure 2 The (A) part of the network, namely the region of interest processing network, is composed of... Figure 2 In (A), the region of interest feature extractor extracts R. i Feature vectors are generated from the features. The classification branch performs classification regression on the feature vectors to generate a classification loss, while the location branch performs coordinate regression on the feature vectors to generate a bounding box regression loss. An online difficulty sampler sorts the samples in descending order of their classification loss and selects the most difficult samples. Difficulty Sample EnterFigure 2 The (B) part of the network, namely the difficult sample processing network, is composed of... Figure 2 (B) Difficulty in extracting region of interest features from samples Feature vectors are generated from the feature vectors. The classification branch performs classification regression on the feature vectors to generate classification loss, while the location branch performs coordinate regression on the feature vectors to generate bounding box regression loss. To avoid redundant computation and increase model training and inference speed, for generated similar anchor boxes with high overlap, IOU (Intersection over Union) calculation is introduced. Anchor boxes with an overlap greater than 0.7 are discarded.

[0086] Specifically, the anchor box generation method is as follows: Anchor box templates suitable for this foreign object detection dataset are generated using fixed anchor box areas of 32, 64, 128, 256, and 512, combined with the aspect ratio of the anchor boxes. During training of the ResNet101 feature extraction network, the aspect ratio of the anchor boxes for each category in the label files of the foreign object detection dataset is statistically analyzed. Anchor box templates suitable for this foreign object detection dataset are generated using fixed anchor box areas of 32, 64, 128, 256, and 512, combined with the aspect ratio of the anchor boxes. The feature matrix is ​​scaled to 7×7 size by region-of-interest pooling. The semantic features of the feature matrix are extracted by the region-of-interest feature extractor to form a 1×1024 feature vector. The feature vector is input into the network's classification and regression branches for classification regression and coordinate regression, respectively. Finally, the target category and coordinates of the input category image are output. The improved FSCE network model is validated on the validation set of the foreign object detection dataset, and the weights and hyperparameters with the highest detection accuracy on the validation set are saved. The saved weights and hyperparameters are used to test the test set and visualize the results to obtain the final foreign object detection effect. During the validation and testing, the input category image is scaled to (1333, 800) for validation or testing.

[0087] In this embodiment, the foreign object detection dataset contains 96 sample images, divided into three categories: 70 images of tension clamps and sleeves (each image containing 4-13 instances), and 26 images of foreign objects. The images of tension clamps and sleeves are divided into training, validation, and test sets in an 8:1:1 ratio, while the images of foreign objects are divided into training, validation, and test sets in a 6:2:2 ratio. Figure 8 These are some examples from the foreign object detection data folder. Figure 8 Images (a) and (b) are example images of bushings, (c) and (d) are example images of tension clamps, and (e) and (f) are images of bird nests on transmission line towers.

[0088] In this embodiment, after step S106, the method further includes testing the target detection network model using the saved optimal weight parameters and hyperparameters, and visualizing the test results.

[0089] For comparison, all models in this embodiment use the PyTorch framework, are accelerated with CUDA 11.3, and are trained, validated, and tested on a GTX 3080 GPU. To verify the improvement of the FSCE network model by the method in this embodiment, the following experiments were conducted.

[0090] (1) Ablation experiment of the improved model

[0091] In the second stage of transfer learning, the improved FSCE model was tested. The input image was 1333×800, the batch size was 2, and the SGD optimizer with a momentum of 0.9 was used for optimization. The learning rate was adjusted using a warmup strategy, with warmup_iter = 500, warmup_ratio = 0.001, step set to [60000, 80000], initial learning rate of 0.02, and 90000 training iterations. A validation set test was performed every 7500 training iterations. During model training, the hyperparameters in the model were adjusted, and the set of parameters that achieved the highest foreign object detection accuracy on the validation set was used as the final model test parameters on the test set. The final test results are shown in Table 1.

[0092] Table 1: Ablation experiment results of the improved FSCE network model.

[0093]

[0094] In Table 1, FSCE represents the original FSCE network model, ASE represents the ASE module with added channel attention mechanism, Involution represents the convolutional module proposed in this embodiment, ODSS represents online difficulty sample selection, Anchor represents the new anchor box template proposed in this embodiment, and FPN represents the pyramid module. Referring to Table 1: Compared to the original FSCE network model, the improved FSCE network model in this embodiment achieves a 2.4% improvement in foreign object detection AP, demonstrating the feasibility of the ASE module for small sample detection. It effectively increases the network's attention to foreign object samples when detecting small samples, effectively solving the model offset problem caused by sample quantity bias, thereby improving foreign object detection accuracy. Replacing the 3*3 convolution in the out branch of FPN with an convolutional module increases the foreign object detection AP by 0.8%, demonstrating that the convolutional module effectively reduces model parameters while improving model detection accuracy. For the foreign object detection dataset, statistical methods were used to analyze the true anchor boxes in the dataset, obtaining the aspect ratio distribution of the anchor boxes. Based on the aspect ratio and anchor box area, a new anchor box template was obtained. Testing with the new template resulted in a 0.2% increase in the foreign object detection AP. In the second stage of transfer training, an online difficulty sample selection method was used to sample positive and negative samples from the generated anchor boxes. This method effectively solves the imbalance problem between positive and negative samples in the two-stage algorithm and improves the network model's ability to judge difficult samples. Using this method increased the foreign object detection AP by 0.8%. Finally, for the network model of this invention, the foreign object detection AP increased by 4.4% compared to the original network.

[0095] (2) Improved FSCE network model and compared with other models.

[0096] To verify the effectiveness of the improved model compared to other algorithms, comparative experiments were conducted with other algorithms in the same environment and on the same foreign object detection dataset. Six state-of-the-art (SOTA) network models were tested on the foreign object detection dataset, including Attention-rpn, FSDetView, Meta R-CNN, TFA, MPSR, and FSCE. Among them, Attention-rpn can be found in the literature: Q.Fan, W.Zhuo, C.-K.Tang and Y.-W.Tai, "Few-Shot ObjectDetection With Attention-RPN and Multi-Relation Detector," 2020 IEEE / CVFConference on Computer Vision and Pattern Recognition(CVPR), Seattle, WA, USA, 2020, pp.4012-4021; FSDetView see literature: Y. Xiao, V. Lepetit and R. Marlet, "Few-ShotObject Detection and Viewpoint Estimation for Objects in the Wild," in IEEETransactions on Pattern Analysis and Machine Intelligence, vol.45, no.3, pp.3090-3106, 1March 2023, doi:10.1109 / TPAMI.2022.3174072; Meta For R-CNN, please refer to the literature: X.Yan, Z.Chen, A.Xu, X.Wang, Darrell, Joseph E Gonzalez, and Fisher Yu. Frustratingly simple few-shot object detection.In International Conference on Machine Learning (ICML), July 2020. Jiaxi Wu, Songtao Liu, Di Huang, and Yunhong Wang. Multi-scale positive sample refinement for few-shot object detection. In European Conference on Computer Vision, 2020; MPSR see literature: Jiaxi Wu, Songtao Liu, Di Huang, and Yunhong Wang. Vision, 2020; FSCE see literature: B. Sun, B. Li, S. Cai, Y. Yuan and C. Zhang, "FSCE: Few-Shot Object Detection via Contrastive Proposal Encoding," 2021 IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 2021, pp.7348-7358. .

[0097] Of the six state-of-the-art (SOTA) network models mentioned above, three few-shot object detection models based on meta-learning were tested: Attention-rpn, FSDetView, and Meta R-CNN. The models were trained using 1, 2, 3, 5, and 15 instances respectively, and then tested on the test set. Similarly, four few-shot object detection models based on transfer learning were tested: TFA, MPSR, FSCE, and the improved FSCE network model in this embodiment. The models were trained using 1, 2, 3, 5, and 15 instances respectively, and then tested on the test set. The test results are shown in Table 2.

[0098] Table 2: Comparison of the improved FSCE network model with other models.

[0099]

[0100] As shown in Table 2, the improved FSCE network model (Ours) achieved the highest foreign object detection accuracy when sampling with different numbers of instances. When trained with 15 instances and tested on the test set using their weights, its foreign object detection AP value reached 98.6%, effectively verifying the advantages of the method of this invention for foreign object detection in transmission lines.

[0101] In summary, this embodiment trains a feature extraction network using a dataset of common tension clamps and bushings found in distribution network lines. The network parameters are then frozen, and the network is transferred to a foreign object dataset. The detection head parameters are initialized, and an equal number of images of tension clamps, bushings, and foreign objects are used to train the network's detection head. This method reduces manual intervention and ensures real-time detection. Compared to other small-sample detection methods, this method offers high detection accuracy and performs well even with a very small number of foreign object samples. This embodiment enables high-accuracy foreign object detection in distribution network lines with limited datasets, aiming to improve model robustness while enhancing detection accuracy, making it suitable for use in real-world industrial scenarios.

[0102] Furthermore, this embodiment also provides a foreign object detection system for distribution network lines based on few-sample target detection, including a microprocessor and a memory interconnected thereto. The microprocessor is programmed or configured to execute the foreign object detection method for distribution network lines based on few-sample target detection. Additionally, this embodiment also provides a computer-readable storage medium storing a computer program for being programmed or configured by the microprocessor to execute the foreign object detection method for distribution network lines based on few-sample target detection.

[0103] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the process. Figure 1One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 Figure 1 The steps of the function specified in one or more boxes.

[0104] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for detecting foreign objects in distribution network lines based on small sample target detection, characterized in that, include: S101, collect the base class image corresponding to the target of the distribution network line and the foreign object image corresponding to the foreign object; train the foreign object image generation network model with the collected foreign object images, and use the foreign object image generation network model to generate new foreign object images to increase the number of samples. S102, for base class images and foreign object images, a unified format is adopted and labeled, and the images are divided into training set, validation set and test set according to the proportion to form a foreign object detection dataset containing the base class training set. S103, Calculate the aspect ratio of the anchor boxes for all categories of images in the foreign object detection dataset and generate anchor box templates; S104 uses anchor box templates and trains a feature extraction network based on the base class training set; S105, Construct an object detection network model. The object detection network model is an improved FSCE network model obtained by improving the FSCE network model as the base model. The improvement includes adding a channel attention mechanism module ASE to each horizontal branch of the feature pyramid module of the base model, and replacing the convolutional layer on each output branch of the feature pyramid module with an inner convolutional layer. S106, Freeze the parameters of the feature extraction network, transfer the feature extraction network to the object detection network model, use an equal number of instances in the foreign object dataset to complete the training of the object detection network model, and save the optimal weight parameters and hyperparameters obtained from the training. The improved FSCE network model in step S105 includes: The feature pyramid module is used to fuse the semantic features of the upper and lower layers of the input feature map through the feature pyramid structure to obtain a fused multi-layer feature map. The Channel Attention Mechanism (ASE) module is used to assign attention weights to multi-layer feature maps and output feature maps with channel attention weights. The inner convolution module is used to perform inner convolution processing on feature maps with channel attention weights; A region generation network is used to generate anchor boxes based on the feature maps processed by involution. The region of interest pooling module is used to project the anchor boxes generated by the region generation network onto the feature map to generate regions of interest. ; Region of Interest (ROI) processing networks are used to process regions of interest. Regions of Interest (ROI) features are extracted using a Region of Interest (ROI) feature extractor. The target type is predicted via a classification branch, and the anchor box position is predicted via a location branch. Classification and location losses are calculated separately. The classification losses are then sorted in descending order of magnitude using an online difficulty sampler to select the most difficult samples. ; Difficulty Sample Processing Network, used to process difficult samples Features of the region of interest (ROI) are extracted using a ROI feature extractor. The target type is predicted by a classification branch, the anchor box position is predicted by a location branch, and the comparison is performed by a contrastive head. The classification loss, location loss, and contrastive loss are calculated to train and improve the FSCE network model.

2. The method for detecting foreign objects in distribution network lines based on small sample target detection according to claim 1, characterized in that, The target of the distribution network line includes tension clamps and bushings, and the foreign object is a bird's nest.

3. The method for detecting foreign objects in distribution network lines based on small sample target detection according to claim 1, characterized in that, Before the annotation process in step S102, new images are generated for the base class image and the foreign object image by performing some or all of the operations in Gaussian noise, horizontal and vertical flipping, and brightness adjustment to increase the number of samples.

4. The method for detecting foreign objects in distribution network lines based on small sample target detection according to claim 1, characterized in that, The foreign object image generation network model in step S101 is the ConSinGAN network model; the feature extraction network in step S104 is the ResNet101 network model.

5. The method for detecting foreign objects in distribution network lines based on small sample target detection according to claim 1, characterized in that, The channel attention mechanism module ASE includes: The AdaptiveAvgPool2d layer is used to perform spatial feature compression on the input feature map of size H×W×256, resulting in a feature map of size 1×1×256, where H and W are the height and width of the feature map, respectively. A one-dimensional convolutional layer with a kernel size of 5×5 and a padding size of 2 is used to learn channel feature weights on the compressed 1×1×256 feature map to obtain a feature map of size 1×1×256. The fusion module is used to normalize the 1×1×256 feature map output by the one-dimensional convolutional layer using the Sigmoid function, and then multiply it with the corresponding channel of the input feature map of size H×W×256, finally outputting a feature map with channel attention weights.

6. The method for detecting foreign objects in distribution network lines based on small sample target detection according to claim 1, characterized in that, The inconvolution module consists of a kernel generation module and a multiply-accumulate module. The kernel generation module is used to generate an inconvolution kernel based on the input feature map using a kernel function, where the kernel function is expressed as follows: ， In the above formula, For the generated involution kernel, For kernel function, For position pixel set Input feature mapping, For position pixel set , and For two linear transformations, This is represented as a batch normalization layer and a non-linear activation function layer; the multiply-accumulate module is used to generate an involution kernel. By location Using a single pixel as a condition, the channels are rearranged in space before multiplication and addition operations are performed. The product operation in the multiplication and addition operation involves multiplying the tensors of the C channels with the inner convolution kernel. Multiplication, the addition operation in multiplication-addition operations, is to multiply the kernel of the involution. Elements within the range are added to the inner convolution kernel. The core of it.

7. The method for detecting foreign objects in distribution network lines based on small sample target detection according to claim 1, characterized in that, Step S106 includes testing the object detection network model using the saved optimal weight parameters and hyperparameters, and visualizing the test results.

8. A foreign object detection system for distribution network lines based on small sample target detection, comprising a microprocessor and a memory interconnected, characterized in that, The microprocessor is programmed or configured to execute the foreign object detection method for distribution network lines based on small sample target detection as described in any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program, characterized in that, The computer program is used to be programmed or configured by a microprocessor to execute the foreign object detection method for distribution network lines based on small sample target detection as described in any one of claims 1 to 7.

Images