Power transmission line insulator detection method and system suitable for power grid line patrol unmanned aerial vehicle

Abstract

According to the power transmission line insulator detection method and system suitable for the power grid line patrol unmanned aerial vehicle, an insulator defect identification model is constructed based on a compressed DEIM architecture of cue word fine tuning optimization; according to the insulator defect identification model, the model is effectively lightened through a method of abandoning an edge weight which has small influence on an identification effect and reducing the number of layers of a codec, so that the insulator defect identification model can be carried on an unmanned aerial vehicle, insulator defect detection is carried out in real time through the unmanned aerial vehicle in a power grid line patrol process, and the detection efficiency is improved. And the identified fault information and image are uploaded to the unmanned aerial vehicle nest, and all insulator images do not need to be transmitted. According to the design, insulator defect detection can be carried out in real time through the unmanned aerial vehicle carrying the insulator defect identification model, the data transmission pressure is effectively reduced, the timeliness is enhanced, local defects of the insulator can be identified through a representative detail labeling method and a dense matching strategy, and the detection precision is effectively improved.

Description

Technical Field

[0001] The present invention relates to an insulator defect detection method, and in particular to a transmission line insulator detection method and system suitable for a power grid line inspection drone. Background Art

[0002] With the explosive growth of my country's power grid, the contradiction between the wide distribution range and long transmission distance of the power grid and the low inspection efficiency and high labor intensity at the current stage has become increasingly prominent. It is necessary to continuously explore and improve the power grid inspection methods to adapt to the needs of the rapid development of the power grid.

[0003] In order to cope with the problems of complex regions, harsh environments and large distances that are difficult to handle with traditional manual inspection methods, drone intelligent inspection came into being. It uses efficient and intelligent drone cruising mode to patrol the power grid, while taking images of insulators along the power grid. After returning to the drone nest, the insulator images are transmitted to the drone nest for identification. It can not only complete a variety of difficult tasks such as range inspections, fixed-point inspections, and tower inspections, but is also widely used in low-altitude measurement, special equipment micro-photography, oil field inspections and other fields, improving the safety of the power main grid, enhancing the level of control over the equipment operating status, and making up for the shortcomings of traditional manual inspections.

[0004] Although this drone inspection method can effectively reduce the operation and maintenance pressure of the line, it still has the following defects:

[0005] The existing drone transmission signal has a limited transmission distance and needs to return to the drone nest to uniformly transmit the insulator image, which causes high pressure on data transmission and poor timeliness.

[0006] The information disclosed in this background technology section is only intended to increase understanding of the overall background of the application and should not be considered as an admission or any form of suggestion that the information constitutes the prior art already known to ordinary technicians in this field. Summary of the Invention

[0007] The purpose of the present invention is to overcome the shortcomings of the prior art of high data transmission pressure and poor timeliness, and to provide a transmission line insulator detection method and system suitable for power grid patrol drones with low data transmission pressure and good timeliness.

[0008] To achieve the above objectives, the technical solution of the present invention is:

[0009] A transmission line insulator detection method suitable for power grid patrol drones obtains a dataset of insulators in the power grid and performs representative detail annotation and preprocessing on the insulator dataset. The representative detail annotation involves dividing defective insulators into multiple small blocks and annotating each defective block separately.

[0010] An insulator defect recognition model is constructed based on the detrended prediction DEIM architecture with fine-tuning optimization of prompt words. The insulator defect recognition model is trained based on the preprocessed insulator dataset to obtain a trained insulator defect recognition model.

[0011] The trained insulator defect recognition model is deployed on a line patrol drone. The drone obtains images of each insulator in the line patrol mission. The trained insulator defect recognition model is used to identify the insulator image, and the identified fault information and images are uploaded to the drone's nest.

[0012] The insulator defect recognition model includes a convolutional neural network (CNN) feature extraction network and a feature extraction and recognition network;

[0013] The CNN feature extraction network is used to extract features from the input insulator image and convert the information in the insulator image into a feature map suitable for subsequent processing. The feature extraction and recognition network is used to extract and recognize defect features in the feature map.

[0014] The feature extraction and recognition network includes a Transformer codec, a feedforward prediction network, and a prompt word update module. The input of the Transformer codec is a feature map, and the output of the Transformer codec is feature information. The prompt word update module is used to provide additional context information to the insulator defect recognition model through the recognition results to help the model better understand the target features in the input image. The context information sequence and the feature map are used together as the input of the Transformer codec.

[0015] The feedforward prediction network includes three fully connected layers. The input of the feedforward prediction network is feature information. The feedforward prediction network is used to extract the output of the Transformer decoder into the corresponding image recognition result. The recognition result is the defect type, coordinate information and confidence level of the target.

[0016] The CNN feature extraction network uses the ResNet50 network as the backbone network. The ResNet50 network includes multiple convolutional layers, pooling layers, and residual connection layers. Each of the convolutional layers includes multiple convolution kernels. Each of the convolutional layers is used to slide the multiple convolution kernels on the image to perform convolution operations, thereby extracting different feature maps, and downsampling is performed by convolution kernels or pooling layers with a step size greater than 1.

[0017] The residual connection layer includes a main path and a shortcut connection, wherein the main path is used for convolution operation and the shortcut connection is used to directly add the input to the output of the main path;

[0018] The input of the CNN feature extraction network is:

[0019]

[0020] In the above formula, Image input Represents the input image of the CNN feature extraction network, R is the input image array, the image array has three dimensions, 3 is the RGB three channels, H0 is the height of the input image, and W0 is the width of the input image;

[0021] The output of the CNN feature extraction network is:

[0022]

[0023] In the above formula, C is the feature map, H1 is the height of the output image, and W1 is the width of the output image;

[0024] The Transformer encoder / decoder has 4 layers and converts the context information in the feature map into a vector form. The output of the encoder is:

[0025]

[0026] In the above formula, Encode is the output of the encoder, LayerNorm() is the normalization function, Attention() is the multi-head self-attention mechanism, FFN() is the feedforward neural network, PromptToken is the prompt word sequence, and X is the input of the encoding layer. The input of the encoding layer is the result of splicing the feature map, position information, and prompt word sequence after feature extraction in order;

[0027] The decoding layer is used to generate a sequence containing feature information and context information based on the context information provided by the encoder and the information of the feature map itself. The output of the decoding layer is:

[0028] Decode

[0029]

[0030] In the above formula, Decode is the output of the decoder, Encode is the output of the encoder, FFN() is the feedforward neural network, PromptToken is the prompt word sequence, LayerNorm() is the normalization function, CrossAttention() is the encoder-decoder attention mechanism, and Query refers to the target query, which is a set of learnable vectors.

[0031] The Transformer codec further includes a position information code, which is used to add a position code to the input of the Transformer codec to introduce sequence information. The position information code includes:

[0032]

[0033] In the above formula, PE() is the position information encoding function, pos is the position number corresponding to the image segmentation, i is the index variable used to control the loop traversal, and d is the encoding dimension.

[0034] The training steps of the insulator defect recognition model include:

[0035] The prompt word sequence, the feature map output by the CNN feature extraction network, and its corresponding position encoding information are used as the initial input of the Transformer encoder-decoder;

[0036] In each training iteration, the prompt word sequence and feature Figure 1 The cue word sequence is encoded into the Transformer codec. The cue word sequence is used to provide additional contextual information for the Transformer codec to help the model better understand the target features in the input image. The cue word update module measures the degree of difference between the recognition results of the insulator defect recognition model and the annotation information through the cosine similarity coefficient and dynamically adjusts the content of the cue word sequence according to the degree of difference. The cosine similarity coefficient includes:

[0037]

[0038] In the above formula, x is the identification result of the insulator defect identification model, P n is the current prompt word sequence;

[0039] The cosine similarity coefficient is used to measure the similarity between the recognition result and the prompt word sequence. The closer the value is to 1, the more similar they are.

[0040] At the same time, in each training iteration, the prompt word update module updates the prompt word through the gradient descent method. The gradient descent method updates the prompt word through the mean square error loss function M and the current prompt word sequence P. n Calculate the gradient of the prompt word sequence and update the prompt word sequence. The gradient descent method includes:

[0041]

[0042] In the above formula, μ is the learning rate, which is used to control the step size of each update. is the partial derivative of the mean square error loss function with respect to the recognition result, is the partial derivative of the recognition result with respect to the prompt word sequence, and BackPropagation is the back propagation algorithm;

[0043] In each training iteration, the prompt word update module updates the prompt word sequence according to the gradient descent method until the recognition accuracy of the Transformer codec reaches the optimal level. The criterion for determining whether the recognition accuracy reaches the optimal level is that the average recognition accuracy rate reaches the highest level.

[0044] The training steps of the insulator defect recognition model include:

[0045] Use the training set to train the model and select the model with the best recognition effect through P, R, mAP@0.5, and mAP@0.5:0.95;

[0046] The indicator P represents the accuracy of the model, and its calculation formula includes:

[0047]

[0048] In the above formula, tp is the number of correctly predicted objects, fp is the number of incorrectly predicted objects, and 1E-16 is a constant term;

[0049] The indicator R represents the recall rate of the model, and its calculation formula includes:

[0050]

[0051] In the above formula, tp represents the number of correctly predicted objects, fn represents the number of correct objects predicted as incorrect, and 1E-16 is a constant term;

[0052] mAP@0.5 represents the average accuracy of each category of images when the intersection over union (IoU) is set to 0.5. The intersection over union (IoU) is the threshold for the intersection over union (IoU). Finally, the average value of all categories of images is calculated to obtain mAP.

[0053] The mAP@0.5:0.95 represents the average accuracy of each category of images when the intersection over union ratio starts at 0.5 and gradually increases to 0.95 in steps of 0.05. The intersection over union ratio is the threshold of IoU. Finally, the average value of all categories of images is calculated to obtain mAP.

[0054] The training step of the insulator defect recognition model further includes:

[0055] The power consumption performance coefficient is calculated by the evaluation index, and the model with the highest power consumption performance coefficient is selected as the final insulator defect identification model. The evaluation index controlling the power consumption performance coefficient includes:

[0056]

[0057] In the above formula, PinC is the power performance coefficient, Memory is the storage consumption of the algorithm, and K is the average CPU occupancy within one hour of operation.

[0058] The insulator dataset includes images of normal insulators, images of faulty insulators, and images of defects on the faulty insulators;

[0059] The pre-processing step comprises:

[0060] Performing data augmentation on the insulator dataset by using a feature enhancement method, wherein the feature enhancement method includes a mosaic method, a color space adjustment method, and a rotation and translation transformation method;

[0061] The augmented insulator dataset is augmented using a consistency model that includes:

[0062] x k-1 =α(x k -γε θ (x k ,k)+N(0,σ 2 I));

[0063]

[0064] In the above formula, x k represents the state vector at step k, α is the scaling factor, which is used to control the attenuation or amplification degree during the state transition process, N(0,σ 2 I) represents a multidimensional Gaussian distribution, where the mean vector is the zero vector, I is the identity matrix, and σ 2 is the variance, the Gaussian distribution is used to introduce random noise, γ is the learning rate, which is used to control the update amplitude, is the objective function E(x k ) About parameter x k The gradient is a vector, the direction of the gradient points to the direction in which the function value increases fastest, and the consistency model uses the negative gradient direction to update the state;

[0065] Performing data augmentation on the data-augmented insulator dataset using a generative adversarial network, wherein the generative adversarial network includes a generator and a discriminator, both of which are neural networks. The generator is configured to take a random low-dimensional noise vector as input and output an image of a specific size and number of channels that has the same characteristics as the real data. The discriminator is configured to distinguish the image output by the generator from the real data, and the output of the discriminator is the probability that the input data is the real data.

[0066] The insulator dataset is pre-processed for image enhancement using a denoising algorithm and a sharpening algorithm. The denoising algorithm is used to reduce the impact of image noise on image processing through Gaussian filtering, and the sharpening algorithm is used to enhance image details and improve target detection accuracy through the Sobel operator. The sharpening algorithm includes:

[0067]

[0068] In the above formula, H1 and H2 are both convolution kernel matrices;

[0069] The insulator defect recognition model uses matching-aware loss as a loss function. The matching-aware loss adjusts the penalty based on the matching quality by combining the target box intersection-over-union ratio and classification confidence between the matching query and the target. The matching-aware loss includes:

[0070]

[0071] In the above formula, MAL() is the matching perception loss, p is the target classification confidence, q is the target box intersection over union (IoU), y is the category identifier, when y=1, the target is the foreground, and when y=0, the target is the background.

[0072] The training steps of the generative adversarial network include:

[0073] The training of the generator includes sampling a batch of noise vectors from a noise distribution, inputting the generator to generate a set of forged data, inputting the forged data into the discriminator, calculating the probability that the discriminator judges the forged data to be real, calculating the loss of the generator using a binary cross entropy loss function, and backpropagating to update the parameters of the generator so that the generated data is closer to the real data;

[0074] The training of the discriminator includes extracting a batch of real data from a real data set and generating a group of fake data through a generator, inputting the real data and the fake data into the discriminator respectively, calculating the probability that the discriminator judges the real data as real and the probability that the fake data is fake, calculating the loss of the discriminator using a binary cross entropy loss function, and backpropagating to update the parameters of the discriminator so that it can better distinguish between real data and fake data;

[0075] Repeat the training of the generator and the discriminator until the difference between the generator loss and log2 is less than 0.05 and the fluctuation is less than 0.05, and the average value of the discriminator's last 100 judgments is greater than 0.6.

[0076] A transmission line insulator detection system suitable for a power grid line patrol drone, the system is used to execute the aforementioned transmission line insulator detection method suitable for a power grid line patrol drone, specifically comprising: a data collection module, a data collection module, and a defect detection module;

[0077] The data collection module is used to obtain the insulator data set in the power grid, and perform representative detail marking and preprocessing on the insulator data set;

[0078] The model building module is used to build an insulator defect recognition model based on the compressed DEIM architecture fine-tuned and optimized by the prompt words, and train the insulator defect recognition model based on the preprocessed insulator data set to obtain a trained insulator defect recognition model;

[0079] The defect detection module is used to deploy the trained insulator defect recognition model on the line patrol drone, obtain the images of each insulator in the line patrol mission through the line patrol drone, adopt a dense matching strategy to identify the insulator images, and upload the identified fault information and images to the drone's nest.

[0080] A transmission line insulator detection device suitable for a power grid patrol drone includes a memory and a processor, wherein the memory is used to store computer program code and transmit the computer program code to the processor;

[0081] The processor is used to execute the aforementioned transmission line insulator detection method applicable to a power grid patrol drone according to the instructions in the computer program code.

[0082] A computer storable medium stores a computer program, which is executed by a processor to implement the aforementioned transmission line insulator detection method applicable to a power grid patrol drone.

[0083] Compared with the prior art, the present invention has the following beneficial effects:

[0084] 1. In a transmission line insulator detection method suitable for a power grid patrol drone, the present invention constructs an insulator defect recognition model based on a compressed DEIM architecture optimized by prompt word fine-tuning. The insulator defect recognition model effectively lightweights the model by discarding edge weights that have little impact on the recognition effect and reducing the number of codec layers, so that it can be carried on a drone. Then, the drone can perform real-time insulator defect detection during the power grid patrol process, and upload the identified fault information and images to the drone's nest without transmitting all insulator images. Therefore, this design can perform real-time insulator defect detection by a drone equipped with an insulator defect recognition model, effectively reducing data transmission pressure and enhancing timeliness.

[0085] 2. In a transmission line insulator inspection method suitable for power grid patrol drones, this invention employs a representative detail annotation method and a dense matching strategy to annotate insulator defects. The representative detail annotation method segments the captured insulator image into multiple small blocks and labels each defective block. The dense matching strategy increases the number of positive samples in each image, thereby guiding the model to focus on the target's true features, thereby better identifying local insulator defects. Therefore, this design can identify local insulator defects through the representative detail annotation method and dense matching strategy, effectively improving detection accuracy.

[0086] 3. In the transmission line insulator detection method suitable for power grid patrol drones of the present invention, the insulator defect recognition model adopts the DEIM architecture. This architecture model has high training costs, but consumes less hardware computing power after model deployment. At the same time, this design compresses and improves the DEIM architecture through prompt word fine-tuning and optimization, further improving the algorithm's operating efficiency and saving power. This allows the design to only carry less memory, reducing computing power requirements, reducing energy consumption, and reducing drone load and volume requirements, thereby extending the drone's flight time and achieving good recognition results under the premise of controlling storage consumption. Therefore, this design can reduce the algorithm hardware requirements by compressing the DEIM architecture and effectively adapt to the drone's operating environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] Figure 1 is a flow chart of the method of the present invention.

[0088] Figure 2 It is a logic flow chart used by the present invention.

[0089] Figure 3 It is a simplified structural diagram of the insulator defect identification model in the present invention.

[0090] Figure 4 It is a logical diagram of the insulator defect identification model in the present invention.

[0091] Figure 5 It is a structural diagram of the system of the present invention.

[0092] Figure 6 This is a structural diagram of the device described in Example 3.

[0093] Figure 7 This is a sample diagram of the Mosaic method of the present invention.

[0094] Figure 8 This is a demonstration effect diagram of the image preprocessing algorithm of the present invention. DETAILED DESCRIPTION

[0095] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0096] Example 1:

[0097] See also Figures 1 to 2 A transmission line insulator detection method suitable for power grid patrol drones obtains an insulator dataset from the power grid, performs representative detail annotation and preprocessing on the insulator dataset. The representative detail annotation involves dividing defective insulators into multiple small blocks and annotating the defective small blocks separately.

[0098] An insulator defect recognition model is constructed based on the DEIM architecture with compressed and improved matching for fast convergence based on cue word fine-tuning optimization. The insulator defect recognition model is trained based on the preprocessed insulator dataset to obtain a trained insulator defect recognition model.

[0099] The trained insulator defect recognition model is deployed on a line patrol drone. The drone obtains images of each insulator in the line patrol mission. The insulator images are identified using the trained insulator defect recognition model, and the identified fault information and images are uploaded to the drone's nest.

[0100] The insulator dataset includes images of normal insulators, images of faulty insulators, and images of defects on the faulty insulators;

[0101] During the acquisition phase, representative details of the target are annotated to adapt to the dense matching strategy of the insulator defect recognition model. For most natural image targets, their representative detail features are the main distinguishing points. Insulator defects often appear as local cracks, damage, or stains. These detail features are the key to distinguishing normal insulators from defective insulators. When annotating the dataset, the representative detail annotation method does not annotate the entire target, but instead annotates the representative details of the target one by one.

[0102] The pre-processing step comprises:

[0103] See also Figure 7 , performing data augmentation on the insulator dataset by using a feature enhancement method, wherein the feature enhancement method includes a mosaic method, a color space adjustment method, and a rotation and translation transformation method;

[0104] The augmented insulator dataset is augmented using a consistency model that includes:

[0105] x k-1 =α(x k -γε θ (x k ,k)+N(0,σ 2 I));

[0106]

[0107] In the above formula, x k represents the state vector at step k, α is the scaling factor, which is used to control the attenuation or amplification degree during the state transition process, N(0,σ 2 I) represents a multidimensional Gaussian distribution, where the mean vector is the zero vector, I is the identity matrix, and σ 2 is the variance, the Gaussian distribution is used to introduce random noise, γ is the learning rate, which is used to control the update amplitude, is the objective function E(x k ) About parameter x k The gradient is a vector, the direction of the gradient points to the direction in which the function value increases fastest, and the consistency model uses the negative gradient direction to update the state;

[0108] Performing data augmentation on the data-augmented insulator dataset using a generative adversarial network, wherein the generative adversarial network includes a generator and a discriminator, both of which are neural networks. The generator is configured to take a random low-dimensional noise vector as input and output an image of a specific size and number of channels that has the same characteristics as the real data. The discriminator is configured to distinguish the image output by the generator from the real data, and the output of the discriminator is the probability that the input data is the real data.

[0109] See also Figure 8 The insulator dataset is pre-processed for image enhancement using a denoising algorithm and a sharpening algorithm. The denoising algorithm is used to reduce the impact of image noise on image processing through Gaussian filtering, and the sharpening algorithm is used to enhance image details and improve target detection accuracy through the Sobel operator. The sharpening algorithm includes:

[0110]

[0111] In the above formula, H1 and H2 are both convolution kernel matrices;

[0112] The insulator defect recognition model uses matching-aware loss as a loss function. The matching-aware loss adjusts the penalty based on the matching quality by combining the target box intersection-over-union ratio and classification confidence between the matching query and the target. The matching-aware loss includes:

[0113]

[0114] In the above formula, MAL() is the matching perception loss, p is the target classification confidence, q is the target box intersection over union (IoU), y is the category identifier, when y=1, the target is the foreground, and when y=0, the target is the background;

[0115] The core idea of ​​the dense matching strategy is to increase the number of positive samples in each image, so as to guide the insulator defect recognition model to pay attention to the true features of the target. The dense matching strategy is used in conjunction with the above representative detail labeling method. By decomposing the overall labeling of the large target into one-to-one labeling of representative details, the insulator defect recognition model naturally ignores unnecessary features and realizes one-time recognition of the main target. In the insulator defect detection task, the combination of the representative detail labeling method and the dense matching strategy enables the insulator defect recognition model to better identify local defects of the insulator. At the same time, it can avoid the use of non-maximum suppression modules, thereby improving the operating efficiency and stability of the insulator defect recognition model, thereby meeting the real-time and reliability requirements of power grid inspection.

[0116] See also Figures 3 and 4 , the insulator defect recognition model includes a convolutional neural network (CNN) feature extraction network and a feature extraction and recognition network;

[0117] The CNN feature extraction network is used to extract features from the input insulator image and convert the information in the insulator image into a feature map suitable for subsequent processing. The feature extraction and recognition network is used to extract and recognize defect features in the feature map.

[0118] The feature extraction and recognition network includes a Transformer codec and a feedforward prediction network. The input of the Transformer codec is a feature map, and the output of the Transformer codec is feature information. The feature information is the bounding box coordinates of the target object output in a normalized form. The bounding box coordinates include the center point, width, and height of the target box. For each predicted target box, the decoding layer also outputs the probability distribution of the target belonging to each category. The decoding layer outputs a probability vector for each predicted box with a number of dimensions equal to the number of target categories. Each dimension corresponds to a category. The value in the vector represents the probability that the target box belongs to the corresponding category. The probability that the target box belongs to the corresponding category is the confidence level.

[0119] The input of the feedforward prediction network is feature information, and the output of the feedforward prediction network is the recognition result of the corresponding insulator image. Based on the input information, the feedforward prediction network associates the target box with its possible corresponding target category through a multi-layer perceptron, and selects the target category with the highest confidence through an activation function. The recognition result is the defect type, coordinate information and confidence level of the target;

[0120] The feature extraction and recognition network also includes a prompt word update module, which is used to provide additional contextual information to the insulator defect recognition model through recognition results, helping the model better understand the target features in the input image. The prompt word update module generates an empty context information sequence during the first round of training, and uses the output recognition results to calculate the loss and update the context information sequence in the final stage of each subsequent round of training. The context information sequence and the feature map are used together as the input of the Transformer codec.

[0121] The context information sequence initially generated by the prompt word updating module is an empty sequence.

[0122] The CNN feature extraction network uses the ResNet50 network as the backbone network. The ResNet50 network includes multiple convolutional layers, pooling layers, and residual connection layers. Each of the convolutional layers includes multiple convolution kernels. Each of the convolutional layers is used to slide the multiple convolution kernels on the image to perform convolution operations, thereby extracting different feature maps. At the same time, downsampling is performed through convolution kernels or pooling layers with a step size greater than 1, so that as the number of convolutional layers increases, the number of extracted feature map channels gradually increases, while the image size gradually decreases.

[0123] The residual connection layer includes a main path and a shortcut connection, wherein the main path is used for convolution operation and the shortcut connection is used to directly add the input to the output of the main path;

[0124] The input of the CNN feature extraction network is:

[0125]

[0126] In the above formula, Image input Represents the input image of the CNN feature extraction network, R is the input image array, the image array has three dimensions, 3 is the RGB three channels, H0 is the height of the input image, and W0 is the width of the input image;

[0127] The output of the CNN feature extraction network is:

[0128]

[0129] In the above formula, C is the feature map, H1 is the height of the output image, and W1 is the width of the output image;

[0130] In the Transformer codec, in order to reduce the performance consumption during operation and improve the operating efficiency, we simplified the Transformer part of the insulator defect recognition model and reduced the number of layers of the encoder and decoder from 6 to 4 to save operating computing power and improve the operational efficiency of the algorithm, so as to better adapt to the application environment corresponding to the present invention. The number of layers of the Transformer codec is 4. The encoding layer includes a multi-head self-attention mechanism and a feedforward neural network. The multi-head self-attention mechanism and the feedforward neural network are surrounded by residual connections and normalization technology. The encoding layer is used to capture the context information of the feature map and convert the context information in the feature map into a vector form. The output of the encoding layer is:

[0131]

[0132] In the above formula, Encode is the output of the encoding layer, LayerNorm() is the normalization function, Attention() is the multi-head self-attention mechanism, FFN() is the feedforward neural network, PromptToken is the prompt word sequence, and X is the input of the encoding layer. The input of the encoding layer is the result of splicing the feature map, position information, and prompt word sequence after feature extraction in order;

[0133] The decoding layer includes a masked multi-head self-attention mechanism, an encoder-decoder attention mechanism, and a feedforward neural network. The masked multi-head self-attention mechanism, the encoder-decoder attention mechanism, and the feedforward neural network all use residual connections and normalization techniques. The decoding layer is used to generate a sequence containing feature information and context information based on the context information provided by the encoder and the information of the feature map itself. The output of the decoding layer is:

[0134] Decode

[0135]

[0136] In the above formula, Decode is the output of the decoder, Encode is the output of the encoder, FFN() is the feedforward neural network, PromptToken is the prompt word sequence, LayerNorm() is the normalization function, CrossAttention() is the encoder-decoder attention mechanism, and Query refers to the target query, which is a set of learnable vectors; the target query is initialized to a random value at the beginning of training and is continuously updated during model training. Each Query vector represents a potential target to be detected by the model. When output, the parameter contains the category and position information of the target, and the category and target box of each target are obtained after subsequent forward prediction network parsing. For example, in the target detection task of the present invention, if the model is expected to detect a maximum of 100 targets in the image, 100 Query vectors will be set, but targets with too low confidence levels will be discarded during the forward prediction network parsing process, and only targets with confidence levels greater than 90% will be retained.

[0137] The feedforward prediction network includes three fully connected layers, and is used to extract the output of the Transformer decoder as the corresponding image recognition result to obtain the coordinate information and confidence level of the target.

[0138] The Transformer codec further includes a position information code, which is used to add a position code to the input of the Transformer codec to introduce sequence information. The position information code includes:

[0139]

[0140] In the above formula, PE() is the position information encoding function, pos is the position number corresponding to the image segmentation, i is the index variable used to control the loop traversal, and d is the encoding dimension.

[0141] In the insulator defect recognition model, a prompt word fine-tuning module is added to compensate for the loss of accuracy caused by the reduction in the number of encoder and decoder layers. This compensates for this loss of accuracy, allowing the lightweight insulator defect recognition model to still maintain high recognition accuracy. Since the present invention focuses on the detection of insulator defects in power grids, a target with a high degree of commonality, a suitable prompt word sequence can effectively capture the common characteristics of insulator defects, thereby improving the recognition capability of the insulator defect recognition model. The working steps of the prompt word update module include:

[0142] Initialize the prompt word sequence to an empty sequence, and use the initialized prompt word sequence, the feature map output by the CNN feature extraction network, and its corresponding position encoding information as the initial input of the Transformer encoder-decoder;

[0143] In each training iteration, the prompt word sequence is encoded into the insulator defect recognition model together with the input image sequence. The prompt word sequence is used to provide additional contextual information for the insulator defect recognition model, helping it better understand the target features in the input image. The prompt word update module measures the degree of difference between the recognition results of the insulator defect recognition model and the annotation information through the cosine similarity coefficient and dynamically adjusts the content of the prompt word sequence based on the degree of difference. The cosine similarity coefficient includes:

[0144]

[0145] In the above formula, x is the recognition result of the insulator defect recognition model, and P is the current prompt word sequence;

[0146] The cosine similarity coefficient is used to measure the similarity between the recognition result and the prompt word sequence. The closer the value is to 1, the more similar they are.

[0147] At the same time, in each training iteration, the prompt word update module updates the prompt word through the gradient descent method. The gradient descent method updates the prompt word through the mean square error loss function M and the current prompt word sequence P. n Calculate the gradient of the prompt word sequence and update the prompt word sequence. The gradient descent method includes:

[0148]

[0149] In the above formula, μ is the learning rate, which is used to control the step size of each update. is the partial derivative of the mean square error loss function with respect to the recognition result, It is the partial derivative of the recognition result with respect to the prompt word sequence, which is calculated by the back propagation algorithm;

[0150] Finally, in each training iteration, the prompt word update module updates the prompt word sequence according to the above formula until the prompt word sequence can fully capture the characteristics of the insulator defect, so that the recognition accuracy of the Transformer codec is optimized. The criterion for achieving optimal recognition accuracy is to achieve the highest average recognition accuracy rate.

[0151] In the same drone insulator defect recognition dataset, the peak memory consumption of the lightweight object detection model based on YOLOv5 was about 2GB, and the average recognition accuracy (mAP) was 0.53; the peak memory consumption of the object detection model based on ViT was about 10GB, and the average recognition accuracy (mAP) was 0.76; while the peak memory consumption of the proposed method was about 6GB, and the average recognition accuracy (mAP) was 0.83.

[0152] Since peak memory consumption is reduced by at least 40%, this design requires less memory, which reduces computing power requirements, energy consumption, and the drone's payload and volume requirements, thereby extending the drone's flight time. It can be considered that the present invention achieves good recognition results while controlling storage consumption.

[0153] The insulator defect recognition model adopts the DEIM architecture. The training of this architecture model is very expensive, but the hardware computing power consumed after the model is deployed is relatively low. At the same time, this design compresses and improves the DEIM architecture through prompt word fine-tuning optimization, further improving the algorithm's operating efficiency and saving power consumption.

[0154] Example 2:

[0155] The training steps of the generative adversarial network include:

[0156] The training of the generator includes sampling a batch of noise vectors from a noise distribution, inputting the generator to generate a set of forged data, inputting the forged data into the discriminator, calculating the probability that the discriminator judges the forged data to be real, calculating the loss of the generator using a binary cross entropy loss function, and backpropagating to update the parameters of the generator so that the generated data is closer to the real data;

[0157] The training of the discriminator includes extracting a batch of real data from a real data set and generating a group of fake data through a generator, inputting the real data and the fake data into the discriminator respectively, calculating the probability that the discriminator judges the real data as real and the probability that the fake data is fake, calculating the loss of the discriminator using a binary cross entropy loss function, and backpropagating to update the parameters of the discriminator so that it can better distinguish between real data and fake data;

[0158] Repeat the training of the generator and the discriminator until the difference between the generator loss and log2 is less than 0.05 and the fluctuation is less than 0.05, and the average value of the discriminator's last 100 judgments is greater than 0.6.

[0159] The evaluation indicators of the insulator defect recognition model include P, R, mAP@0.5, and mAP@0.5:0.95, and the insulator defect recognition model with the best recognition effect is selected;

[0160] The index P represents the accuracy of the insulator defect recognition model, and its calculation formula includes:

[0161]

[0162] In the above formula, tp is the number of correctly predicted objects, fp is the number of incorrectly predicted objects, and 1E-16 is a constant term;

[0163] The indicator R represents the recall rate of the insulator defect recognition model, and its calculation formula includes:

[0164]

[0165] In the above formula, tp represents the number of correctly predicted objects, fn represents the number of correct objects predicted as incorrect, and 1E-16 is a constant term;

[0166] mAP@0.5 represents the average accuracy of each category of images when the intersection over union (IoU) is set to 0.5. The intersection over union (IoU) is the threshold for the intersection over union (IoU). Finally, the average value of all categories of images is calculated to obtain mAP.

[0167] The mAP@0.5:0.95 represents the average accuracy of each category of images when the intersection over union ratio starts at 0.5 and gradually increases to 0.95 in steps of 0.05. The intersection over union ratio is the threshold of IoU. Finally, the average value of all categories of images is calculated to obtain mAP.

[0168] The training step of the insulator defect recognition model further includes:

[0169] The power consumption performance coefficient is calculated by the evaluation index, and the model with the highest power consumption performance coefficient is selected as the final insulator defect identification model. The evaluation index controlling the power consumption performance coefficient includes:

[0170]

[0171] In the above formula, PinC is the power performance coefficient, Memory is the storage consumption of the algorithm, and K is the average CPU occupancy within one hour of operation.

[0172] Example 3:

[0173] See also Figure 5 , a transmission line insulator detection system suitable for a power grid line patrol drone, the system is used to execute the transmission line insulator detection method suitable for a power grid line patrol drone as described in Example 1, specifically comprising: a data collection module, a data collection module, and a defect detection module;

[0174] The data collection module is used to obtain the insulator data set in the power grid, and perform representative detail marking and preprocessing on the insulator data set;

[0175] The model building module is used to build an insulator defect recognition model based on the compressed DEIM architecture fine-tuned and optimized by the prompt words, and train the insulator defect recognition model based on the preprocessed insulator data set to obtain a trained insulator defect recognition model;

[0176] The defect detection module is used to deploy the trained insulator defect recognition model on the line patrol drone, obtain the images of each insulator in the line patrol mission through the line patrol drone, adopt a dense matching strategy to identify the insulator images, and upload the identified fault information and images to the drone's nest.

[0177] See also Figure 6 , a transmission line insulator detection device suitable for a power grid patrol drone, comprising a memory and a processor, wherein the memory is used to store computer program code and transmit the computer program code to the processor;

[0178] The processor is configured to execute the transmission line insulator detection method applicable to a power grid patrol drone as described in Example 1 according to instructions in the computer program code.

[0179] A computer storable medium stores a computer program, and the computer program is executed by a processor to implement the transmission line insulator detection method suitable for a power grid patrol drone as described in Example 1.

[0180] The above description is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiment. Any equivalent modifications or changes made by ordinary technicians in this field based on the contents disclosed in the present invention should be included in the protection scope recorded in the claims.

Claims

1. A method for detecting insulators of power transmission lines suitable for a power grid patrol drone, characterized in that: Obtain an insulator dataset from the power grid, perform representative detail annotation and preprocessing on the insulator dataset. The representative detail annotation involves dividing defective insulators into multiple small blocks and labeling each defective block separately. Construct an insulator defect recognition model based on a detrended prediction (DEIM) architecture optimized with cue word fine-tuning. The insulator defect recognition model is trained based on the preprocessed insulator dataset to obtain a trained insulator defect recognition model. The trained insulator defect recognition model is deployed on a line patrol drone. The drone obtains images of each insulator in the line patrol mission. The trained insulator defect recognition model is used to identify the insulator image, and the identified fault information and images are uploaded to the drone's nest.

2. A method for detecting transmission line insulators suitable for a power grid inspection drone according to claim 1, characterized in that: The insulator defect recognition model includes a convolutional neural network (CNN) feature extraction network and a feature extraction and recognition network; The CNN feature extraction network is used to extract features from the input insulator image and convert the information in the insulator image into a feature map suitable for subsequent processing. The feature extraction and recognition network is used to extract and recognize defect features in the feature map. The feature extraction and recognition network includes a Transformer codec, a feedforward prediction network, and a prompt word update module. The input of the Transformer codec is the feature map, and the output of the Transformer codec is feature information. The prompt word update module is used to provide additional context information to the insulator defect recognition model through the recognition results, helping the model to better understand the target features in the input image. The context information sequence and the feature map are jointly used as the input of the Transformer codec. The feedforward prediction network includes three fully connected layers. The input of the feedforward prediction network is feature information. The feedforward prediction network is used to extract the output of the Transformer decoder into the corresponding image recognition result. The recognition result is the defect type, coordinate information and confidence level of the target.

3. The method for detecting transmission line insulators suitable for a power grid inspection drone according to claim 2, characterized in that: The CNN feature extraction network uses the ResNet50 network as the backbone network. The ResNet50 network includes multiple convolutional layers, pooling layers, and residual connection layers. Each of the convolutional layers includes multiple convolution kernels. Each of the convolutional layers is used to slide the multiple convolution kernels on the image to perform convolution operations, thereby extracting different feature maps, and downsampling is performed by convolution kernels or pooling layers with a step size greater than 1. The residual connection layer includes a main path and a shortcut connection, wherein the main path is used for convolution operation and the shortcut connection is used to directly add the input to the output of the main path; The input of the CNN feature extraction network is: In the above formula, Image input Represents the input image of the CNN feature extraction network, R is the input image array, the image array has three dimensions, 3 is the RGB three channels, H0 is the height of the input image, and W0 is the width of the input image; The output of the CNN feature extraction network is: In the above formula, C is the feature map, H1 is the height of the output image, and W1 is the width of the output image; The Transformer encoder / decoder has 4 layers and converts the context information in the feature map into a vector form. The output of the encoder is: In the above formula, Encode is the output of the encoder, LayerNorm() is the normalization function, Attention() is the multi-head self-attention mechanism, FFN() is the feedforward neural network, PromptToken is the prompt word sequence, and X is the input of the encoding layer. The input of the encoding layer is the result of splicing the feature map, position information, and prompt word sequence after feature extraction in order; The decoding layer is used to generate a sequence containing feature information and context information based on the context information provided by the encoder and the information of the feature map itself. The output of the decoding layer is: Decode In the above formula, Decode is the output of the decoder, Encode is the output of the encoder, FFN() is the feedforward neural network, PromptToken is the prompt word sequence, LayerNorm() is the normalization function, CrossAttention() is the encoder-decoder attention mechanism, and Query refers to the target query, which is a set of learnable vectors.

4. The method for detecting insulators of a power transmission line suitable for a power grid inspection drone according to claim 2, characterized in that: The Transformer codec further includes a position information code, which is used to add a position code to the input of the Transformer codec to introduce sequence information. The position information code includes: In the above formula, PE() is the position information encoding function, pos is the position number corresponding to the image segmentation, i is the index variable used to control the loop traversal, and d is the encoding dimension.

5. The method for detecting transmission line insulators suitable for a power grid inspection drone according to claim 2, characterized in that: The training steps of the insulator defect recognition model include: The prompt word sequence, the feature map output by the CNN feature extraction network, and its corresponding position encoding information are used as the initial input of the Transformer encoder-decoder; In each training iteration, the cue word sequence is encoded into the Transformer codec along with the feature map. The cue word sequence is used to provide additional contextual information to the Transformer codec, helping the model better understand the target features in the input image. The cue word update module measures the degree of difference between the recognition results of the insulator defect recognition model and the annotation information through the cosine similarity coefficient and dynamically adjusts the content of the cue word sequence based on the degree of difference. The cosine similarity coefficient includes: In the above formula, x is the identification result of the insulator defect identification model, P n is the current prompt word sequence; The cosine similarity coefficient is used to measure the similarity between the recognition result and the prompt word sequence. The closer the value is to 1, the more similar they are. At the same time, in each training iteration, the prompt word update module updates the prompt word through the gradient descent method. The gradient descent method updates the prompt word through the mean square error loss function M and the current prompt word sequence P. n Calculate the gradient of the prompt word sequence and update the prompt word sequence. The gradient descent method includes: In the above formula, μ is the learning rate, which is used to control the step size of each update. is the partial derivative of the mean square error loss function with respect to the recognition result, is the partial derivative of the recognition result with respect to the prompt word sequence, and BackPropagation is the back propagation algorithm; In each training iteration, the prompt word update module updates the prompt word sequence according to the gradient descent method until the recognition accuracy of the Transformer codec reaches the optimal level. The criterion for determining whether the recognition accuracy reaches the optimal level is that the average recognition accuracy rate reaches the highest level.

6. A method for detecting transmission line insulators suitable for a power grid inspection drone according to any one of claims 2 to 5, characterized in that: The training steps of the insulator defect recognition model include: Use the training set to train the model, and evaluate the training results using four indicators: P, R, mAP@0.5, and mAP@0.5:0.95, to select the model with the best recognition effect; The indicator P represents the accuracy of the model, and its calculation formula includes: In the above formula, tp is the number of correctly predicted objects, fp is the number of incorrectly predicted objects, and 1E-16 is a constant term; The indicator R represents the recall rate of the model, and its calculation formula includes: In the above formula, tp represents the number of correctly predicted objects, fn represents the number of correct objects predicted as incorrect, and 1E-16 is a constant term; mAP@0.5 represents the average accuracy of each category of images when the intersection over union (IoU) is set to 0.

5. The intersection over union (IoU) is the threshold for the intersection over union (IoU). Finally, the average value of all categories of images is calculated to obtain mAP. The mAP@0.5:0.95 represents the average accuracy of each category of images when the intersection over union ratio starts at 0.5 and gradually increases to 0.95 in steps of 0.

05. The intersection over union ratio is the threshold of IoU. Finally, the average value of all categories of images is calculated to obtain mAP. The training step of the insulator defect recognition model further includes: The power consumption performance coefficient is calculated by the evaluation index, and the model with the highest power consumption performance coefficient is selected as the final insulator defect identification model. The evaluation index controlling the power consumption performance coefficient includes: In the above formula, PinC is the power performance coefficient, Memory is the storage consumption of the algorithm, and K is the average CPU occupancy within one hour of operation.

7. The method for detecting insulators of a power transmission line suitable for a power grid inspection drone according to claim 1, characterized in that: The insulator dataset includes images of normal insulators, images of faulty insulators, and images of defects on the faulty insulators; The pre-processing step comprises: Performing data augmentation on the insulator dataset by using a feature enhancement method, wherein the feature enhancement method includes a mosaic method, a color space adjustment method, and a rotation and translation transformation method; The augmented insulator dataset is augmented using a consistency model that includes: x k-1 =α(x k -ge θ (x k ,k)+N(0,σ 2 I)); In the above formula, x k represents the state vector at step k, α is the scaling factor, which is used to control the attenuation or amplification degree during the state transition process, N(0,σ 2 I) represents a multidimensional Gaussian distribution, where the mean vector is the zero vector, I is the identity matrix, and σ 2 is the variance, the Gaussian distribution is used to introduce random noise, γ is the learning rate, which is used to control the update amplitude, is the objective function E(x k ) About parameter x k The gradient is a vector, the direction of the gradient points to the direction in which the function value increases fastest, and the consistency model uses the negative gradient direction to update the state; Performing data augmentation on the data-augmented insulator dataset using a generative adversarial network, wherein the generative adversarial network includes a generator and a discriminator, both of which are neural networks. The generator is configured to take a random low-dimensional noise vector as input and output an image of a specific size and number of channels that has the same characteristics as the real data. The discriminator is configured to distinguish the image output by the generator from the real data, and the output of the discriminator is the probability that the input data is the real data. The insulator dataset is pre-processed for image enhancement using a denoising algorithm and a sharpening algorithm. The denoising algorithm is used to reduce the impact of image noise on image processing through Gaussian filtering, and the sharpening algorithm is used to enhance image details and improve target detection accuracy through the Sobel operator. The sharpening algorithm includes: In the above formula, H1 and H2 are both convolution kernel matrices; The insulator defect recognition model uses matching-aware loss as a loss function. The matching-aware loss adjusts the penalty based on the matching quality by combining the target box intersection-over-union ratio and classification confidence between the matching query and the target. The matching-aware loss includes: In the above formula, MAL() is the matching perception loss, p is the target classification confidence, q is the target box intersection over union (IoU), y is the category identifier, when y=1, the target is the foreground, and when y=0, the target is the background; The training steps of the generative adversarial network include: The training of the generator includes sampling a batch of noise vectors from a noise distribution, inputting the generator to generate a set of forged data, inputting the forged data into the discriminator, calculating the probability that the discriminator judges the forged data to be real, calculating the loss of the generator using a binary cross entropy loss function, and backpropagating to update the parameters of the generator so that the generated data is closer to the real data; The training of the discriminator includes extracting a batch of real data from a real data set and generating a group of fake data through a generator, inputting the real data and the fake data into the discriminator respectively, calculating the probability that the discriminator judges the real data as real and the probability that the fake data is fake, calculating the loss of the discriminator using a binary cross entropy loss function, and backpropagating to update the parameters of the discriminator so that it can better distinguish between real data and fake data; Repeat the training of the generator and the discriminator until the difference between the generator loss and log2 is less than 0.05 and the fluctuation is less than 0.05, and the average value of the discriminator's last 100 judgments is greater than 0.

6.

8. A matching accelerated transmission line insulator defect detection system suitable for power grid patrol drones, characterized in that: The system is used to perform the transmission line insulator detection method applicable to a power grid patrol drone as claimed in any one of claims 1 to 7, and specifically comprises: a data collection module, a data collection module, and a defect detection module; The data collection module is used to obtain the insulator data set in the power grid, and perform representative detail marking and preprocessing on the insulator data set; The model building module is used to build an insulator defect recognition model based on the detrended DEIM architecture fine-tuned and optimized by the prompt words, and train the insulator defect recognition model based on the preprocessed insulator data set to obtain a trained insulator defect recognition model; The defect detection module is used to deploy the trained insulator defect recognition model on the line patrol drone, obtain the images of each insulator in the line patrol mission through the line patrol drone, adopt a dense matching strategy to identify the insulator images, and upload the identified fault information and images to the drone's nest.

9. A matching accelerated transmission line insulator defect detection device suitable for power grid patrol drones, characterized in that: comprising a memory and a processor, wherein the memory is configured to store computer program code and transmit the computer program code to the processor; The processor is configured to execute the transmission line insulator detection method applicable to a power grid patrol drone according to any one of claims 1 to 7 according to the instructions in the computer program code.

10. A computer storable medium storing a computer program, wherein: The computer program is executed by a processor to implement the transmission line insulator detection method suitable for a power grid patrol drone as described in any one of claims 1 to 7.

Images