Power transmission line multi-defect detection method and system based on improved YOLOv8s

Abstract

The invention relates to a power transmission line multi-defect detection method and system based on improved YOLOv8s, and the method comprises the steps: collecting insulator defects and bird nest images on a power transmission line, and obtaining an initial data set; adopting an SAM large model to preprocess the initial data set to obtain a target data set; dividing the target data set into a training set, a verification set and a test set; taking the YOLOv8s network model as a reference model to construct an improved YOLOv8s network model as a power transmission line multi-defect detection network model; inputting the training set into a model for model training; inputting a verification set into the trained power transmission line multi-defect detection network model for model evaluation; and inputting the test set into the model to obtain a power transmission line defect detection result, and evaluating the model. According to the improved YOLOv8s network model, the problems of missing detection and false detection of defects can be avoided, the detection precision of the model is improved, and the light weight of the model is considered at the same time.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and system for detecting multiple defects in a power transmission line based on improved YOLOv8s. Background Art

[0002] In recent years, with the continuous development of the electric power industry, the number of power grid equipment built in complex environments has also increased day by day, which has also put forward higher requirements for the safe maintenance of power system equipment.

[0003] However, with the continuous expansion of the scale of the power grid, transmission lines are prone to failure under the influence of external environmental factors. Traditional transmission line operation failures include insulator disconnection, bird nests on the top of power towers, pollution flashover failures, etc. Insulators are important components to ensure the normal operation of transmission lines, and they are also the most likely to fail in transmission lines. Insulators are exposed to complex environments for a long time and will be affected by adverse factors such as ice damage, wind damage, and lightning, resulting in varying degrees of failure. Severe faults may affect the operation of the entire power system, leading to large-scale power outages and huge economic losses. Traditional fault classification methods based on mathematical modeling and signal processing usually require manual or robot detection, which not only consumes a lot of human resources, but also easily leads to detection errors.

[0004] Therefore, it is necessary to provide a new transmission line multi-defect detection method and system based on improved YOLOv8s. Summary of the invention

[0005] Based on the above-mentioned problems existing in the prior art, the purpose of the embodiments of the present invention is to provide a transmission line multi-defect detection method and system based on improved YOLOv8s, which can efficiently and accurately detect and extract transmission line defects in the target-dense background of transmission lines, avoid the problems of defect omission and false detection, improve the detection accuracy of the model while taking into account the lightweight of the model.

[0006] To achieve the above object, the technical solution adopted by the present invention is: a transmission line multi-defect detection method based on improved YOLOv8s, comprising:

[0007] S1, collect images of insulator defects and bird nests on the transmission line to obtain the initial data set;

[0008] S2, preprocessing the initial data set using the SAM large model to obtain a target data set;

[0009] S3, dividing the target data set into a training set, a validation set and a test set;

[0010] S4, using the YOLOv8s network model as the benchmark model to build an improved YOLOv8s network model as a transmission line multi-defect detection network model;

[0011] S5, inputting the training set into the transmission line multi-defect detection network model for model training, and improving the transmission line multi-defect detection network model parameters based on the training results to obtain the trained transmission line multi-defect detection network model;

[0012] S6, inputting the verification set into the trained power transmission line multi-defect detection network model for model evaluation;

[0013] S7, inputting the test set into the trained power transmission line multi-defect detection network model to obtain a power transmission line defect detection result.

[0014] Furthermore, in S1, the insulator defects and bird nest images on the transmission line are collected to obtain an initial data set, including:

[0015] The drone is equipped with a high-resolution camera so that it can clearly capture the details of the transmission line. All the collected images are then transferred to a local storage device or server. The images are preliminarily screened to remove images that are too poor to be used due to shooting errors such as blur, overexposure or underexposure, and poor shooting angles. Finally, the remaining images are classified and sorted according to the recorded metadata information, and the insulator defect images and bird's nest images are stored in folders or database tables to preliminarily form the collected initial data set.

[0016] Further, in S2, the initial data set is preprocessed using the SAM large model to obtain the target data set, including: using the SAM large model to segment the images in the initial data set, extracting the masks of the defects of the original images, and then randomly pasting the segmented objects onto another image by copy-paste;

[0017] Randomly paste the segmented object onto another image. The specific formula for mixed pasting is:

[0018] y=I1*α+I2*(1-α)

[0019] Among them, y is the output image, I1 is the image where the pasted object is located, I2 is the main image, and α is the mask.

[0020] Further, in S4, the improved YOLOv8s network model is constructed as a transmission line multi-defect detection network model using the YOLOv8s network model as a benchmark model, including:

[0021] Step S41, replacing the C2f modules of the 7th and 9th layers in the backbone network with C2f-M2SA-CGLU modules;

[0022] Step S42, adding the 11th layer CAFM module to the backbone network;

[0023] Step S43, replace the Conv modules of the 18th and 21st layers in the neck network with LDConv modules.

[0024] Furthermore, the process of extracting features from the LDConv module includes:

[0025] Step S431, according to the size of N, generate the initial sampling coordinates P of the convolution kernel by any size convolution algorithm N , get the original coordinates (P o +P N );

[0026] Step S432, obtaining the offset of the corresponding kernel through convolution operation, whose dimension is (B, 2N, H, W), adding the offset to the original coordinates to generate new sampling coordinates corresponding to the convolution;

[0027] Step S433, obtaining features of corresponding positions through interpolation and resampling.

[0028] Furthermore, the CAFM module is divided into a global branch and a local branch. The global branch adopts a self-attention mechanism to capture a wider range of hyperspectral data information, while the local branch focuses on extracting local features for comprehensive denoising. Convolution and attention complement each other.

[0029] Further, in S6, the inputting of the validation set into the trained power line multi-defect detection network model for model evaluation includes: inputting the test set into the trained power line multi-defect detection network model, and evaluating the power line multi-defect detection network model based on its precision P, recall R, false positive rate F, average precision AP and mean average precision mAP. The indicator formula for evaluating the model performance is as follows:

[0030]

[0031]

[0032]

[0033]

[0034] Among them, TP is the number of positive samples correctly identified as positive samples, FP is the number of negative samples incorrectly identified as positive samples, FN is the number of positive samples incorrectly identified as negative samples, N is the total number of categories of detection targets, AP is the area under the PR curve, and mAP is the average AP of all types of faults detected.

[0035] The transmission line multi-defect detection system based on improved YOLOv8s is applied to the above-mentioned transmission line multi-defect detection method based on improved YOLOv8s, and the system includes:

[0036] A data set acquisition module is used to acquire images of insulator defects and bird nests on transmission lines to obtain an initial data set;

[0037] A data set preprocessing module, used to preprocess the initial data set using the SAM large model to obtain a target data set;

[0038] A data set partitioning module, used to partition the target data set into a training set, a validation set and a test set;

[0039] Improved modeling module, used to build an improved YOLOv8s network model as a transmission line multi-defect detection network model based on the YOLOv8s network model as a benchmark model;

[0040] A model training module is used to input the training set into the transmission line multi-defect detection network model for model training, and improve the parameters of the transmission line multi-defect detection network model based on the training results to obtain the trained transmission line multi-defect detection network model;

[0041] A model verification and evaluation module, used for inputting a verification set into the trained power transmission line multi-defect detection network model for model evaluation;

[0042] The detection result generation module is used to input the test set into the trained power transmission line multi-defect detection network model to obtain the power transmission line defect detection result.

[0043] The embodiment of the present invention further provides a network side server, including:

[0044] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned transmission line multi-defect detection method based on improved YOLOv8s.

[0045] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program, which implements the above-mentioned transmission line multi-defect detection method based on improved YOLOv8s when executed by a processor.

[0046] The beneficial effects of the present invention are as follows: the transmission line multi-defect detection method based on improved YOLOv8s of the present invention comprises collecting insulator defects and bird nest images on the transmission line to obtain an initial data set; preprocessing the initial data set with a SAM large model to obtain a target data set; dividing the target data set into a training set, a validation set and a test set; constructing an improved YOLOv8s network model as a transmission line multi-defect detection network model with a YOLOv8s network model as a benchmark model; inputting the training set into the transmission line multi-defect detection network model for model training, and improving the transmission line multi-defect detection network model parameters based on the training results to obtain the trained transmission line multi-defect detection network model; inputting the validation set into the trained transmission line multi-defect detection network model for model evaluation; and inputting the test set into the trained transmission line multi-defect detection network model to obtain the transmission line defect detection result. The transmission line multi-defect detection method based on improved YOLOv8s of the present invention can efficiently and accurately detect and extract transmission line defects in the target-dense background of transmission lines, avoid the problems of defect omission and false detection, improve the detection accuracy of the model while taking into account the lightweight of the model; the SAM large model is used to preprocess the initial data set, including balancing the number of labels by using image segmentation and image superposition methods, so as to prevent overfitting caused by too many labels of a certain type; the improved C2f-M2SA-CGLU module is used in the Backbone part, and the C2f-M2SA-CGLU module can pay attention to more effective features and can better complete the extraction of small target features; by adding the 11th layer CAFM module to the Backbone network, the design combining convolution and attention enables the model to comprehensively model global and local features, thereby improving the denoising performance; the LDConv module can well complete the irregular convolution feature extraction process, and can flexibly adjust the sampling shape according to the offset, bringing more exploration options for the convolution sampling shape; the LDConv module can easily replace the convolution operations in some efficient modules, thereby improving the network performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0048] In the figure:

[0049] Figure 1 A flowchart of a method for detecting multiple defects in a power transmission line based on improved YOLOv8s provided in Embodiment 1 of the present invention;

[0050] Figure 2 A schematic diagram of a data preprocessing process provided in Embodiment 1 of the present invention;

[0051] Figure 3 A schematic diagram of the structure of the improved YOLOv8s provided in the first embodiment of the present invention;

[0052] Figure 4 A schematic diagram of the structure of the M2SA module provided in the first embodiment of the present invention;

[0053] Figure 5 A schematic diagram of the structure of a CGLU module provided in Embodiment 1 of the present invention;

[0054] Figure 6 A schematic diagram of the structure of a C2f-M2SA-CGLU module provided in Example 1 of the present invention;

[0055] Figure 7 A schematic diagram of the structure of a CAFM module provided in the first embodiment of the present invention;

[0056] Figure 8 A schematic diagram of the structure of the LDConv module provided in the first embodiment of the present invention;

[0057] Fig. 9 A schematic diagram of a module of a power transmission line multi-defect detection system based on improved YOLOv8s provided in the second embodiment of the present invention;

[0058] Fig.10 It is a structural diagram of a network-side server provided according to a third embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] In order to facilitate subsequent understanding, the professional terms appearing in the following embodiments are explained here:

[0061] SAM large model: Segment Anything Model, artificial intelligence image segmentation model.

[0062] M2SA module: Multi-Scale Self-Attention, is a module used to process multi-scale features and enhance self-attention mechanism in deep learning.

[0063] GLU: Gated Linear Unit, gated linear unit.

[0064] CAFM module: Convolution and Attention Fusion Module, convolution and attention fusion module.

[0065] Conv module: Convolutional Module, convolution module.

[0066] LDConv module: Low-Dimensional Convolution, a convolution module used in deep learning architecture.

[0067] First embodiment:

[0068] The first embodiment of the present invention provides a transmission line multi-defect detection method based on improved YOLOv8s, including: collecting insulator defects and bird nest images on the transmission line to obtain an initial data set; using the SAM large model to preprocess the initial data set to obtain a target data set; dividing the target data set into a training set, a validation set and a test set; using the YOLOv8s network model as a benchmark model to construct an improved YOLOv8s network model as a transmission line multi-defect detection network model; inputting the training set into the transmission line multi-defect detection network model for model training, and improving the transmission line multi-defect detection network model parameters based on the training results to obtain the trained transmission line multi-defect detection network model; inputting the validation set into the trained transmission line multi-defect detection network model for model evaluation; and inputting the test set into the trained transmission line multi-defect detection network model to obtain the transmission line defect detection result. The power transmission line multi-defect detection method based on improved YOLOv8s of the present invention can efficiently and accurately detect and extract transmission line defects in the target-dense background of transmission lines, avoid the problems of defect omission and false detection, improve the detection accuracy of the model while taking into account the lightweight of the model.

[0069] The following is a detailed description of the implementation details of the transmission line multi-defect detection method based on improved YOLOv8s in this embodiment. The following content is only for the convenience of understanding the implementation details, which is not necessary for the implementation of this solution. The specific process of this embodiment is as follows Figure 1 As shown, this embodiment is applied to a transmission line multi-defect detection system based on improved YOLOv8s.

[0070] Step S1, collecting images of insulator defects and bird nests on the transmission line to obtain an initial data set.

[0071] Specifically, the drone is equipped with a high-resolution camera, such as a high-definition camera with optical zoom function, so that the details on the transmission line can be clearly captured; and the drone is ensured to have good flight stability and control performance, and can accurately hover and shoot at a safe distance close to the transmission line. All the collected images are then transferred to a local storage device or server; the images are preliminarily screened to remove images that are too poor to be used due to shooting errors such as blur, overexposure or underexposure, and poor shooting angles; finally, the remaining images are classified and sorted according to the recorded metadata information, and the insulator defect images and bird's nest images are stored in folders or database tables to preliminarily form the collected initial data set.

[0072] Step S2, using the SAM large model to preprocess the initial data set to obtain a target data set.

[0073] Specifically, due to the wide variety of defects in electrical equipment on transmission lines, the embodiments of the present invention mainly perform multi-defect recognition on insulator breakage defects, insulator flashover defects, and bird's nests. From the morphological perspective of the images in the initial data set, the types of defects are different, and the texture features presented on the images are also inconsistent. In the case of unbalanced categories, the classification boundaries will "invade" the areas of the minority classes, which will affect the model's learning of more essential features and affect the robustness of the model. By balancing the data set method, the problem of focusing on the majority category in actual prediction is solved.

[0074] When the initial data set has more than 3 categories of labels, in order to balance the number of labels of transmission line defects, the SAM large model is used to segment the images in the initial data set, extract the masks of the defects in the original images, and then randomly paste the segmented objects to another image through copy-paste.

[0075] Furthermore, the segmented object is randomly pasted onto another image. The specific formula for mixed pasting is:

[0076] y=I1*α+I2*(1-α)

[0077] Among them, y is the output image, I1 is the image where the pasted object is located, I2 is the main image, and α is the mask.

[0078] During the pasting process, the random function is used to randomly paste the source image and the target image.

[0079] Specifically, the operation flow chart is as follows Figure 2 As shown, first input an original image such as Figure 2(a) in the figure shows the bird nest defect image of the transmission line. Then the original image Figure 2 (a) is input into the SAM model. The encoder of the SAM model extracts feature maps from the reference image and the test image respectively. The feature map of the reference image is masked and pooled to obtain a feature vector. Then, the cosine similarity between the feature vector and each pixel of the feature map of the test image is calculated, and the point with the highest similarity is used as the foreground cue point. Figure 2 In (b), the point with the lowest similarity is used as the background prompt point. Figure 2 (c) in the figure, and then input it into the Prompt encoder of the SAM model to achieve the segmentation of specific targets, and obtain the mask as follows Figure 2 (d) and the visualization diagram are shown in Figure 2 (e) in ; finally, the segmented object is randomly pasted to another image by copy-paste, as shown in the figure Figure 2 (f) in.

[0080] Furthermore, the initial data set is preprocessed using the SAM large model, including balancing the number of labels by using image segmentation and image superposition methods, thereby preventing overfitting caused by too many labels of a certain type.

[0081] Step S3, dividing the target data set into a training set, a validation set and a test set.

[0082] Specifically, the target dataset images are randomly divided into a training set, a validation set, and a test set in a ratio of approximately 8:1:1.

[0083] Step S4, using the YOLOv8s network model as a benchmark model to construct an improved YOLOv8s network model as a transmission line multi-defect detection network model.

[0084] Specifically, the existing YOLOv8s model is mainly composed of a backbone network, a neck network, and a detection head. The backbone network, as a feature extractor, is responsible for extracting meaningful features from the input image; the neck network acts as a bridge, connecting the backbone network and the detection head, performing feature fusion operations and integrating contextual information; the detection head is the last part of the network, used to generate the network output, such as predicting the category and location of objects in the image. YOLOv8 changes the size of the model by adjusting the network depth and width control factors. There are five models from small to large: v8n, v8s, v8m, v8l, and v8x.

[0085] The detailed structure of the network improved based on YOLOv8s is as follows Figure 3 As shown, the specific steps are as follows:

[0086] Step S41, replacing the C2f modules of the 7th and 9th layers in the backbone network with C2f-M2SA-CGLU modules.

[0087] Specifically, the M2SA module combines the channel attention mechanism, the multi-scale feature mechanism, and the multi-scale self-attention mechanism. Figure 4 As shown, a dual-branch structure is adopted to extract global features and channel information.

[0088] In one of the branches, after the multi-attention scale branch is reduced in dimension by 1x1 convolution, it enters three dilated convolutions for feature extraction, then the features of the three branches are added and fused, and finally the 1x1 convolution result and residual connection are used. Then the multi-head attention calculation is performed. In summary, the formula for the above process is:

[0089]

[0090]

[0091]

[0092] P = AdaptivePool(X out )

[0093] Among them, X i (i=1, 2, 3) are the outputs of three depth-separable sub-convolutions, X is the original input feature, Conv 1×1 is a 1×1 convolution, is a 3×3 depthwise separable sub-convolution with sub-rate R i , X i (i={1,2,3}) is the output of three 1×1 convolution branches, is the element-by-element multiplication, X out is the final output.

[0094] In the other branch, average pooling is first performed to reduce the dimension, and then convolution is performed to reduce the number of channels. After Relu6 and convolution, the sigmoid function is used to obtain the attention score of each channel, and then multiplied with the original input feature to obtain the data feature. Finally, the features of the two branches are fused through the summation operation to obtain the final output feature. The above process can be expressed as:

[0095] X p =Avpool(X)

[0096] X c =ReLu6(Conv 1×1 (X p ))

[0097]

[0098] Among them, X p is the channel attention map, Avpool is the average pooling layer, X is the input feature, X c is the channel attention mapping, ReLu6 is the activation function, ReLu6 is the activation function, Attention c Conv is the output feature. 1×1 is a 1×1 convolution.

[0099] Furthermore, GLU is a channel mixer that has been shown to outperform multilayer perceptrons in various natural language processing tasks. GLU consists of two linear projections that are element-wise multiplied, one of which is activated by a gating function. However, adding a minimal 3×3 depthwise convolution before the activation function of the gating branch of GLU and introducing 3×3 zero-padded convolution into the visual Transformer can be regarded as a conditional position encoding, which effectively captures the position information brought by zero padding. The CGLU module is as follows Figure 5 As shown, the value branch of this model still maintains the same depth as MLP and GLU, making it easy to back-propagate.

[0100] Feature extraction, i.e. the backbone part, is a layer-by-layer mining process of image data. Under standard convolution operation, due to its fixed convolution kernel and small receptive field, multi-classification transmission line fault detection may result in missed detection. The overall structure of C2f-M2SA-CGLU is shown in the figure. Figure 6 As shown in the figure, the improved C2f-M2SA-CGLU module is used in the Backbone part. The C2f-M2SA-CGLU module can focus on more effective features and better extract small target features.

[0101] Step S42, adding the 11th layer CAFM module to the backbone network.

[0102] Specifically, the CAFM module, i.e., the convolution and attention fusion module, is divided into a global branch and a local branch, such as Figure 7 As shown in the figure, the self-attention mechanism is used in the global branch to capture a wider range of hyperspectral data information, while the local branch focuses on extracting local features for comprehensive denoising. Convolution and attention complement each other.

[0103] Among them, the local branch first uses 1×1 convolution to adjust the channel size. After that, a channel transformation operation is performed to further mix the channel information. Channel shuffling divides the input tensor into groups along the channel dimension, where each group uses depth-separable convolution to induce channel shuffling. Subsequently, the output tensors of each group are concatenated along the channel dimension to generate a new output tensor. Subsequently, we use 3×3×3 convolution to extract features. The formula is expressed as:

[0104] F conv =W 3×3×3 (cs(W 1×1 (Y)))

[0105] Among them, F conv is the local branch output, W 3×3×3 is a 3×3×3 convolution, CS is a channel shuffle operation, and W 1×1 is a 1×1 convolution, and Y is the input feature.

[0106] In the global branch, we first generate the query Q, key K, and value V through 1×1 convolution and 3×3 depth convolution to obtain three tensors. Then, we calculate the attention map A through the interaction between Q and K, which reduces the amount of calculation. The output F of the global branch att Defined as:

[0107]

[0108]

[0109] Among them, F att is the output of the global branch, W 1×1 is a 1×1 convolution, Q is a 1×1 convolution and a 3×3 depth convolution to generate the query, K is a 1×1 convolution and a 3×3 depth convolution to generate the key, V is a 1×1 convolution and a 3×3 depth convolution to generate V, and α is a learnable scaling parameter used to control the size of the multiplication of matrices K and Q before applying the softmax function.

[0110] Finally, the output formula calculated by the CAFM module is:

[0111] F out =F att +F conv

[0112] Among them, F out is the output calculated by the CAFM module, F att is the output of the global branch, F conv It is the local branch output.

[0113] By adding the 11th layer of CAFM module to the backbone network, the design combining convolution and attention enables the model to comprehensively model global and local features, thereby improving the denoising performance.

[0114] Step S43, replace the Conv modules of the 18th and 21st layers in the neck network with LDConv modules.

[0115] Specifically, the standard convolution sampling position is fixed, which causes the convolution to only extract local information but not capture information at other locations. The LDConv module can well complete the irregular convolution feature extraction process, and can flexibly adjust the sampling shape according to the offset, bringing more exploration options to the convolution sampling shape.

[0116] like Figure 8 As shown, the feature extraction process of the LDConv module includes the following steps:

[0117] Step S431, according to the size of N, generate the initial sampling coordinates P of the convolution kernel by any size convolution algorithm N , get the original coordinates (P o +P N ).

[0118] Step S432, obtain the offset of the corresponding kernel through convolution operation, whose dimension is (B, 2N, H, W), add the offset to the original coordinates to generate new sampling coordinates corresponding to the convolution.

[0119] Step S433, obtaining features of corresponding positions through interpolation and resampling.

[0120] The LDConv module completes the feature extraction process with any number of parameters by generating initial sampling coordinates, so the LDConv module can easily replace the convolution operations in some efficient modules, thereby improving network performance.

[0121] Step S5, inputting the training set into the power transmission line multi-defect detection network model for model training, and improving the parameters of the power transmission line multi-defect detection network model based on the training results to obtain the trained power transmission line multi-defect detection network model.

[0122] Specifically, the experiment uses the Pytorch framework and calls the GPU for the experiment. The experimental environment and basic training parameters are shown in Table 1.

[0123] Table 1 Experimental environment

[0124]

[0125] Part of the transmission line fault detection dataset comes from drone inspection aerial images, and the other part is data augmented images after image augmentation technology. The collected insulator images are manually annotated using the LabelImg annotation tool in the Python environment. The experiment needs to identify three types of defects: broken insulators, flashover insulators, and bird's nests, so a total of four labels are set: insulator, broken, pollution-flashover, and nest.

[0126] Since the number of defect labels for a single image of a bird's nest defect is far less than that of the other two types of defects, the image segmentation and superposition method is used to randomly increase the number of labels for a single bird's nest image to balance the data set and prevent overfitting. The data set distribution is shown in Table 2, and the label distribution is shown in Table 3.

[0127] Table 2 Dataset distribution

[0128]

[0129] Table 3 Label distribution

[0130]

[0131] Step S6: input the verification set into the trained power transmission line multi-defect detection network model for model evaluation.

[0132] Specifically, the main evaluation indicators of the target detection algorithm are detection accuracy and model complexity. In order to comprehensively and objectively evaluate the performance of the improved YOLOv8s model, the test set is input into the trained power line multi-defect detection network model, and the transmission line multi-defect detection network model is evaluated based on its precision (Precision, P), recall (Recall, R), false positive rate (False, F), average precision (Average Precision, AP), and mean average precision (mAP). The indicator formula for evaluating the model performance is as follows:

[0133]

[0134]

[0135]

[0136]

[0137] Among them, TP is the number of positive samples correctly identified as positive samples, FP is the number of negative samples incorrectly identified as positive samples, FN is the number of positive samples incorrectly identified as negative samples, N is the total number of categories of detection targets, AP is the area under the PR curve, and mAP is the average value of the total AP of various types of faults detected. Among them, the larger the mAP, the better the performance of the network.

[0138] All ablation experiments are conducted on the same dataset, and all convolution training is done from scratch without using weight files. In order to verify the impact of the proposed improved module on the detection effect, YOLOv8s is used as the benchmark model, and ablation experiments are conducted on CAFM, C2f-M2SA-CGLU and LDConv, as shown in Table 4.

[0139] Table 4 Ablation experiment results

[0140]

[0141] Table 5 below shows the detection results of 253 defect images in the test set. Comparison of three types of defects between the improved YOLOv8 model and the original model. The mAP values ​​of each defect of the improved model are higher than those of the original model.

[0142] Table 5 Data results of three types of defects

[0143]

[0144] Step S7, inputting the test set into the trained power transmission line multi-defect detection network model to obtain a power transmission line defect detection result.

[0145] The first embodiment of the present invention provides a transmission line multi-defect detection method based on improved YOLOv8s, comprising: collecting insulator defects and bird nest images on the transmission line to obtain an initial data set; using the SAM large model to preprocess the initial data set to obtain a target data set; dividing the target data set into a training set, a validation set and a test set; using the YOLOv8s network model as a benchmark model to construct an improved YOLOv8s network model as a transmission line multi-defect detection network model; inputting the training set into the transmission line multi-defect detection network model for model training, and improving the transmission line multi-defect detection network model parameters based on the training results to obtain a trained transmission line multi-defect detection network model; inputting the validation set into the trained transmission line multi-defect detection network model for model evaluation; and inputting the test set into the trained transmission line multi-defect detection network model to obtain a transmission line defect detection result. The transmission line multi-defect detection method based on improved YOLOv8s of the present invention can efficiently and accurately detect and extract transmission line defects in the target-dense background of transmission lines, avoid the problems of defect omission and false detection, improve the detection accuracy of the model while taking into account the lightweight of the model, and adopt the SAM large model to preprocess the initial data set, including balancing the number of labels by adopting image segmentation and image superposition methods, so as to prevent overfitting caused by too many labels of a certain type; an improved C2f-M2SA-CGLU module is adopted in the Backbone part, and the C2f-M2SA-CGLU module can pay attention to more effective features and can better complete the extraction of small target features; by adding the 11th layer CAFM module in the Backbone network, the design combining convolution and attention enables the model to comprehensively model global and local features, thereby improving the denoising performance; the LDConv module can well complete the irregular convolution feature extraction process, and can flexibly adjust the sampling shape according to the offset, bringing more exploration options for the convolution sampling shape; the LDConv module can easily replace the convolution operations in some efficient modules, thereby improving the network performance.

[0146] The step division of the above methods is only for the purpose of clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent; adding insignificant modifications to the algorithm or process or introducing insignificant designs without changing the core design of the algorithm and process are all within the scope of protection of this patent.

[0147] Second implementation method:

[0148] like Fig. 9As shown, the second embodiment of the present invention provides a transmission line multi-defect detection system based on improved YOLOv8s, and the system includes: a data set acquisition module 201, a data set preprocessing module 202, a data set partitioning module 203, an improved modeling module 204, a model training module 205, a model verification and evaluation module 206, and a detection result generation module 207.

[0149] Specifically, the data set acquisition module 201 is used to collect insulator defects and bird nest images on the transmission line to obtain an initial data set; the data set preprocessing module 202 is used to preprocess the initial data set using the SAM large model to obtain a target data set; the data set division module 203 is used to divide the target data set into a training set, a validation set and a test set; the improved modeling module 204 is used to construct an improved YOLOv8s network model as a transmission line multi-defect detection network model with the YOLOv8s network model as a benchmark model; the model training module 205 is used to input the training set into the transmission line multi-defect detection network model for model training, and improve the transmission line multi-defect detection network model parameters based on the training results to obtain the trained transmission line multi-defect detection network model; the model verification and evaluation module 206 is used to input the verification set into the trained transmission line multi-defect detection network model for model evaluation; the detection result generation module 207 is used to input the test set into the trained transmission line multi-defect detection network model to obtain the transmission line defect detection result.

[0150] It is not difficult to find that this embodiment is a system embodiment corresponding to the first embodiment, and this embodiment can be implemented in conjunction with the first embodiment. The relevant technical details mentioned in the first embodiment are still valid in this embodiment, and in order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied in the first embodiment.

[0151] It is worth mentioning that all modules involved in this embodiment are logic modules. In practical applications, a logic unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, in order to highlight the innovative part of the present invention, this embodiment does not introduce units that are not closely related to solving the technical problem proposed by the present invention, but this does not mean that there are no other units in this embodiment.

[0152] A third embodiment of the present invention relates to a network side server, such as Fig.10As shown, it includes at least one processor 302; and a memory 301 that is communicatively connected to the at least one processor 302; wherein the memory 301 stores instructions that can be executed by the at least one processor 302, and the instructions are executed by the at least one processor 302 so that the at least one processor 302 can execute the above-mentioned data processing method.

[0153] The memory 301 and the processor 302 are connected in a bus manner, and the bus may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors 302 and the memory 301 together. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor 302 is transmitted on a wireless medium via an antenna, and further, the antenna also receives data and transmits the data to the processor 302.

[0154] The processor 302 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management and other control functions. The memory 301 can be used to store data used by the processor 302 when performing operations.

[0155] The fourth embodiment of the present invention relates to a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the method for detecting multiple defects in a power transmission line based on improved YOLOv8s in the first embodiment is implemented.

[0156] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0157] The above is only an embodiment of the present invention. The common sense such as the known specific structure and characteristics in the scheme is not described in detail here. The ordinary technicians in the relevant field know all the common technical knowledge in the technical field of the invention before the application date or priority date, can obtain all the existing technologies in the field, and have the ability to apply the conventional experimental means before that date. The ordinary technicians in the relevant field can improve and implement this scheme in combination with their own abilities under the enlightenment given by this application. Some typical known structures or known methods should not become obstacles for ordinary technicians in the relevant field to implement this application. It should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several deformations and improvements can be made, which should also be regarded as the protection scope of the present invention, which will not affect the effect of the implementation of the present invention and the practicality of the patent. The protection scope required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

[0158] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-defect detection method for transmission lines based on improved YOLOv8s, characterized in that: include: S1, collect images of insulator defects and bird nests on the transmission line to obtain the initial data set; S2, preprocessing the initial data set using the SAM large model to obtain a target data set; S3, dividing the target data set into a training set, a validation set and a test set; S4, using the YOLOv8s network model as the benchmark model to build an improved YOLOv8s network model as a transmission line multi-defect detection network model; S5, inputting the training set into the transmission line multi-defect detection network model for model training, and improving the transmission line multi-defect detection network model parameters based on the training results to obtain the trained transmission line multi-defect detection network model; S6, inputting the verification set into the trained power transmission line multi-defect detection network model for model evaluation; S7, inputting the test set into the trained power transmission line multi-defect detection network model to obtain a power transmission line defect detection result.

2. The method for detecting multiple defects in a power transmission line based on improved YOLOv8s according to claim 1, characterized in that: In S1, the insulator defects and bird nest images on the transmission line are collected to obtain an initial data set, including: The drone is equipped with a high-resolution camera so that it can clearly capture the details of the transmission line. All the collected images are then transferred to a local storage device or server. The images are preliminarily screened to remove images that are too poor to be used due to shooting errors such as blur, overexposure or underexposure, and poor shooting angles. Finally, the remaining images are classified and sorted according to the recorded metadata information, and the insulator defect images and bird's nest images are stored in folders or database tables to preliminarily form the collected initial data set.

3. The method for detecting multiple defects in a power transmission line based on improved YOLOv8s according to claim 1, characterized in that: In S2, the initial data set is preprocessed using the SAM large model to obtain a target data set, including: using the SAM large model to segment the images in the initial data set, extracting masks of defects in the original image, and then randomly pasting the segmented objects onto another image by copy-paste; Randomly paste the segmented object onto another image. The specific formula for mixed pasting is: y=I1*α+I2*(1-α) Among them, y is the output image, I1 is the image where the pasted object is located, I2 is the main image, and α is the mask.

4. The method for detecting multiple defects in a power transmission line based on improved YOLOv8s according to claim 1, characterized in that: In S4, the improved YOLOv8s network model is constructed as a transmission line multi-defect detection network model using the YOLOv8s network model as a benchmark model, including: Step S41, replacing the C2f modules of the 7th and 9th layers in the backbone network with C2f-M2SA-CGLU modules; Step S42, adding the 11th layer CAFM module to the backbone network; Step S43, replace the Conv modules of the 18th and 21st layers in the neck network with LDConv modules.

5. The method for detecting multiple defects in a power transmission line based on improved YOLOv8s according to claim 4, characterized in that: The process of feature extraction of the LDConv module includes: Step S431, according to the size of N, generate the initial sampling coordinates P of the convolution kernel by any size convolution algorithm N , get the original coordinates (P o +P N ); Step S432, obtaining the offset of the corresponding kernel through convolution operation, whose dimension is (B, 2N, H, W), adding the offset to the original coordinates to generate new sampling coordinates corresponding to the convolution; Step S433, obtaining features of corresponding positions through interpolation and resampling.

6. The method for detecting multiple defects in a power transmission line based on improved YOLOv8s according to claim 4, characterized in that: The CAFM module is divided into a global branch and a local branch. The global branch adopts a self-attention mechanism to capture a wider range of hyperspectral data information, while the local branch focuses on extracting local features for comprehensive denoising. Convolution and attention complement each other.

7. The method for detecting multiple defects in a power transmission line based on improved YOLOv8s according to claim 6, characterized in that: In S6, the step of inputting the validation set into the trained power line multi-defect detection network model for model evaluation includes: inputting the test set into the trained power line multi-defect detection network model, and evaluating the transmission line multi-defect detection network model based on its precision P, recall R, false positive rate F, average precision AP, and mean average precision mAP. The indicator formula for evaluating the model performance is as follows: Among them, TP is the number of positive samples correctly identified as positive samples, FP is the number of negative samples incorrectly identified as positive samples, FN is the number of positive samples incorrectly identified as negative samples, N is the total number of categories of detection targets, AP is the area under the PR curve, and mAP is the average AP of all types of faults detected.

8. A transmission line multi-defect detection system based on improved YOLOv8s, characterized in that: Applied to the transmission line multi-defect detection method based on improved YOLOv8s as described in claim 1, the system comprises: A data set acquisition module is used to acquire images of insulator defects and bird nests on transmission lines to obtain an initial data set; A data set preprocessing module, used to preprocess the initial data set using the SAM large model to obtain a target data set; A data set partitioning module, used to partition the target data set into a training set, a validation set and a test set; Improved modeling module, used to build an improved YOLOv8s network model as a transmission line multi-defect detection network model based on the YOLOv8s network model as a benchmark model; A model training module is used to input the training set into the transmission line multi-defect detection network model for model training, and improve the parameters of the transmission line multi-defect detection network model based on the training results to obtain the trained transmission line multi-defect detection network model; A model verification and evaluation module, used for inputting a verification set into the trained power transmission line multi-defect detection network model for model evaluation; The detection result generation module is used to input the test set into the trained power transmission line multi-defect detection network model to obtain the power transmission line defect detection result.

9. A network side server, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the transmission line multi-defect detection method based on improved YOLOv8s as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for detecting multiple defects in a power transmission line based on improved YOLOv8s as described in any one of claims 1 to 7 is implemented.

Images