YOLOv5 transformer substation target detection method based on multi-scale feature fusion

Abstract

The invention provides a YOLOv5 transformer substation target detection method based on multi-scale feature fusion, and relates to the technical field of target detection and transformer substation safety inspection, and the method comprises the following steps: marking an image in transformer substation inspection image data by adopting a defect label mode, and according to the marked transformer substation image data, carrying out target detection on the transformer substation inspection image data; constructing a target detection data set; performing target candidate frame length-width ratio clustering calculation on the substation inspection image data of the target detection data set to obtain an optimal anchor frame parameter; based on the optimal anchor frame parameter, training the improved YOLOv5 neural network model through the target detection data set, generating a training weight and outputting a prediction result; and performing post-processing on the prediction result through softening non-maximum suppression, and outputting a substation target detection result according to a post-processing result. According to the invention, high accurate measurement precision is obtained in the substation defect detection task, and the actual requirements of substation safety inspection can be met.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection and substation safety inspection, and in particular to a YOLOv5 substation target detection method based on multi-scale feature fusion. Background Art

[0002] Substations are an important part of the power system, and regular inspections are a key measure to ensure the normal operation of substations. Traditional manual inspections are time-consuming and labor-intensive, and are easily affected by human factors, resulting in inconsistent inspection results. With the rapid development of deep learning and computer vision technology, the application of target detection algorithms in automatic inspections has received widespread attention.

[0003] As the current mainstream target detection algorithm, YOLOv5 is widely used in the field of industrial detection due to its good detection accuracy and real-time performance. However, for defect detection in substation scenarios, the existing algorithms still have certain problems of missed detection and false detection.

[0004] Currently, no effective solution has been proposed for the problems in the related technologies. Summary of the invention

[0005] In view of this, the present invention provides a YOLOv5 substation target detection method based on multi-scale feature fusion to solve the above-mentioned problems.

[0006] In order to solve the above problems, the specific technical solutions adopted by the present invention are as follows:

[0007] A YOLOv5 substation target detection method based on multi-scale feature fusion, the method includes the following steps:

[0008] S 1. Collect substation inspection image data, annotate the images in the substation inspection image data using defect labels according to the defect types of the substation, and construct a target detection dataset based on the annotated substation image data;

[0009] S2. Based on the K-means++ clustering algorithm, combined with the hierarchical clustering method and Gaussian mixture model, the aspect ratio clustering calculation of the target candidate box is performed on the substation inspection image data of the target detection dataset to obtain the optimal anchor box parameters;

[0010] S3. Based on the optimal anchor frame parameters, the improved YOLOv5 neural network model is trained through the target detection data set to generate training weights and output prediction results; the prediction results are post-processed by softening non-maximum suppression, and the substation target detection results are output based on the post-processing results.

[0011] Preferably, collecting substation inspection image data, annotating images in the substation inspection image data by means of defect labels according to the defect type of the substation, and constructing a target detection data set according to the annotated substation image data includes the following steps:

[0012] S11, collecting substation inspection image data under different weather conditions, photo conditions and time periods, and annotating defects on the substation inspection image data according to the defect type of the substation, to obtain a data set of substation inspection scenes;

[0013] S12, formatting and resizing the image data in the data set of the substation inspection scene, and dividing the resized data set of the substation inspection scene into a training set, a test set, and a verification set;

[0014] S 1 3. The image data in the training set is expanded by data enhancement technology, and the expanded training set, test set and validation set are combined to obtain the target detection data set.

[0015] Preferably, the defect types of the substation include: open pressure plate, closed pressure plate, hanging suspended objects, bird's nest, broken insulator, damaged dial shell, blurred dial, abnormal door closure, discoloration of respirator silicone, damaged cover and not wearing work clothes.

[0016] Preferably, the data enhancement techniques include: random cropping, rotation, translation, color perturbation and noise addition.

[0017] Preferably, based on the K-means++ clustering algorithm, combined with the hierarchical clustering method and the Gaussian mixture model, the target candidate box aspect ratio clustering calculation is performed on the substation inspection image data of the target detection data set, and the optimal anchor box parameters are obtained, including the following steps:

[0018] S2 1. Extract the aspect ratio of the annotation box of the substation inspection image data according to the training set expanded and processed in the target detection data set, and establish an aspect ratio data set according to the aspect ratio of the annotation box of the substation inspection image data;

[0019] S22, clustering the aspect ratio data set by using the K-means++ clustering algorithm to obtain several initial cluster centers and obtain preliminary clustering results;

[0020] S23. Optimize the preliminary clustering results obtained by using a hierarchical clustering method, and fit the optimization results using a Gaussian mixture model to generate optimal anchor frame parameters.

[0021] Preferably, clustering the aspect ratio data set by using a K-means++ clustering algorithm to obtain a number of initial cluster centers, and obtaining a preliminary clustering result includes the following steps:

[0022] S221, randomly selecting a data point from the aspect ratio data set as an initial centroid, and calculating the shortest distance between the remaining data points and the initial centroid;

[0023] S222, selecting a data point as a new centroid from the remaining data points by probabilistic selection according to the shortest distance between each data point and the initial centroid;

[0024] S223. Recalculate the distances between the remaining data points and the new centroids, and repeat step S222 until a preset number of centroids are selected to obtain a preliminary clustering result.

[0025] Preferably, the obtained preliminary clustering results are optimized by using a hierarchical clustering method, and the optimized results are fitted by using a Gaussian mixture model. The generation of optimal anchor frame parameters includes the following steps:

[0026] S23 1. A bottom-up agglomerative hierarchical clustering method is used, and the centroids in the preliminary clustering results are used as the initial clusters;

[0027] S232, calculating the similarity between the centroids by using the Euclidean distance calculation method, and according to the similarity between the centroids, gradually merging the centroids into clusters through a bottom-up method, generating clustering structures at different levels, and obtaining a hierarchical clustering result;

[0028] S233. Perform probability analysis on the hierarchical clustering results through a Gaussian mixture model, and based on the probability analysis results, extract the mean of each Gaussian distribution as the aspect ratio parameter of the optimal anchor box through a maximum expectation algorithm.

[0029] Preferably, performing probability analysis on the hierarchical clustering results by using a Gaussian mixture model, and extracting the mean of each Gaussian distribution as the aspect ratio parameter of the optimal anchor box by using a maximum expectation algorithm according to the probability analysis results, comprises the following steps:

[0030] S233 1. Based on the hierarchical clustering results, initialize Gaussian distribution parameters for each hierarchical clustering result, where the Gaussian distribution parameters include mean, covariance matrix and mixing coefficient;

[0031] S2332. For each data point in the hierarchical clustering result, calculate the probability that it belongs to each Gaussian distribution;

[0032] S2333. According to the probability that each data point in the hierarchical clustering result belongs to each Gaussian distribution, update the Gaussian distribution parameters of each hierarchical clustering result to maximize the probability that each data point in the hierarchical clustering result belongs to each Gaussian distribution;

[0033] S2334. Repeat steps S2332-S2333 until the preset number of iterations is reached to obtain the Gaussian distribution model of each hierarchical clustering result, and extract the mean of each Gaussian distribution as the aspect ratio parameter of the optimal anchor box according to the Gaussian distribution model of each hierarchical clustering result.

[0034] Preferably, for each data point in the hierarchical clustering result, the calculation formula for calculating the probability that it belongs to each Gaussian distribution is:

[0035]

[0036] In the formula, P(z k |x i ) represents the data point x in the hierarchical clustering result i The probability of belonging to the kth Gaussian distribution;

[0037] N(x i |u k ,C k ) represents the data point x in the hierarchical clustering result i Belongs to the kth Gaussian distribution probability density;

[0038] N(x i |u j ,C j ) represents the data point x in the hierarchical clustering result i Belongs to the jth Gaussian distribution probability density;

[0039] K represents the total number of Gaussian distributions;

[0040] π k The formula for the mixing coefficient of the kth Gaussian distribution;

[0041] j represents the number of Gaussian distributions;

[0042] μj represents the mean of the th Gaussian distribution;

[0043] μ k represents the mean of the kth Gaussian distribution;

[0044] C represents the covariance matrix.

[0045] Preferably, based on the optimal anchor frame parameters, the improved YOLOv5 neural network model is trained through the target detection data set to generate training weights and output prediction results; the prediction results are post-processed by softening non-maximum suppression, and the substation target detection results are output according to the post-processing results, including the following steps:

[0046] S3 1. Configure the training parameters of the improved YOLOv5 neural network model, and use the YOLOv5 regression loss function as the loss value between each predicted box and the true box;

[0047] S32, training the improved YOLOv5 neural network model using the extended training set in the target detection data set; saving the model weights during the training process, using the validation set in the target detection data set, and performing performance testing on the improved YOLOv5 neural network model, and dynamically adjusting the learning rate of the improved YOLOv5 neural network model according to the performance test results;

[0048] S33, after the training of the improved YOLOv5 neural network model is completed, the model weights are loaded into the improved YOLOv5 model, and the test set in the target detection data set is predicted by the improved YOLOv5 model to obtain a prediction result;

[0049] S34. According to the obtained prediction results, the candidate boxes are filtered by softening non-maximum suppression to obtain the substation target detection results, and the substation target detection results are visualized.

[0050] Preferably, the improved YOLOv5 neural network model adopts a multi-level feature detection architecture, and the feature map output sizes of the improved YOLOv5 neural network model include: 19×19, 38×38, 76×76, 104×104 and 208×208.

[0051] Preferably, the regression loss function of YOLOv5 includes: bounding box regression loss, confidence loss and classification loss;

[0052] Bounding box regression loss, which is used to calculate the positioning difference between each predicted box and the true box, using full intersection-over-union loss;

[0053] Confidence loss, used to measure whether the predicted box contains the target, using binary cross entropy loss;

[0054] Classification loss, predicting the category of the target, also uses binary cross entropy loss;

[0055] The regression loss function expression of YOLOv5 is:

[0056] L=λbox*Lbox+λconf*Lconf+λcls*Lcls;

[0057] Where L represents the regression loss function of YOLOv5;

[0058] Lbox represents the bounding box regression loss;

[0059] Lconf represents the confidence loss;

[0060] Lcls represents classification loss;

[0061] λbox represents the weight parameter of the bounding box regression loss;

[0062] λconf represents the weight parameter of confidence loss;

[0063] λcls represents the weight parameter of classification loss.

[0064] Preferably, after the training of the improved YOLOv5 neural network model is completed, the model weights are loaded into the improved YOLOv5 model, and the test set in the target detection data set is predicted by the improved YOLOv5 model to obtain the prediction result, which includes the following steps:

[0065] S33 1. After the model weights are loaded into the improved YOLOv5 model, the test set in the target detection dataset is input into the improved YOLOv5 model to obtain the predicted bounding box, confidence and category probability of each target;

[0066] S332, based on a preset confidence threshold and category probability threshold, and in combination with the confidence and category probability of each target, calculating a comprehensive score of the predicted bounding box;

[0067] S333. According to the calculated comprehensive score of the predicted bounding box, the predicted bounding box is screened, and the predicted bounding box with a comprehensive score higher than the set score threshold is taken as the prediction result.

[0068] Preferably, according to the obtained prediction results, the candidate box is filtered by softening non-maximum suppression to obtain the substation target detection result, and the substation target detection result is visualized, which includes the following steps:

[0069] S341, sorting the obtained prediction results in descending order according to the confidence of the prediction bounding box, and for each predicted bounding box after sorting, calculating the intersection-and-union ratio between it and all other predicted bounding boxes;

[0070] S342, reducing the confidence of the predicted bounding box using a confidence reduction calculation formula according to the intersection-and-union ratio between each predicted bounding box and other predicted bounding boxes;

[0071] S343, comparing the reduced confidence of the predicted bounding box with a preset confidence threshold, and screening out the predicted bounding box that meets the preset confidence threshold;

[0072] S344: The predicted bounding box is selected as the final substation target detection result, and the final substation target detection result is visualized using visualization technology.

[0073] Preferably, the confidence reduction calculation formula is:

[0074] score i '=score i ×(1-IoU(b i , b j ));

[0075] In the formula, score i represents the reduced confidence of the i-th predicted bounding box;

[0076] score i represents the confidence of the i-th predicted bounding box;

[0077] IoU(b i , b j ) represents the intersection-over-union ratio between the i-th predicted bounding box and the j-th predicted bounding box.

[0078] The beneficial effects of the present invention are:

[0079] 1. The present invention significantly improves the detection capability of small objects and subtle defects by using an improved YOLOv5 model and feature fusion strategy. In the 11 common defect detection tasks in substations, the mAP is significantly improved, achieving a higher detection accuracy and having good real-time performance. It is suitable for substation automation inspection systems and achieves a higher precision in substation defect detection tasks, which can meet the actual needs of substation safety inspections.

[0080] 2. The present invention uses K-means++ to perform preliminary centroid selection, which can obtain more dispersed and reasonable initial clustering results. K-means++ gives priority to points with a large distance from the selected centroid during the centroid selection process to improve clustering accuracy and accelerate convergence. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0082] Figure 1 It is a flowchart of a YOLOv5 substation target detection method based on multi-scale feature fusion according to an embodiment of the present invention. DETAILED DESCRIPTION

[0083] In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0084] According to an embodiment of the present invention, a YOLOv5 substation target detection method based on multi-scale feature fusion is provided.

[0085] The present invention is further described with reference to the accompanying drawings and specific embodiments. Figure 1 As shown, according to the YOLOv5 substation target detection method based on multi-scale feature fusion according to an embodiment of the present invention, the method includes the following steps:

[0086] S 1. Collect substation inspection image data, annotate the images in the substation inspection image data using defect labels according to the defect types of the substation, and construct a target detection dataset based on the annotated substation image data;

[0087] As a preferred implementation, collecting substation inspection image data, annotating images in the substation inspection image data by using defect labels according to the defect type of the substation, and constructing a target detection data set based on the annotated substation image data includes the following steps:

[0088] S11, collecting substation inspection image data under different weather conditions, photo conditions and time periods, and annotating defects on the substation inspection image data according to the defect type of the substation, to obtain a data set of substation inspection scenes;

[0089] It should be noted that the dataset based on the substation inspection scene uses 11 common defect types for annotation, namely: open pressure plate, closed pressure plate, hanging suspended objects, bird nest, broken insulator, damaged dial shell, blurred dial, abnormal door closure, discolored respirator silicone, damaged cover, and no work clothes. A large number of substation images in different weather, lighting conditions, and different time periods were collected. The dataset contains high-definition and blurred images to ensure data diversity, with a total of 8,307 images.

[0090] S12, formatting and resizing the image data in the data set of the substation inspection scene, and dividing the resized data set of the substation inspection scene into a training set, a test set, and a verification set;

[0091] S 1 3. The image data in the training set is expanded by data enhancement technology, and the expanded training set, test set and validation set are combined to obtain the target detection data set.

[0092] It should be noted that the dataset is divided into 70% training set, 20% test set, and 10% validation set. In order to make the dataset format compatible with the YOLOv5 training framework, all images and annotations are organized into VOC format, and the size of each image is uniformly adjusted to 640x640 pixels to meet the input size requirements of YOLOv5. At the same time, for special scenes in inspections, common data enhancement techniques are used to expand the training set, including random cropping, rotation, translation, color perturbation, noise addition and other operations to enhance the model's adaptability to complex scenes.

[0093] At the same time, a series of innovative data enhancement technologies are used to expand the training set and enhance the model's adaptability to complex scenarios. In addition, by simulating substation scenes under different weather conditions (such as haze, rain, strong light, etc.), defect images under various climatic conditions are generated to improve the model's adaptability to light and weather changes.

[0094] Through the above innovative enhancement strategies, the diversity and complexity of the data set have been significantly improved, effectively improving the robustness and detection accuracy of the YOLOv5 model in complex inspection scenarios.

[0095] S2. Based on the K-means++ clustering algorithm, combined with the hierarchical clustering method and Gaussian mixture model, the aspect ratio clustering calculation of the target candidate box is performed on the substation inspection image data of the target detection dataset to obtain the optimal anchor box parameters;

[0096] It should be noted that in order to optimize the efficiency and accuracy of prediction box generation, the K-means++ clustering algorithm is used, combined with other optimization algorithms to calculate the aspect ratio of the GT box (Ground Truth box) to generate anchor boxes suitable for substation defect detection tasks. These anchor boxes will be used to initialize the candidate box generation process in the YOLOv5 model to ensure that each candidate box can better match the actual target size. In order to further improve the clustering effect, hierarchical clustering (HierarchicalClustering) and Gaussian mixture model (Gaussian Mixture Model, GMM) are combined for multiple clustering calculations. This combination effectively makes up for the limitations of K-means++ in dealing with complex shape distributions, and can more comprehensively characterize the aspect ratio distribution in the data set.

[0097] First, K-means++ is used to perform preliminary centroid selection to obtain a more dispersed and reasonable initial clustering result. Specifically, K-means++ gives priority to points with a large distance from the selected centroid during the centroid selection process to improve clustering accuracy and accelerate convergence.

[0098] Then, hierarchical clustering is used to perform hierarchical analysis on the initial centroids, further subdivide the subclass clustering relationship in the data set, and improve the distribution density of the centroids. The theoretical basis of hierarchical clustering is to recursively construct the hierarchical structure of the data, and analyze the similarity relationship between data points or clusters through bottom-up step-by-step merging (agglomerative clustering) or top-down divisive clustering. For the initial centroids, similar centroid points can be gradually merged into clusters through a bottom-up method to construct clustering structures of different levels, and then the potential subclasses in the data set can be discovered. The distance metric used in the calculation is usually the Euclidean distance, and the clustering stop condition is to reach the set number of layers or the average distance of the merged clusters reaches the specified threshold.

[0099] After obtaining the clustering results after hierarchical analysis, the Gaussian mixture model (GMM) is used to perform probability analysis on each clustering result to ensure that the target aspect ratio distribution within each category is more consistent with the multidimensional normal distribution, thereby further improving the stability and accuracy of clustering. The theoretical basis of the Gaussian mixture model is to regard each cluster as a combination of multiple Gaussian distributions, and to estimate the parameters through the maximum expectation algorithm (EM algorithm) to maximize the probability that a given data point belongs to a cluster.

[0100] As a preferred implementation, based on the K-means++ clustering algorithm, combined with the hierarchical clustering method and the Gaussian mixture model, the target candidate box aspect ratio clustering calculation is performed on the substation inspection image data of the target detection data set, and the optimal anchor box parameters are obtained, including the following steps:

[0101] S2 1. Extract the aspect ratio of the annotation box of the substation inspection image data according to the training set expanded and processed in the target detection data set, and establish an aspect ratio data set according to the aspect ratio of the annotation box of the substation inspection image data;

[0102] S22, clustering the aspect ratio data set by using the K-means++ clustering algorithm to obtain several initial cluster centers and obtain preliminary clustering results;

[0103] As a preferred implementation, the aspect ratio data set is clustered using the K-means++ clustering algorithm to obtain a number of initial cluster centers. Obtaining a preliminary clustering result includes the following steps:

[0104] S221, randomly selecting a data point from the aspect ratio data set as an initial centroid, and calculating the shortest distance between the remaining data points and the initial centroid;

[0105] S222, selecting a data point as a new centroid from the remaining data points by probabilistic selection according to the shortest distance between each data point and the initial centroid;

[0106] S223. Recalculate the distances between the remaining data points and the new centroids, and repeat step S222 until a preset number of centroids are selected to obtain a preliminary clustering result.

[0107] It should be noted that the difference between K-means++ and traditional K-means is that it selects the initial centroid in an optimized way.

[0108] First, randomly select a data point as the first centroid c1 and calculate the distance from the remaining data points to the existing centroid: for each remaining data point x i (i=2, 3, ..., n), calculate its minimum distance D(x) to the selected centroid C={c1} i )=min{d(x i , c1)}, where d(x i , c1) represents the data point x i The Euclidean distance from the centroid c1.

[0109] The next centroid is selected probabilistically, and the remaining data points are selected according to the weighted distribution of the distance from the current centroid set. Specifically, the probability of selecting each data point as the centroid is proportional to the square of its distance to the nearest centroid, that is, the selection probability is:

[0110]

[0111] In the formula, P(x i ) represents the probability of selecting the jth data point as the centroid from the remaining data points; D(x j ) 2 represents the jth data point x j The square of the Euclidean distance from the centroid, Represents data point x j The distance to the nearest centroid and n represents the number of remaining data points.

[0112] Based on the above distance weighted selection process, continue to select the remaining centroids from the data set until a predetermined number of centroids (i.e., the number of anchor boxes, such as 9 centroids) are selected. All data points (i.e., the aspect ratio of the GT box) will be assigned to the centroid closest to them, and based on all the data points assigned to each centroid, the average of these points is calculated, and the position of the centroid is updated. This process is repeated until the position of the centroid no longer changes significantly, indicating that the clustering process converges.

[0113] The aspect ratio of the resulting cluster center (i.e., centroid) is used to generate anchor boxes. These anchor boxes represent the size of typical objects in the dataset. Finally, K-means++ clustering generates 9 anchor box sizes. These anchor boxes cover defect objects of different types and sizes in the dataset, from small to large, and can effectively match the actual size of the target object.

[0114] By using K-means++ clustering to generate anchor boxes, the YOLOv5 model can more accurately fit defect targets of different sizes. Compared with the method of randomly initializing anchor boxes, the anchor boxes optimized based on K-means++ can reduce the deviation in the model training process and improve the detection accuracy and efficiency of the model.

[0115] S23. Optimize the preliminary clustering results obtained by using a hierarchical clustering method, and fit the optimization results using a Gaussian mixture model to generate optimal anchor frame parameters.

[0116] As a preferred implementation, the hierarchical clustering method is used to optimize the obtained preliminary clustering results, and the Gaussian mixture model is used to fit the optimization results. The generation of the optimal anchor frame parameters includes the following steps:

[0117] S23 1. A bottom-up agglomerative hierarchical clustering method is used, and the centroids in the preliminary clustering results are used as the initial clusters;

[0118] S232, calculating the similarity between the centroids by using the Euclidean distance calculation method, and according to the similarity between the centroids, gradually merging the centroids into clusters through a bottom-up method, generating clustering structures at different levels, and obtaining a hierarchical clustering result;

[0119] S233. Perform probability analysis on the hierarchical clustering results through a Gaussian mixture model, and based on the probability analysis results, extract the mean of each Gaussian distribution as the aspect ratio parameter of the optimal anchor box through a maximum expectation algorithm.

[0120] As a preferred implementation, a probability analysis is performed on the hierarchical clustering results by using a Gaussian mixture model, and based on the probability analysis results, the mean of each Gaussian distribution is extracted by a maximum expectation algorithm as the aspect ratio parameter of the optimal anchor box, including the following steps:

[0121] S233 1. Based on the hierarchical clustering results, initialize Gaussian distribution parameters for each hierarchical clustering result, where the Gaussian distribution parameters include mean (μ), covariance matrix (∑) and mixing coefficient (π);

[0122] S2332. For each data point in the hierarchical clustering result, calculate the probability that it belongs to each Gaussian distribution;

[0123] As a preferred implementation, for each data point in the hierarchical clustering result, the calculation formula for calculating the probability that it belongs to each Gaussian distribution is:

[0124]

[0125] In the formula, P(z k |x i ) represents the data point x in the hierarchical clustering result i Belongs to the kth Gaussian distribution probability; N(x i |u k ,C k ) represents the data point x in the hierarchical clustering result i Belongs to the kth Gaussian distribution probability density; N(x i |u j ,C j ) represents the data point x in the hierarchical clustering result i belongs to the jth Gaussian distribution probability density; K represents the total number of Gaussian distributions; π k The formula for the mixing coefficient of the kth Gaussian distribution; j represents the number of Gaussian distributions; μ j represents the mean of the jth Gaussian distribution; μ k represents the mean of the kth Gaussian distribution; C represents the covariance matrix.

[0126] S2333. According to the probability that each data point in the hierarchical clustering result belongs to each Gaussian distribution, update the Gaussian distribution parameters of each hierarchical clustering result to maximize the probability that each data point in the hierarchical clustering result belongs to each Gaussian distribution;

[0127] S2334. Repeat steps S2332-S2333 until the preset number of iterations is reached to obtain the Gaussian distribution model of each hierarchical clustering result, and extract the mean of each Gaussian distribution as the aspect ratio parameter of the optimal anchor box according to the Gaussian distribution model of each hierarchical clustering result.

[0128] S3. Based on the optimal anchor frame parameters, the improved YOLOv5 neural network model is trained through the target detection data set to generate training weights and output prediction results; the prediction results are post-processed by softening non-maximum suppression, and the substation target detection results are output based on the post-processing results.

[0129] It should be noted that the improved YOLOv5 model adopts a multi-level feature detection architecture to extract information from feature maps of different resolutions. The feature map sizes are 19×19, 38×38, 76×76, 104×104, and 208×208. Compared with the original YOLOv5 model, the feature map outputs of 104×104 and 208×208 are added, which improves the detection capability of small-sized targets. Nine anchor boxes are allocated to the feature maps of each scale for target detection, ensuring effective detection of substation defects of different sizes.

[0130] In addition, in order to improve the robustness and detection speed of the model, the model introduces depth-separable convolution on each layer of feature maps to reduce the amount of calculation while retaining the accuracy of feature extraction. Residual connections and cross-layer connections are used to strengthen the fusion of feature maps at different layers, enabling the model to better capture the multi-scale information of the target.

[0131] As a preferred implementation, based on the optimal anchor frame parameters, the improved YOLOv5 neural network model is trained through the target detection data set to generate training weights and output prediction results; the prediction results are post-processed by softening non-maximum suppression, and the substation target detection results are output according to the post-processing results, including the following steps:

[0132] S3 1. Configure the training parameters of the improved YOLOv5 neural network model, and use the YOLOv5 regression loss function as the loss value between each predicted box and the true box;

[0133] S32, training the improved YOLOv5 neural network model using the extended training set in the target detection data set; saving the model weights during the training process, using the validation set in the target detection data set, and performing performance testing on the improved YOLOv5 neural network model, and dynamically adjusting the learning rate of the improved YOLOv5 neural network model according to the performance test results;

[0134] It should be noted that during the training process, the following training parameters were set: batch size was set to 32; learning rate was set to 0.001; epoch was set to 1000 times;

[0135] S33, after the training of the improved YOLOv5 neural network model is completed, the model weights are loaded into the improved YOLOv5 model, and the test set in the target detection data set is predicted by the improved YOLOv5 model to obtain a prediction result;

[0136] S34. According to the obtained prediction results, the candidate boxes are filtered by softening non-maximum suppression to obtain the substation target detection results, and the substation target detection results are visualized.

[0137] As a preferred implementation, the improved YOLOv5 neural network model adopts a multi-level feature detection architecture, and the feature map output sizes of the improved YOLOv5 neural network model include: 19×19, 38×38, 76×76, 104×104 and 208×208.

[0138] As a preferred implementation, the regression loss function of YOLOv5 includes: bounding box regression loss, confidence loss and classification loss;

[0139] Bounding box regression loss, which is used to calculate the positioning difference between each predicted box and the true box, using full intersection-over-union loss;

[0140] Confidence loss, used to measure whether the predicted box contains the target, using binary cross entropy loss;

[0141] Classification loss, predicting the category of the target, also uses binary cross entropy loss;

[0142] The regression loss function expression of YOLOv5 is:

[0143] L=λbox*Lbox+λconf*Lconf+λcls*Lcls;

[0144] In the formula, L represents the regression loss function of YOLOv5; Lbox represents the bounding box regression loss; Lconf represents the confidence loss; Lcls represents the classification loss; λbox represents the weight parameter of the bounding box regression loss; λconf represents the weight parameter of the confidence loss; λcls represents the weight parameter of the classification loss.

[0145] Specifically, the bounding box regression loss is used to calculate the positioning difference between each predicted box and the true box, using the CIoU (Complete Intersection over Union) loss function. CIoU is an improved IoU algorithm that takes into account the overlapping area, center point distance, and aspect ratio difference between target boxes to make it more accurate. The calculation formula of the CIoU loss function is as follows:

[0146] CIoU=1-(IoU-ρ2(b,bg) / c2-α*v);

[0147] Where IoU is the intersection over union of the predicted box and the true box, ρ2(b, bg) is the Euclidean distance between the center points of the predicted box and the true box, c is the diagonal length of the minimum circumscribed rectangle, v is a parameter used to measure the difference between the aspect ratios of the predicted box and the true box, and α is a weight factor used to balance the effects of IoU and aspect ratio differences.

[0148] Through the calculation of the above loss function, the model can continuously adjust the position and size of the predicted box to make it closer to the real box and improve the accuracy of target detection. During the training process, the model updates the weights in each batch, and each image in the training data set is used for training multiple times. The model weights are saved after each iteration, and the model performance is tested on the validation set. The learning rate is dynamically adjusted based on the detection results of the validation set to ensure model convergence. The training process is performed on an NVIDIA GeForceRTX 2080 graphics card, and the entire training process takes about 5 hours.

[0149] As a preferred implementation, after the training of the improved YOLOv5 neural network model is completed, the model weights are loaded into the improved YOLOv5 model, and the test set in the target detection data set is predicted by the improved YOLOv5 model, and the prediction result is obtained by the following steps:

[0150] S33 1. After the model weights are loaded into the improved YOLOv5 model, the test set in the target detection dataset is input into the improved YOLOv5 model to obtain the predicted bounding box, confidence and category probability of each target;

[0151] It should be noted that after the training is completed, the saved model weight file is loaded into the improved YOLOv5 model, and the test image is input for prediction. The network outputs the predicted bounding box, confidence, and category probability of each target. The confidence indicates the model's confidence in whether the predicted box contains the target, and the category probability is normalized by the Sigmoid activation function, indicating the probability that the target belongs to a certain category. In order to make a preliminary judgment on the defects of each target, the confidence threshold and the category probability threshold are first set to screen the valid prediction boxes. Then, the confidence and category probability are combined to calculate the comprehensive score, which is used to comprehensively judge the severity of the defects of the target. The judgment basis is: when the comprehensive score is higher than a preset threshold, the model determines that the target may have defects. By combining confidence and category probability, the target defects can be judged more accurately, avoiding misjudgment, thereby improving the prediction accuracy of the model.

[0152] The calculation formula for the comprehensive score is:

[0153] Sfinal=Confidence×Class Probability;

[0154] In the formula, Sfinal represents the comprehensive score, Confidence represents the confidence, and Class Probability represents the class probability.

[0155] S332, based on a preset confidence threshold and category probability threshold, and in combination with the confidence and category probability of each target, calculating a comprehensive score of the predicted bounding box;

[0156] S333. According to the calculated comprehensive score of the predicted bounding box, the predicted bounding box is screened, and the predicted bounding box with a comprehensive score higher than the set score threshold is taken as the prediction result.

[0157] As a preferred implementation, according to the obtained prediction results, the candidate box is filtered by softening non-maximum suppression to obtain the substation target detection result, and the substation target detection result is visualized, including the following steps:

[0158] S341, sorting the obtained prediction results in descending order according to the confidence of the prediction bounding box, and for each predicted bounding box after sorting, calculating the intersection-and-union ratio between it and all other predicted bounding boxes;

[0159] S342, reducing the confidence of the predicted bounding box using a confidence reduction calculation formula according to the intersection-and-union ratio between each predicted bounding box and other predicted bounding boxes;

[0160] S343, comparing the reduced confidence of the predicted bounding box with a preset confidence threshold, and screening out the predicted bounding box that meets the preset confidence threshold;

[0161] S344: The predicted bounding box is selected as the final substation target detection result, and the final substation target detection result is visualized using visualization technology.

[0162] It should be noted that there may be multiple overlapping boxes in the generated prediction bounding box. In order to avoid repeated detection of the same defect target, the prediction bounding box is screened by using the softened non-maximum suppression (Soft-NMS) strategy. Unlike traditional non-maximum suppression (NMS), Soft-NMS does not directly remove boxes with large overlaps, but gradually reduces the confidence of overlapping boxes to retain more reasonable detection results. Its core theoretical basis is: in the process of adjusting the confidence of overlapping boxes, adjustments are made based on the IoU (Intersection over Union) between boxes to reduce the confidence of predicted boxes with large overlaps, rather than simply deleting overlapping boxes.

[0163] As a preferred implementation, the confidence reduction (Soft-NMS) calculation formula is:

[0164] score i ′=score i ×(1-IoU(b i , b j ));

[0165] In the formula, score iIndicates the confidence of the i-th predicted bounding box after reduction; score i represents the confidence of the i-th predicted bounding box; IoU(b i ,b j ) represents the intersection-over-union ratio (i.e., the degree of overlap) between the i-th predicted bounding box and the j-th predicted bounding box. As the intersection-over-union ratio increases, the confidence of the overlapping boxes will gradually decrease, ensuring that the boxes with higher confidence are retained first.

[0166] It should be noted that the evaluation index used by the algorithm of the present invention is the detection accuracy (AP) index for evaluation, which reflects the performance of the algorithm detection.

[0167] The indicators of each category of detection results are precision, which is the percentage of correctly detected equipment defects in the total detection area; recall, which is the percentage of correctly detected equipment defects in the total equipment defects. A curve is drawn based on the precision and recall of each category of detection results. The area under the curve is the average detection accuracy. The calculation formulas for precision, recall and average precision are as follows:

[0168]

[0169] In the formula, Precision represents precision, Recall represents recall, AP represents detection accuracy, TP represents the number of correctly detected device defects, FP represents the number of device defects detected as background, and FN represents the number of device defects detected as background. p represents the precision on the vertical axis, and r represents the recall on the horizontal axis.

[0170] In addition, in order to verify the superiority of the proposed method, the detection performance of the improved YOLOv5 detection network in this paper is compared with that of the original YOLOv5 detection algorithm. The test data set used is the substation equipment defect test set. The results are shown in Table 1.

[0171] Table 1 Detection results of the improved YOLOv5 detection network and the original YOLOv5 detection algorithm

[0172] category AP(YOLOv5) AP (Improved YOLOv5) Platen open 0.78 0.82 Press plate 0.82 0.87 Hanging suspended objects 0.7 0.76 Bird's Nest 0.68 0.72 Insulator rupture 0.85 0.88 Damaged dial casing 0.84 0.89 Blurred dial 0.76 O.8 The door is closed abnormally 0.83 0.87 Respirator silicone discoloration 0.81 0.84 Cover damaged 0.75 0.82 Not wearing work clothes 0.76 0.82

[0173] It can be seen from Table 1 that the improved YOLOv5 detection algorithm of the present invention greatly improves the detection accuracy of various categories.

[0174] In summary, with the help of the above technical solutions of the present invention, the present invention significantly improves the detection ability of small objects and subtle defects by using the improved YOLOv5 model and feature fusion strategy. In the 11 common defect detection tasks in substations, the mAP is significantly improved, achieving a high detection accuracy, with good real-time performance, and is suitable for substation automation inspection systems. In substation defect detection tasks, a high precision measurement accuracy is obtained, which can meet the actual needs of substation safety inspections. The present invention uses K-means++ to perform preliminary centroid selection, which can obtain more dispersed and reasonable initial clustering results, and K-means++ gives priority to points with a large distance from the selected centroid in the centroid selection process to improve clustering accuracy and accelerate convergence.

[0175] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, optical storage, etc.) containing computer-usable program code.

[0176] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. YOLOv5 substation target detection method based on multi-scale feature fusion, characterized in that: The method comprises the following steps: S1. Collect substation inspection image data, annotate images in the substation inspection image data using defect labels according to the defect types of the substation, and construct a target detection dataset based on the annotated substation image data; S2. Based on the K-means++ clustering algorithm, combined with the hierarchical clustering method and Gaussian mixture model, the aspect ratio clustering calculation of the target candidate box is performed on the substation inspection image data of the target detection dataset to obtain the optimal anchor box parameters; S3. Based on the optimal anchor frame parameters, the improved YOLOv5 neural network model is trained through the target detection data set to generate training weights and output prediction results; the prediction results are post-processed by softening non-maximum suppression, and the substation target detection results are output based on the post-processing results.

2. The YOLOv5 substation target detection method based on multi-scale feature fusion according to claim 1 is characterized in that: The collecting of substation inspection image data, labeling images in the substation inspection image data by defect labels according to the defect type of the substation, and constructing a target detection data set according to the labeled substation image data includes the following steps: S11, collecting substation inspection image data under different weather conditions, photo conditions and time periods, and annotating defects on the substation inspection image data according to the defect type of the substation, to obtain a data set of substation inspection scenes; S12, formatting and resizing the image data in the data set of the substation inspection scene, and dividing the resized data set of the substation inspection scene into a training set, a test set, and a verification set; S13. Expand the image data in the training set through data enhancement technology, and combine the expanded training set, test set and validation set to obtain a target detection data set.

3. The YOLOv5 substation target detection method based on multi-scale feature fusion according to claim 2 is characterized in that: The types of defects in the substation include: open pressure plates, closed pressure plates, hanging suspended objects, bird nests, broken insulators, damaged dial casings, blurred dials, abnormal door closure, discoloration of respirator silicone, damaged covers and not wearing work clothes.

4. The YOLOv5 substation target detection method based on multi-scale feature fusion according to claim 2 is characterized in that: The data enhancement techniques include: random cropping, rotation, translation, color perturbation and noise addition.

5. The YOLOv5 substation target detection method based on multi-scale feature fusion according to claim 1 is characterized in that: The method based on the K-means++ clustering algorithm, combined with the hierarchical clustering method and the Gaussian mixture model, performs the target candidate frame aspect ratio clustering calculation on the substation inspection image data of the target detection data set, and obtains the optimal anchor frame parameters, including the following steps: S21, extracting the aspect ratio of the annotation box of the substation inspection image data according to the training set expanded and processed in the target detection data set, and establishing an aspect ratio data set according to the aspect ratio of the annotation box of the substation inspection image data; S22, clustering the aspect ratio data set by using the K-means++ clustering algorithm to obtain several initial cluster centers and obtain preliminary clustering results; S23. Optimize the preliminary clustering results obtained by using a hierarchical clustering method, and fit the optimization results using a Gaussian mixture model to generate optimal anchor frame parameters.

6. The YOLOv5 substation target detection method based on multi-scale feature fusion according to claim 5 is characterized in that: The method of clustering the aspect ratio data set using the K-means++ clustering algorithm to obtain a number of initial cluster centers and obtaining a preliminary clustering result includes the following steps: S221, randomly selecting a data point from the aspect ratio data set as an initial centroid, and calculating the shortest distance between the remaining data points and the initial centroid; S222, selecting a data point as a new centroid from the remaining data points by probabilistic selection according to the shortest distance between each data point and the initial centroid; S223. Recalculate the distances between the remaining data points and the new centroids, and repeat step S222 until a preset number of centroids are selected to obtain a preliminary clustering result.

7. The YOLOv5 substation target detection method based on multi-scale feature fusion according to claim 5 is characterized in that: The method of optimizing the obtained preliminary clustering results by using the hierarchical clustering method and fitting the optimized results by using the Gaussian mixture model to generate the optimal anchor frame parameters includes the following steps: S231, adopting a bottom-up agglomerative hierarchical clustering method, and taking the centroid of the preliminary clustering results as the initial cluster; S232, calculating the similarity between the centroids by using the Euclidean distance calculation method, and according to the similarity between the centroids, gradually merging the centroids into clusters through a bottom-up method, generating clustering structures at different levels, and obtaining a hierarchical clustering result; S233. Perform probability analysis on the hierarchical clustering results through a Gaussian mixture model, and based on the probability analysis results, extract the mean of each Gaussian distribution as the aspect ratio parameter of the optimal anchor box through a maximum expectation algorithm.

8. The YOLOv5 substation target detection method based on multi-scale feature fusion according to claim 7 is characterized in that: The method of performing probability analysis on the hierarchical clustering results by using a Gaussian mixture model and extracting the mean of each Gaussian distribution as the aspect ratio parameter of the optimal anchor box by using a maximum expectation algorithm according to the probability analysis results includes the following steps: S2331. Based on the hierarchical clustering result, initialize Gaussian distribution parameters for each hierarchical clustering result, wherein the Gaussian distribution parameters include mean, covariance matrix and mixing coefficient; S2332. For each data point in the hierarchical clustering result, calculate the probability that it belongs to each Gaussian distribution; S2333. According to the probability that each data point in the hierarchical clustering result belongs to each Gaussian distribution, update the Gaussian distribution parameters of each hierarchical clustering result to maximize the probability that each data point in the hierarchical clustering result belongs to each Gaussian distribution; S2334. Repeat steps S2332-S2333 until the preset number of iterations is reached to obtain the Gaussian distribution model of each hierarchical clustering result, and extract the mean of each Gaussian distribution as the aspect ratio parameter of the optimal anchor box according to the Gaussian distribution model of each hierarchical clustering result.

9. The YOLOv5 substation target detection method based on multi-scale feature fusion according to claim 8 is characterized in that: The calculation formula for calculating the probability of each data point belonging to each Gaussian distribution in the hierarchical clustering result is: In the formula, P(z k |x i ) represents the data point x in the hierarchical clustering result i The probability of belonging to the kth Gaussian distribution; N(x i |u k , C k ) represents the data point x in the hierarchical clustering result i Belongs to the kth Gaussian distribution probability density; N(x i |u j ,C j ) represents the data point x in the hierarchical clustering result i Belongs to the jth Gaussian distribution probability density; K represents the total number of Gaussian distributions; π k The formula for the mixing coefficient of the kth Gaussian distribution; j represents the number of Gaussian distributions; μ j represents the mean of the ,th Gaussian distribution; μ k represents the mean of the kth Gaussian distribution; C represents the covariance matrix.

10. The YOLOv5 substation target detection method based on multi-scale feature fusion according to claim 1 is characterized in that: Based on the optimal anchor frame parameters, the improved YOLOv5 neural network model is trained through the target detection data set to generate training weights and output prediction results; Post-processing the prediction results by softening non-maximum suppression and outputting the substation target detection results according to the post-processing results includes the following steps: S31, configure the training parameters of the improved YOLOv5 neural network model, and use the regression loss function of YOLOv5 as the loss value between each predicted box and the true box; S32, training the improved YOLOv5 neural network model using the extended training set in the target detection data set; saving the model weights during the training process, using the validation set in the target detection data set, and performing performance testing on the improved YOLOv5 neural network model, and dynamically adjusting the learning rate of the improved YOLOv5 neural network model according to the performance test results; S33, after the training of the improved YOLOv5 neural network model is completed, the model weights are loaded into the improved YOLOv5 model, and the test set in the target detection data set is predicted by the improved YOLOv5 model to obtain a prediction result; S34. According to the obtained prediction results, the candidate boxes are filtered by softening non-maximum suppression to obtain the substation target detection results, and the substation target detection results are visualized.

11. The YOLOv5 substation target detection method based on multi-scale feature fusion according to claim 10 is characterized in that: The improved YOLOv5 neural network model adopts a multi-level feature detection architecture, and the feature map output sizes of the improved YOLOv5 neural network model include: 19×19, 38×38, 76×76, 104×104 and 208×208.

12. The YOLOv5 substation target detection method based on multi-scale feature fusion according to claim 11 is characterized in that: The regression loss function of YOLOv5 includes: bounding box regression loss, confidence loss and classification loss; The bounding box regression loss is used to calculate the positioning difference between each predicted box and the real box, using the complete intersection-over-union loss; The confidence loss is used to measure whether the predicted box contains the target, and adopts binary cross entropy loss; The classification loss, predicting the category of the target, also uses binary cross entropy loss; The regression loss function expression of YOLOv5 is: L=λbox*Lbox+λconf*Lconf+λcls*Lcls; Where L represents the regression loss function of YOLOv5; Lbox represents the bounding box regression loss; Lconf represents the confidence loss; Lcls represents classification loss; λbox represents the weight parameter of the bounding box regression loss; λconf represents the weight parameter of confidence loss; λcls represents the weight parameter of classification loss.

13. The YOLOv5 substation target detection method based on multi-scale feature fusion according to claim 10 is characterized in that: After the training of the improved YOLOv5 neural network model is completed, the model weights are loaded into the improved YOLOv5 model, and the test set in the target detection data set is predicted by the improved YOLOv5 model to obtain the prediction result, which includes the following steps: S331, after the model weights are loaded into the improved YOLOv5 model, the test set in the target detection dataset is input into the improved YOLOv5 model to obtain the predicted bounding box, confidence and category probability of each target; S332, based on a preset confidence threshold and category probability threshold, and in combination with the confidence and category probability of each target, calculating a comprehensive score of the predicted bounding box; S333. According to the calculated comprehensive score of the predicted bounding box, the predicted bounding box is screened, and the predicted bounding box with a comprehensive score higher than the set score threshold is taken as the prediction result.

14. The YOLOv5 substation target detection method based on multi-scale feature fusion according to claim 10 is characterized in that: According to the obtained prediction results, the candidate box is filtered by softening non-maximum suppression to obtain the substation target detection result, and the substation target detection result is visualized, which includes the following steps: S341, sorting the obtained prediction results in descending order according to the confidence of the prediction bounding box, and for each predicted bounding box after sorting, calculating the intersection-and-union ratio between it and all other predicted bounding boxes; S342, reducing the confidence of the predicted bounding box using a confidence reduction calculation formula according to the intersection-and-union ratio between each predicted bounding box and other predicted bounding boxes; S343, comparing the reduced confidence of the predicted bounding box with a preset confidence threshold, and screening out the predicted bounding box that meets the preset confidence threshold; S344: The predicted bounding box is selected as the final substation target detection result, and the final substation target detection result is visualized using visualization technology.

15. The YOLOv5 substation target detection method based on multi-scale feature fusion according to claim 14 is characterized in that: The confidence reduction calculation formula is: score i ′=score i ×(1-IoU(b i ,b j )); In the formula, score i ′ represents the reduced confidence of the i-th predicted bounding box; score i represents the confidence of the i-th predicted bounding box; IoU(b i , b j ) represents the intersection-over-union ratio between the i-th predicted bounding box and the j-th predicted bounding box.

Images