A two-stage visual defect detection method for substation equipment based on open set classification

Abstract

The present invention discloses a two-stage substation equipment visual defect detection method based on open set classification, by constructing a data set; constructing an improved YOLOv8n network model; training the improved YOLOv8n network model; constructing an open set classification network with Swin Transformer as the backbone network; training the open set classification network; and testing the effect of the proposed two-stage target detection algorithm. The two-stage substation equipment visual defect detection method based on open set classification provided by the present invention proposes a novel two-stage target detection framework for the diverse target categories, complex environment and a large amount of visual interference in substation defect detection, adopts a one-stage target detection algorithm combined with an open set classification algorithm, achieves a stronger anti-interference ability, and shows excellent performance in substation defect detection.

Description

Technical Field

[0001] The invention relates to the technical field of target detection, and in particular discloses a two-stage substation equipment visual defect detection method based on open set classification. Background Art

[0002] In modern power systems, substations are key power dispatching and distribution centers, and the safety and reliability of their equipment are crucial. Equipment defects may lead to power supply interruptions, equipment damage, and even safety accidents. Therefore, it is particularly important to detect substation equipment defects in a timely and accurate manner. However, traditional defect detection methods often rely on expert experience and manual inspection, which are inefficient and easily affected by subjective factors. In recent years, with the development of deep learning and computer vision technology, image-based defect detection methods have gradually attracted attention. However, existing detection models usually assume that training data and test data are from the same distribution. However, there are usually a large amount of unknown data in actual detection tasks, which brings huge challenges to actual defect detection tasks. Therefore, it is particularly important to study substation equipment defect detection methods based on open set classification. Open set classification allows the model to make reasonable judgments in the face of complex environments, thereby improving the robustness and accuracy of defect detection. The present invention aims to propose a two-stage defect detection method, combined with open set classification technology, to optimize the monitoring and maintenance strategy of substation equipment and improve the overall safety and reliability of the power system.

[0003] At present, the defect inspection methods of power equipment are mainly divided into traditional inspection and intelligent equipment inspection. The manual inspection method is mainly carried out by engineering and technical personnel to visually inspect, measure, test and record the power equipment regularly to find potential faults and defects. For example, check whether there are obvious faults or rust on the appearance of the power equipment, measure the temperature, current and voltage of the equipment, and perform insulation tests and grounding tests. At the same time, preventive maintenance and overhaul will be carried out according to the use of the equipment and historical data. However, there are several limitations of traditional manual inspection, including the following aspects: First, manual inspection requires a lot of time and manpower costs, especially for large power equipment and equipment-intensive places; second, manual inspection may not be able to detect minor faults and potential problems in time, resulting in missed faults; in addition, manual inspection also has problems of subjective judgment and experience dependence, which may affect the accuracy and reliability of detection. These problems may lead to undesirable consequences such as missed equipment faults, increased safety risks, economic losses and reduced efficiency.

[0004] In recent years, the use of convolutional neural networks (CNNs) for substation defect detection has become the main research direction. Qi et al. improved the Faster R-CNN target recognition network in the paper "Research on Substation Equipment Defect Detection Algorithm Based on Improved Faster R-CNN" to detect common substation defects. Although the network achieved certain results, the training model of the network was large and required strong computing equipment. Chen et al. improved the YOLOv4 algorithm in the paper "Substation Defect Detection Based on Improved YOLOv4" to achieve substation defect detection, which has certain advantages over the two-stage target detection model Faster R-CNN. Li et al. proposed a substation multi-target detection based on attention mechanism and feature balance in "Substation Multi-target Detection Based on Attention Mechanism and Feature Balance", which effectively realized the multi-classification target detection of substations, but did not effectively consider the lightweight problem while realizing the multi-classification detection of defect targets.

[0005] Although the above algorithms have achieved certain effects, they are only oriented to single scenes, are easily disturbed by image noise, have poor robustness, and are difficult to adapt to environments with strong interference. Summary of the invention

[0006] The present invention provides a two-stage substation equipment visual defect detection method based on open set classification, aiming to solve at least one defect existing in the above-mentioned prior art.

[0007] The present invention relates to a two-stage substation equipment visual defect detection method based on open set classification, comprising the following steps:

[0008] Constructing data sets: constructing secondary category target detection data sets, primary category target detection data sets and open set classification data sets of power equipment with defects. The secondary category target detection data sets are used to label the secondary categories of substation defect image data, the primary category target detection data sets are used to label the primary categories of substation defect image data, and the open set classification data sets are used to perform open set classification of substation defect image data;

[0009] Build an improved YOLOv8n network model: Build the ELAN module, YOLOv8n backbone network, YOLOv8n neck network, YOLOv8n detection head and YOLOV8n improved loss function in the improved YOLOv8n network model;

[0010] Training the improved YOLOv8n network model: Load the first-level target detection dataset, and send the first-level target detection dataset to the improved YOLOv8n network model for model training after data preprocessing; Based on the improved loss function of YOLOV8n, follow the principle of forward construction loss to calculate the total loss during the model training of the first-level target detection dataset, and back-propagate to update the network parameters;

[0011] Construct an open-set classification network with Swin Transformer as the backbone network: Based on the Swin Transformer block, construct the backbone network, category output layer, loss function and open-set classification post-processing algorithm of the open-set classification network;

[0012] Training the open set classification network: Load the open set classification dataset, preprocess the open set classification dataset and send it to the open set classification network for model training;

[0013] The effect of the proposed two-stage target detection algorithm is tested: based on the improved YOLOv8n network model after loading and training and the weight model of the open set classification network, the images to be tested in the substation equipment defect test set are input into the target detection network, and the substation equipment defect test set is predicted to obtain the bounding boxes and category confidence of the images on the substation equipment defect test set.

[0014] Furthermore, the steps of constructing the dataset include:

[0015] Construct a secondary category target detection dataset: photograph defective power equipment to obtain substation defect image data; annotate the substation defect image data with secondary category defects;

[0016] Constructing a first-level category target detection data set: Based on the second-level category target detection data set and the preset correspondence between the second-level category and the first-level category, the second-level categories are merged to obtain the first-level category target detection data set;

[0017] Construct an open-set classification dataset: Based on the secondary category target detection dataset and the preset secondary category segmentation, use the label data of the training set in the secondary category target detection dataset to capture images of the corresponding area and obtain an open-set classification dataset.

[0018] Furthermore, the steps to build an improved YOLOv8n network model include:

[0019] Construct the ELAN module: replace the convolution of the Bottleneck module with a grouped convolution with a group number of 3, and add a channel reorganization operation between adjacent grouped convolution layers;

[0020] Build the YOLOv8n backbone network: use two 3×3 convolutional layers with a stride of 2 to downsample the feature map of the input image by four times, pass it through four ELAN modules in sequence, and set a 3×3 convolutional layer with a stride of 2 between every two ELAN modules;

[0021] Construct the YOLOv8n neck network: three feature layers are derived from the backbone network as the input of the neck network, among which the large-scale feature layer comes from the output of the second ELAN module, the medium-scale feature layer comes from the output of the third ELAN module, and the small-scale feature layer comes from the output of the SPPF module; after upsampling the small-scale feature layer, it is spliced ​​with the medium-scale feature layer through the ELAN module to obtain a new medium-scale feature layer; after transforming the medium-scale feature layer, it is merged with the large-scale feature layer; the large-scale feature layer is directly output to the head network, and at the same time, after downsampling the large-scale feature layer, it is merged with the medium-scale feature layer through the ELAN module to obtain a new medium-scale feature layer, which is then output to the head network; the medium-scale feature layer is fused with the small-scale feature layer, and then the small-scale feature layer is output to the head network; the neck network outputs three fused feature layers of large, medium and small;

[0022] Build YOLOv8n detection head: The YOLOV8 network contains three scales of detection heads, each of which adopts a design that decouples positioning prediction and classification prediction;

[0023] Construct an improved loss function: Find the appropriate weight coefficient by repeatedly adjusting the proportion of YOLOV8 network positioning box regression loss, target classification loss and DF Loss in the total loss.

[0024] Furthermore, the steps of training the improved YOLOv8n network model include:

[0025] Pre-training preparation and training configuration: Load the first-level target detection dataset, and send it to the improved YOLOv8n network model for model training after data preprocessing. The data preprocessing includes normalization, image scale transformation, random cropping, rotation and color transformation;

[0026] Network parameter update and loss calculation: Based on the improved loss function of YOLOV8n, the total loss is calculated during the model training of the first-level category target detection dataset in accordance with the principle of forward construction loss, and the network parameters are updated through back propagation. In the model training process, the model parameter averaging strategy is adopted to automatically average the model parameters of three adjacent rounds, making the model training smoother.

[0027] Furthermore, the steps of constructing an open set classification network with Swin Transformer as the backbone network include:

[0028] Build the backbone network: The basic unit of the backbone network is the Swin Transformer block. Each Swin Transformer block includes a window self-attention layer and a feedforward neural network layer.

[0029] Construct the category output layer: The category output layer includes a fully connected layer and a Softmax activation function;

[0030] Construct loss function: The loss function uses cross entropy loss;

[0031] Construct an open set classification post-processing algorithm: After the training of the improved YOLOv8n network model is completed, the training set data is passed through the improved YOLOv8n network model to obtain a corresponding vector, and the average activation vector of each category is obtained; the vector is used to establish a Weibull distribution model of known categories, and the Weibull distribution model is saved as a file; the confidence of each category is calculated based on the Weibull distribution model. If the confidence of the sample is lower than the set threshold, the sample is marked as an unknown category; if the confidence of the sample is higher than the set threshold, the sample is marked as a known category.

[0032] Furthermore, in the construction of the open set classification post-processing algorithm, the midpoint of the feature space of each category is calculated as:

[0033]

[0034] in, is the midpoint of the feature space for each category, is the feature vector of each sample output by the network, is the number of samples;

[0035] The distance from each eigenvector to the center is:

[0036]

[0037] in, is the distance from each eigenvector to the center, is the feature vector of each sample output by the network, is the midpoint of the feature space for each category.

[0038] Furthermore, in constructing the open set classification post-processing algorithm, the probability density function of the Weibull distribution is:

[0039]

[0040] in, is the probability density function of the Weibull distribution, is the scale parameter, is the shape parameter, is the Euclidean distance from the feature vector to the center;

[0041] The cumulative distribution function after substituting the specific data is:

[0042]

[0043] in, is the cumulative distribution function, is the scale parameter, is the shape parameter, is a natural constant, is the Euclidean distance from the feature vector to the center.

[0044] Furthermore, in the construction of the open set classification post-processing algorithm, the probability after reduction by Weibull distribution is:

[0045]

[0046] in, is the probability after reduction through Weibull distribution, is the original Softmax output, is the cumulative distribution function.

[0047] Furthermore, the OpenMax probability of each target is:

[0048]

[0049] in, is the OpenMax probability for each target, is the probability after reduction through Weibull distribution, is the probability after the Weibull distribution is reduced, is the number of categories.

[0050] Furthermore, the steps of training the open set classification network include:

[0051] Data preprocessing and training parameters: Scale and crop the images in the open set classification dataset to unify the input size; apply data enhancement techniques to increase the diversity of data in the open set classification dataset; normalize the pixel values ​​of the images in the open set classification dataset;

[0052] Parameter update: Calculate the model output through forward propagation, and calculate the loss value based on the true label and the predicted result; use the backpropagation algorithm to calculate the gradient of the model parameters in each open set classification network; update the model parameters according to the calculated gradient.

[0053] The beneficial effects achieved by the present invention are:

[0054] The present invention provides a two-stage substation equipment visual defect detection method based on open set classification, by constructing a data set; constructing an improved YOLOv8n network model; training the improved YOLOv8n network model; constructing an open set classification network with Swin Transformer as the backbone network; training the open set classification network; and testing the effect of the proposed two-stage target detection algorithm. The two-stage substation equipment visual defect detection method based on open set classification provided by the present invention proposes a new two-stage target detection framework for the diverse target categories, complex environment and a large amount of visual interference in substation defect detection, and adopts a one-stage target detection algorithm combined with an open set classification algorithm to achieve a stronger anti-interference ability, and shows excellent performance in substation defect detection. The one-stage target detection algorithm adopts the advanced YOLOV8 algorithm as the detection network, and for substation defect detection, the network structure is lightweight, reducing model parameters while ensuring model performance. The backbone network of the open set classification is constructed by the Swin Transformer network, and the Openmax method is used as the open set classification post-processing algorithm, which achieves excellent performance in the open set classification of substation defects. Therefore, the two-stage target detection algorithm proposed in this paper achieves excellent performance in fine-grained defect detection in substations, with an mAP50 of 68.67% and an mAP of 44.71%, which reduces 12M parameters compared with the YOLOV8-1 model with slightly better accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a flow chart of an embodiment of a two-stage substation equipment visual defect detection method based on open set classification according to the present invention;

[0056] Figure 2 This is a two-stage target detection framework diagram of the present invention. DETAILED DESCRIPTION

[0057] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0058] like Figure 1 and Figure 2 As shown, the first embodiment of the present invention proposes a two-stage substation equipment visual defect detection method based on open set classification, comprising the following steps:

[0059] Step S100, constructing a data set: constructing a secondary category target detection data set, a primary category target detection data set and an open set classification data set for defective power equipment, wherein the secondary category target detection data set is used to label the substation defect image data with a secondary category, the primary category target detection data set is used to label the substation defect image data with a primary category, and the open set classification data set is used to perform open set classification on the substation defect image data.

[0060] The defective power equipment is photographed to obtain the substation defect image data. Then the substation defect image data is annotated. Table 1 lists in detail the category information that needs to be annotated and the number of defects photographed, which are divided into primary categories and secondary categories. The secondary categories are used for data annotation of substation defect image data. According to the category characteristics of substation defects, the categories are subdivided according to the similarity of the categories. Therefore, the primary categories in Table 1 may correspond to multiple secondary categories or no subdivision.

[0061] Table 1 Dataset classification

[0062]

[0063] Step S200, constructing an improved YOLOv8n network model: constructing the ELAN module, YOLOv8n backbone network, YOLOv8n neck network, YOLOv8n detection head and YOLOV8n improved loss function in the improved YOLOv8n network model.

[0064] The ELAN module is conducive to further reducing network parameters. Replacing the convolution of the Bottleneck module in this module with a group convolution with a group number of 3 can greatly reduce the parameters of the ELAN module. At the same time, adding a channel reorganization (channel shuffle) operation between adjacent group convolution layers to ensure that the performance is maintained as much as possible while reducing the module parameters. Finally, after ablation experiments, it was found that replacing only the ELAN module in the backbone has the best effect and can achieve a good balance between accuracy and lightness.

[0065] Step S300, training the improved YOLOv8n network model: loading a primary category target detection data set, and sending the primary category target detection data set to the improved YOLOv8n network model for model training after data preprocessing; based on the improved loss function of YOLOV8n, following the principle of forward construction loss, the total loss is calculated during the model training of the primary category target detection data set, and the network parameters are updated through back propagation.

[0066] Step S400, constructing an open set classification network with Swin Transformer as the backbone network: Based on the Swin Transformer block, construct the backbone network, category output layer, loss function and open set classification post-processing algorithm of the open set classification network.

[0067] Step S500, training an open set classification network: loading an open set classification data set, and sending the open set classification data set to the open set classification network for model training after data preprocessing.

[0068] Step S600, test the effect of the proposed two-stage target detection algorithm: based on the improved YOLOv8n network model after loading and training and the weight model of the open set classification network, input the image to be tested in the substation equipment defect test set into the target detection network, predict the substation equipment defect test set, and obtain the bounding box and category confidence of the image on the substation equipment defect test set.

[0069] Load the weight model of the optimal YOLOv8n and Swin Transformer open set classification network after training, and then input the image into the target detection network. The target detection network will then output several pre-selected boxes, and the first 64 pre-selected boxes are obtained by confidence sorting. Input the images of the first 64 pre-selected boxes into the classification network for classification. Correct the output results of the target detection network. Repeat the above steps to predict the test set and obtain the bounding box and category confidence of the image on the test set.

[0070] Further, see Figure 1 The two-stage substation equipment visual defect detection method based on open set classification proposed in this embodiment, step S100 includes:

[0071] Step S110, constructing a secondary category target detection data set: photographing power equipment with defects to obtain substation defect image data; and annotating the substation defect image data with secondary category defects.

[0072] First, the defective power equipment is photographed to obtain the substation defect image data. Then, labelimg and other software are used to annotate the image data. Table 1 lists in detail the category information that needs to be annotated and the number of defects photographed, which are divided into primary categories and secondary categories. The secondary categories are used for data annotation of the image data. According to the category characteristics of substation defects, the categories are subdivided according to the similarity of the categories. Therefore, the primary categories in Table 1 will correspond to multiple secondary categories or no subdivision. Finally, the annotated data set is divided into training set and test set in a ratio of 7:3.

[0073] Step S120, constructing a primary category target detection data set: according to the secondary category target detection data set and the preset correspondence between the secondary category and the primary category, the secondary categories are merged to obtain the primary category target detection data set.

[0074] In step S110, the substation defect data set with the second-level category has been labeled. According to the correspondence between the second-level category and the first-level category in Table 1, the second-level categories are merged to obtain the substation defect data set labeled with the first-level category.

[0075] Step S130, constructing an open-set classification data set: according to the secondary category target detection data set and the preset secondary category category segmentation, using the label data of the training set in the secondary category target detection data set, intercepting the image of the corresponding area, and obtaining the open-set classification data set.

[0076] In step S110, the substation defect dataset of the secondary category has been labeled. The substation defect categories subdivided in Table 1 are used as known categories for open set classification, and a training dataset for open set classification is prepared. The specific preparation method is to use the label data of the training set in the secondary category target detection dataset to capture the image of the corresponding area. Then the verification dataset is prepared in the same way. The difference is that several additional category data need to be added to the verification set as unknown categories to verify the open set classification effect of the algorithm. Therefore, two categories that may appear in the substation scene are added from the PASCAL VOC 2012 dataset as unknown categories, namely people and birds.

[0077] Preferably, see Figure 1 The two-stage substation equipment visual defect detection method based on open set classification proposed in this embodiment, step S200 includes:

[0078] Step S210, constructing an ELAN module: replacing the convolution of the Bottleneck module with a group convolution with a group number of 3, and adding a channel reorganization operation between adjacent group convolution layers.

[0079] Construct the ELAN module, replace the convolution of the Bottleneck module with a group convolution with a group number of 3, and add channel reorganization operations between adjacent group convolution layers to ensure that the performance is maintained as much as possible while reducing the module parameters. Finally, after ablation experiments, it was found that replacing only the ELAN module in the backbone has the best effect and can achieve a good balance between accuracy and lightness.

[0080] Step S220, constructing a YOLOv8n backbone network: using two 3×3 convolutional layers with a stride of 2 to downsample the feature map of the input image by four times, sequentially passing through four ELAN modules, and setting a 3×3 convolutional layer with a stride of 2 between every two ELAN modules.

[0081] First, the feature map of the input image is downsampled four times using two 3×3 convolutional layers with a stride of 2. Then, it passes through four new ELAN modules constructed in step S210 in sequence, and a 3×3 convolutional layer with a stride of 2 is set between every two ELAN modules. Finally, the SPPF (Spatial Pyramid Pooling Fast) module is used to improve the network's ability to extract information of different scales.

[0082] Step S230, constructing a YOLOv8n neck network: three feature layers are derived from the backbone network as inputs of the neck network, wherein the large-scale feature layer comes from the output of the second ELAN module, the medium-scale feature layer comes from the output of the third ELAN module, and the small-scale feature layer comes from the output of the SPPF module; after upsampling the small-scale feature layer, the ELAN module is used to splice the medium-scale feature layer to obtain a new medium-scale feature layer; the medium-scale feature layer is transformed and then merged with the large-scale feature layer; the large-scale feature layer is directly output to the head network, and at the same time, after downsampling the large-scale feature layer, the ELAN module is used to merge the medium-scale feature layer to obtain a new medium-scale feature layer, which is then output to the head network; the medium-scale feature layer is fused with the small-scale feature layer, and then the small-scale feature layer is output to the head network; the neck network outputs three fused feature layers of large, medium and small.

[0083] First, three feature layers are derived from the backbone network as the input of the neck network. The large-scale feature layer comes from the output of the second ELAN module, the medium-scale feature layer comes from the output of the third ELAN module, and the small-scale feature layer comes from the output of the SPPF module. Then the small-scale feature layer is upsampled and passed through the ELAN module, and then concatenated with the medium-scale feature layer to obtain a new medium-scale feature layer. Similarly, the medium-scale feature layer is transformed and merged with the large-scale feature layer. Then the large-scale feature layer is directly output to the head network. At the same time, the large-scale feature layer is downsampled and passed through the ELAN module and merged with the medium-scale feature layer to obtain a new medium-scale feature layer, which is then output to the head network. The medium-scale feature layer must undergo the same steps to fuse with the small-scale feature layer, and then the small-scale feature layer is output to the head network. Finally, the neck network outputs three fused feature layers: large, medium, and small.

[0084] Step S240, constructing a YOLOv8n detection head: The YOLOV8 network includes detection heads of three scales, each of which adopts a design that decouples positioning prediction and classification prediction.

[0085] The YOLOV8 network mainly includes detection heads of three scales. Each detection head adopts a design that decouples positioning prediction and classification prediction. Specifically, there are two branches in each detection head, and each branch contains three convolutional layers, two 3×3 convolutional layers and one 1×1 convolutional layer.

[0086] Step S250, construct an improved loss function: find a suitable weight coefficient by repeatedly adjusting the proportion of YOLOV8 network positioning box regression loss, target classification loss and DF Loss in the total loss.

[0087] As the candidate region proposal network (RPN) of the two-stage algorithm, the YOLOv8n network needs to provide more detection boxes with higher category confidence. Therefore, by repeatedly adjusting the proportion of the positioning box regression loss, target classification loss, and DF Loss in the total loss, a set of suitable weight coefficients were found, which are 0.1, 7.5, and 1.5 respectively.

[0088] Further, see Figure 1 The two-stage substation equipment visual defect detection method based on open set classification proposed in this embodiment, step S300 includes:

[0089] Step S310, pre-training preparation and training configuration: load the first-level category target detection data set, and send the first-level category target detection data set to the improved YOLOv8n network model for model training after data preprocessing, wherein the data preprocessing includes normalization, image scale transformation, random cropping, rotation and color transformation.

[0090] First, the first-level category substation defect target detection data set prepared in step S120 is loaded and sent to the network for training after data preprocessing. In this embodiment, the preprocessing steps of the data mainly include: normalization, image scale transformation, random cropping, rotation and color transformation and other data enhancement methods to improve the generalization ability of the model. In addition, the SGD optimizer is selected during the model training process, and the learning rate is set to 0.01, the learning rate adjustment strategy is cosine decay, and the number of training iterations is 150 rounds.

[0091] Step S320, network parameter update and loss calculation: Based on the improved loss function of YOLOV8n, follow the principle of forward construction loss to calculate the total loss during the model training process of the first-level category target detection dataset, and back propagate to update the network parameters; in the model training process, the model parameter averaging strategy is adopted to automatically average the model parameters of three adjacent rounds, so that the model training is smoother.

[0092] The loss structure of the improved YOLOV8n has been described in step S250. Therefore, during the model training process, it is only necessary to follow the forward-built loss to calculate the total loss and back-propagate to update the network parameters. Finally, the model parameter averaging strategy is adopted in the model training process. During the model training process, the model parameters of three adjacent rounds are automatically averaged to make the model training smoother.

[0093] Preferably, see Figure 1 The two-stage substation equipment visual defect detection method based on open set classification proposed in this embodiment, step S400 includes:

[0094] Step S410, constructing a backbone network: The basic unit of the backbone network is a Swin Transformer block, and each Swin Transformer block includes a window self-attention layer and a feedforward neural network layer.

[0095] The model adopts a layered design, dividing the input image into multiple non-overlapping windows and applying a self-attention mechanism within each window to capture local features. The Swin Transformer introduces a moving window mechanism so that windows overlap between different layers, enhancing the ability to capture global information. The basic unit of the model is the Swin Transformer block, each of which consists of two main parts: a window self-attention layer and a feedforward neural network layer. The window self-attention layer is responsible for feature extraction within the local window, while the feedforward neural network performs a nonlinear transformation on the features at each position.

[0096] Step S420, constructing a category output layer: the category output layer includes a fully connected layer and a Softmax activation function.

[0097] The category output layer structure of the Swin Transformer classification network mainly consists of a fully connected layer and a Softmax activation function. First, after feature extraction through multiple layers of Swin Transformer blocks, the model performs global average pooling on the final feature map and converts it into a fixed-length feature vector. The feature vector is input to a fully connected layer whose output dimension corresponds to the number of categories in the target classification task. In order to enhance the generalization ability of the model, a Dropout layer is applied before the fully connected layer to prevent overfitting.

[0098] Step S430, constructing a loss function: the loss function adopts cross entropy loss.

[0099] The loss function of the Swin Transformer open set classification network uses cross entropy loss. After the model output layer generates the probability distribution of each category, the cross entropy loss function calculates the loss value of the model by comparing the difference between the predicted probability and the true label.

[0100] Step S440, constructing an open set classification post-processing algorithm: after the training of the improved YOLOv8n network model is completed, the training set data is respectively obtained through the improved YOLOv8n network model to obtain a corresponding vector, and the average activation vector of each category is obtained; the vector is used to establish a Weibull distribution model of a known category, and the Weibull distribution model is saved as a file; the confidence of each category is calculated based on the Weibull distribution model, if the confidence of the sample is lower than the set threshold, the sample is marked as an unknown category; if the confidence of the sample is higher than the set threshold, the sample is marked as a known category.

[0101] To construct the open set classification post-processing algorithm, the steps of OpenMax include the following parts. First, after the model training is completed, a corresponding vector can be obtained by passing the training set data through the model respectively. Therefore, the average activation vector of each category can be calculated using the data of the training set. This vector can be used to establish a Weibull distribution model of known categories. This model is saved as a file and can be used directly in the model inference stage. OpenMax calculates the confidence of each category based on the Weibull distribution model calculated above. If the confidence of the sample is lower than the set threshold, the sample will be marked as an unknown category; otherwise, it will be classified as a known category according to the highest probability. The specific processing steps of OpenMax are as follows. First, formula 1 calculates the midpoint of the feature space of each category , is the feature vector of each sample output by the network. In Formula 2 is the distance from each eigenvector to the center. Formula 3 is the probability density function of the Weibull distribution. Formula 4 is the cumulative distribution function after substituting specific data. In Formula 5 is the original Softmax output, is the probability after reduction by Weibull distribution. Finally, the OpenMax probability of each target is calculated using Formula 6.

[0102] In constructing the open set classification post-processing algorithm, the midpoint of the feature space of each category is calculated as:

[0103] (1)

[0104] In formula (1), is the midpoint of the feature space for each category, is the feature vector of each sample output by the network, is the number of samples.

[0105] The distance from each eigenvector to the center is:

[0106] (2)

[0107] In formula (2), is the distance from each eigenvector to the center, is the feature vector of each sample output by the network, is the midpoint of the feature space for each category.

[0108] In constructing the open set classification post-processing algorithm, the probability density function of the Weibull distribution is:

[0109] (3)

[0110] In formula (3), is the probability density function of the Weibull distribution, is the scale parameter, is the shape parameter, is the Euclidean distance from the feature vector to the center;

[0111] The cumulative distribution function after substituting the specific data is:

[0112] (4)

[0113] In formula (4), is the cumulative distribution function, is the scale parameter, is the shape parameter, is a natural constant, is the Euclidean distance from the feature vector to the center.

[0114] In constructing the open set classification post-processing algorithm, the probability after reduction through Weibull distribution is:

[0115] (5)

[0116] In formula (5), is the probability after reduction through Weibull distribution, is the original Softmax output, is the cumulative distribution function.

[0117] The OpenMax probability of each target is:

[0118] (6)

[0119] In formula (6), is the OpenMax probability for each target, is the probability after reduction through Weibull distribution, is the probability after the Weibull distribution is reduced, is the number of categories.

[0120] Preferably, see Figure 1 The two-stage substation equipment visual defect detection method based on open set classification proposed in this embodiment, step S500 includes:

[0121] Step S510, data preprocessing and training parameters: scaling and cropping the images in the open set classification data set to unify the input size; applying data enhancement technology to increase the diversity of data in the open set classification data set; normalizing the pixel values ​​of the images in the open set classification data set.

[0122] First, the image is read and loaded to ensure the integrity and accuracy of the dataset. The image is scaled and cropped before being input into the network to unify the input size. The image is usually adjusted to 128×128 pixels to meet the input requirements of the model. Next, data augmentation techniques such as random rotation, flipping, color jittering, and cropping are applied to increase the diversity of the data and thus improve the generalization ability of the model. Finally, standardization is performed to normalize the pixel values ​​of the image to reduce the differences between different features. Some configuration instructions for the model training process: transfer learning is performed based on pre-trained weights during model training. Therefore, the SGD optimizer is selected during the model training process, the learning rate is set to 0.001, the learning rate adjustment strategy is cosine decay, the minimum learning rate of decay is 0.00002, and the number of training iterations is 50.

[0123] Step S520, parameter update: calculate the model output through forward propagation, and calculate the loss value according to the true label and the prediction result; use the back propagation algorithm to calculate the gradient of the model parameters in each open set classification network; update the model parameters according to the calculated gradient.

[0124] First, the model output is calculated through forward propagation, and the loss value is calculated based on the true label and the predicted result. Next, the gradient of the loss to each model parameter is calculated using the backpropagation algorithm. Then, the model parameters are updated according to the calculated gradient, and the learning rate is usually used to control the update amplitude. Finally, the above steps are repeated and iterative training is performed for 50 epochs until the model converges or reaches the preset performance index. During the training process, ensure that the model continuously adjusts the parameters in a better direction to reduce the loss and improve the classification accuracy.

[0125] Experimental description:

[0126] The comparative experiment of the target detection algorithm in the first phase is shown in Table 2. The dataset used in training the target detection algorithm in the first phase is the target detection dataset constructed by the first-level category in Table 1. In the comparative experiment, YOLOV5-s / l, YOLOX-tiny / l, YOLOV7-tiny / l and YOLOV8-n / l were selected as the comparison methods of this experiment. YOLOV8-n is our improved baseline model. The YOLOV8-n^ model is an improved lightweight model. In view of the parameter redundancy of the ELAN module, a new ELAN module using grouped convolution and channel shuffle is proposed. As can be seen from Table 2, the improved YOLOV8-n model has fewer parameters and stronger performance.

[0127] Table 2 Comparative experiment of first-level category target detection

[0128]

[0129] Table 3 shows the open set classification algorithm comparison experiment. The ones without * indicate that the threshold Softmax method is used for post-processing, and the ones with * indicate that the Openmax method is used for post-processing. Seven classic classification networks are used in the open set classification comparison experiment. The Swin Transformer model network based on the Transformer structure shows significantly better performance than other networks in open set classification. For example, the accuracy is 2.3% higher than Resnet34* when the number of parameters is similar. In terms of lightweight open set classification, Mobilenetv2* has good performance with only 3.5M parameters.

[0130] Table 3 Open set classification comparison experiment

[0131]

[0132] Table 4 is a comparative experiment of substation defect detection of secondary category, where “+St” means using Swin Transformer model and threshold Softmax as post-processing algorithm; “+St*” means using Swin Transformer model and Openmax as post-processing algorithm; “+M” means using Mobilenetv2 model and threshold Softmax as post-processing algorithm; “+M*” means using Mobilenetv2 model Openmax as post-processing algorithm.

[0133] Compared with the YOLOV8-1 model, a baseline model with a large number of parameters, the YOLOV8-n^+St* algorithm proposed in the present invention reduces 12M model parameters compared with the YOLOV8-1 model, and shows superior performance in substation defect detection compared with the YOLOV8-1 model. Compared with traditional single-stage target detectors such as YOLOV8, the YOLOV8-n^+St* algorithm has considerable advantages in both parameter quantity and detection accuracy.

[0134] Compared with the lightweight YOLOV8-n model, the YOLOV8-n^+M* algorithm we proposed has fewer parameters and higher performance, and can still show strong competitiveness in the field of real-time substation defect detection. Compared with the YOLOV8-n model, mAP50 is improved by 2.34% and mAP is improved by 1.15%.

[0135] Table 4 Comparative experiment of secondary category target detection

[0136]

[0137] The two-stage substation equipment visual defect detection method based on open set classification provided in this embodiment, compared with the prior art, constructs a data set; constructs an improved YOLOv8n network model; trains the improved YOLOv8n network model; constructs an open set classification network with Swin Transformer as the backbone network; trains the open set classification network; and tests the effect of the proposed two-stage target detection algorithm. The two-stage substation equipment visual defect detection method based on open set classification provided by the present invention proposes a new two-stage target detection framework for the diverse target categories, complex environment and a large amount of visual interference in substation defect detection, and adopts a combination of a one-stage target detection algorithm and an open set classification algorithm to achieve stronger anti-interference ability and show excellent performance in substation defect detection. The one-stage target detection algorithm adopts the advanced YOLOV8 algorithm as the detection network, and for substation defect detection, the network structure is lightweight, reducing model parameters while ensuring model performance. The backbone network of the open set classification is constructed by the Swin Transformer network, and the Openmax method is used as the open set classification post-processing algorithm, which achieves excellent performance in the open set classification of substation defects. Therefore, the two-stage target detection algorithm proposed in this embodiment achieves excellent performance in substation fine-grained defect detection, with an mAP50 of 68.67% and an mAP of 44.71%. With slightly better accuracy, it reduces 12M parameters compared to the YOLOV8-1 model.

[0138] Although preferred embodiments of the present invention have been described, additional changes and modifications may be made to these embodiments by those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A two-stage substation equipment visual defect detection method based on open set classification, characterized in that: The following steps are involved: Constructing data sets: constructing secondary category target detection data sets, primary category target detection data sets and open set classification data sets of power equipment with defects. The secondary category target detection data sets are used to label the secondary categories of substation defect image data, the primary category target detection data sets are used to label the primary categories of substation defect image data, and the open set classification data sets are used to perform open set classification of substation defect image data; Build an improved YOLOv8n network model: Build the ELAN module, YOLOv8n backbone network, YOLOv8n neck network, YOLOv8n detection head and YOLOV8n improved loss function in the improved YOLOv8n network model; Training the improved YOLOv8n network model: loading the first-level target detection data set, and sending the first-level target detection data set to the improved YOLOv8n network model for model training after data preprocessing; based on the improved loss function of YOLOV8n, following the principle of forward construction loss, the total loss is calculated during the model training of the first-level target detection data set, and the network parameters are updated through back propagation; Construct an open-set classification network with Swin Transformer as the backbone network: Based on the Swin Transformer block, construct the backbone network, category output layer, loss function and open-set classification post-processing algorithm of the open-set classification network; Training the open set classification network: loading the open set classification data set, and sending the open set classification data set to the open set classification network for model training after data preprocessing; The effect of the proposed two-stage target detection algorithm is tested: based on the improved YOLOv8n network model after loading and training and the weight model of the open set classification network, the images to be tested in the substation equipment defect test set are input into the target detection network, and the substation equipment defect test set is predicted to obtain the bounding boxes and category confidence of the images on the substation equipment defect test set.

2. The two-stage substation equipment visual defect detection method based on open set classification as claimed in claim 1 is characterized in that: The steps of constructing the data set include: Constructing a secondary category target detection data set: photographing power equipment with defects to obtain substation defect image data; annotating the substation defect image data with secondary category defects; Constructing a primary category target detection data set: according to the secondary category target detection data set and the preset correspondence between the secondary category and the primary category, merging the secondary categories to obtain a primary category target detection data set; Constructing an open-set classification dataset: Based on the secondary category target detection dataset and the preset secondary category segmentation, using the label data of the training set in the secondary category target detection dataset, images of corresponding areas are captured to obtain an open-set classification dataset.

3. The two-stage substation equipment visual defect detection method based on open set classification as claimed in claim 1 is characterized in that: The steps of constructing the improved YOLOv8n network model include: Construct the ELAN module: replace the convolution of the Bottleneck module with a grouped convolution with a group number of 3, and add a channel reorganization operation between adjacent grouped convolution layers; Build the YOLOv8n backbone network: use two 3×3 convolutional layers with a stride of 2 to downsample the feature map of the input image by four times, pass it through four ELAN modules in sequence, and set a 3×3 convolutional layer with a stride of 2 between every two ELAN modules; Construct the YOLOv8n neck network: three feature layers are derived from the backbone network as the input of the neck network, among which the large-scale feature layer comes from the output of the second ELAN module, the medium-scale feature layer comes from the output of the third ELAN module, and the small-scale feature layer comes from the output of the SPPF module; after upsampling the small-scale feature layer, it is spliced ​​with the medium-scale feature layer through the ELAN module to obtain a new medium-scale feature layer; after transforming the medium-scale feature layer, it is merged with the large-scale feature layer; the large-scale feature layer is directly output to the head network, and at the same time, after downsampling the large-scale feature layer, it is merged with the medium-scale feature layer through the ELAN module to obtain a new medium-scale feature layer, which is then output to the head network; the medium-scale feature layer is fused with the small-scale feature layer, and then the small-scale feature layer is output to the head network; the neck network outputs three fused feature layers of large, medium and small; Build YOLOv8n detection head: The YOLOV8 network contains three scales of detection heads, each of which adopts a design that decouples positioning prediction and classification prediction; Build an improved loss function: Find the appropriate weight coefficient by repeatedly adjusting the YOLOV8 network positioning box regression loss, target classification loss, and the proportion of DFLoss in the total loss.

4. The two-stage substation equipment visual defect detection method based on open set classification as claimed in claim 1 is characterized in that: The steps of training the improved YOLOv8n network model include: Pre-training preparation and training configuration: loading the first-level category target detection data set, and sending the first-level category target detection data set to the improved YOLOv8n network model for model training after data preprocessing, wherein the data preprocessing includes normalization, image scale transformation, random cropping, rotation and color transformation; Network parameter update and loss calculation: Based on the improved loss function of YOLOV8n, the total loss is calculated during the model training of the first-level category target detection dataset in accordance with the principle of forward construction loss, and the network parameters are updated through back propagation. In the model training process, the model parameter averaging strategy is adopted to automatically average the model parameters of three adjacent rounds, so that the model training is smoother.

5. The two-stage substation equipment visual defect detection method based on open set classification as claimed in claim 1 is characterized in that: The steps of constructing an open set classification network with Swin Transformer as the backbone network include: Constructing a backbone network: The basic unit of the backbone network is a Swin Transformer block, each of which includes a window self-attention layer and a feedforward neural network layer; Constructing a category output layer: the category output layer includes a fully connected layer and a Softmax activation function; Construct loss function: The loss function uses cross entropy loss; Construct an open set classification post-processing algorithm: After the training of the improved YOLOv8n network model is completed, the training set data is respectively passed through the improved YOLOv8n network model to obtain a corresponding vector, and the average activation vector of each category is obtained; the vector is used to establish a Weibull distribution model of known categories, and the Weibull distribution model is saved as a file; the confidence of each category is calculated based on the Weibull distribution model. If the confidence of the sample is lower than the set threshold, the sample is marked as an unknown category; if the confidence of the sample is higher than the set threshold, the sample is marked as a known category.

6. The two-stage substation equipment visual defect detection method based on open set classification as claimed in claim 5 is characterized in that: In the open set classification post-processing algorithm, the midpoint of the feature space of each category is calculated as: in, is the midpoint of the feature space for each category, is the feature vector of each sample output by the network, is the number of samples; The distance from each eigenvector to the center is: in, is the distance from each eigenvector to the center, is the feature vector of each sample output by the network, is the midpoint of the feature space for each category.

7. The two-stage substation equipment visual defect detection method based on open set classification as claimed in claim 6 is characterized in that: In the open set classification post-processing algorithm, the probability density function of the Weibull distribution is: in, is the probability density function of the Weibull distribution, is the scale parameter, is the shape parameter, is the Euclidean distance from the feature vector to the center; The cumulative distribution function after substituting the specific data is: in, is the cumulative distribution function, is the scale parameter, is the shape parameter, is a natural constant, is the Euclidean distance from the feature vector to the center.

8. The two-stage substation equipment visual defect detection method based on open set classification as claimed in claim 7 is characterized in that: In the open set classification post-processing algorithm, the probability after reduction by Weibull distribution is: in, is the probability after reduction through Weibull distribution, is the original Softmax output, is the cumulative distribution function.

9. The two-stage substation equipment visual defect detection method based on open set classification as claimed in claim 8, characterized in that: The OpenMax probability of each target is: in, is the OpenMax probability for each target, is the probability after reduction through Weibull distribution, is the probability after the Weibull distribution is reduced, is the number of categories.

10. The two-stage substation equipment visual defect detection method based on open set classification according to claim 1 is characterized in that: The step of training the open set classification network comprises: Data preprocessing and training parameters: scaling and cropping the images in the open set classification dataset to unify the input size; applying data enhancement technology to increase the diversity of data in the open set classification dataset; normalizing the pixel values ​​of the images in the open set classification dataset; Parameter update: Calculate the model output through forward propagation, and calculate the loss value based on the true label and the predicted result; use the backpropagation algorithm to calculate the gradient of the model parameters in each open set classification network; update the model parameters according to the calculated gradient.

Images