Capsule network box-type substation fault diagnosis method based on residual error improvement
By introducing residual modules and dynamic routing mechanisms into the capsule network, combining differential evolution algorithm to optimize hyperparameters, and building a Residual-CapsNet model, the local information loss and insufficient global information mastery in box substation fault diagnosis are solved. The problem of the problem has achieved high accuracy fault diagnosis.
Patent Information
- Application Number
- CN202410974623.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-19
- Publication Date
- 2024-11-19
AI Technical Summary
Traditional convolutional neural networks have problems such as local information loss and insufficient global information mastery in box substation fault diagnosis, resulting in insufficient feature extraction capabilities and degradation of performance.
Using a capsule network based on residual improvement, the Residual-CapsNet model is built by introducing residual modules and dynamic routing mechanisms to avoid information loss caused by pooling operations, and using differential evolution algorithm to optimize hyperparameters to improve the model's feature extraction capability and diagnostic accuracy.
It effectively solves the problem of performance degradation of traditional models, improves the accuracy of fault diagnosis and the feature extraction ability of the model, reduces the difficulty of calculation, and improves training efficiency.
Smart Images

Figure CN118981702A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a capsule network box-type substation fault diagnosis method based on residual improvement, belonging to the technical field of box-type substation fault diagnosis. Background Art
[0002] my country has the world's highest voltage level, largest scale, most complex structure, and largest annual power generation power system. As the end of the power system, the distribution network undertakes the important task of obtaining electricity from the transmission network or regional power plants, building a multi-level distribution network, and distributing electricity to users safely, reliably, and economically. As one of the key equipment in the distribution network, the timely diagnosis and treatment of box-type substations is of great significance to the safety of the distribution network and even the power system. The traditional means of fault diagnosis and treatment of box-type substations is to invest a lot of manpower and material resources in regular inspection and maintenance. This method of regular maintenance of equipment that relies on human subjective judgment lacks sufficient scientific theoretical support and is too mechanized. It not only requires inspection and maintenance personnel to have a lot of prior knowledge and experience, but also is prone to missed inspections, wrong inspections, and re-inspections, resulting in serious waste of manpower and material resources. Relevant data show that the time for manual troubleshooting of faulty components accounts for more than 30% of the entire fault handling process. Therefore, timely judging the fault type of the box-type substation and locating the faulty component according to the fault type can help shorten the fault location time and significantly improve the fault handling efficiency.
[0003] With the development of deep learning in the field of engineering applications, some methods have been applied to the fault diagnosis of box-type substations. Among them, the application of deep learning models represented by convolutional neural networks (CNN) has made the field of fault diagnosis develop in the direction of intelligence. Although convolutional neural networks have achieved certain results in fault diagnosis, the traditional convolutional neural network family usually uses pooling operations to maintain the perspective invariance of the model, resulting in the loss of some valuable information in the input samples, and the inability to pay attention to the global information of the input samples, and it is easy to ignore the correlation between the various indicators of the equipment monitoring data. At the same time, the feature extraction ability of the model will improve with the deepening of the network layer, but when the network depth reaches a certain level, performance degradation problems will occur, resulting in poor performance of the fault diagnosis model. Therefore, in order to avoid the problem of local information loss and insufficient global information grasp caused by the pooling operation of the traditional convolutional neural network family, and to ensure that the fault diagnosis model has a high feature extraction ability and no performance degradation, it is necessary to establish a high-performance fault diagnosis model that avoids pooling operations. Summary of the invention
[0004] The technical problem to be solved by the present invention is to provide a capsule network box-type substation fault diagnosis method based on residual improvement, so as to solve the problem of local information loss and insufficient global information grasp caused by the pooling operation of the above-mentioned traditional convolutional neural network family, ensure that the fault diagnosis model has a high feature extraction capability without performance degradation, and improve the accuracy of fault diagnosis.
[0005] The technical solution adopted by the present invention is: a capsule network box-type substation fault diagnosis method based on residual improvement, the method comprising the following steps:
[0006] Step 1: Process the data set, that is, divide the data set (into training set, validation set, and test set) and perform unique hot encoding on the fault types of the box-type substation for subsequent model training and performance evaluation;
[0007] Step 2: Construct a capsule network based on residual improvement; the capsule network based on residual improvement includes residual convolution layer and capsule layer in sequence, and the capsule layer is divided into main capsule layer and digital capsule layer.
[0008] Step 3: Input the training set and validation set into the residual-based improved capsule network fault diagnosis model for training. Use the back propagation algorithm to train the residual capsule network. Through forward excitation propagation and reverse weight update, continuously optimize the number of convolutional layers, convolution kernel size, model learning rate, training batch size, and maximum number of training iterations of the residual convolutional network in the model to obtain a trained box-type substation fault diagnosis model.
[0009] Step 4: Input the test set into the trained box-type substation fault diagnosis model to obtain the box-type substation fault diagnosis result.
[0010] Furthermore, the data set processing in the above step 1 includes the following steps:
[0011] Step 1.1: Divide the input data set into a training set and a validation set in a ratio of 7:3, and use the original data set that has not been derived from data as the test set. The training set is used to train the model; the validation set is used to verify the generalization performance indicators of the model and serve as a basis for judging hyperparameter tuning; the test set is used as the data basis for the final evaluation of the trained model;
[0012] Step 1.2, the one-hot encoding of the fault type of the box-type substation means that when training the residual capsule network diagnosis model, in order to facilitate the calculation of the loss function, the fault type of the box-type substation needs to be processed in the form of one-hot encoding.
[0013] Furthermore, the specific method of constructing the capsule network based on residual improvement in step 2 is as follows: introducing the residual block into the convolutional layer of the capsule network (CapsNet), establishing a capsule network (Residual-CapsNet) fault diagnosis model based on residual improvement, and avoiding the deterioration of model performance caused by the network being too deep through residual connection, so as to complete accurate fault diagnosis.
[0014] Furthermore, the main function of the residual convolution layer in the above step 2 is to extract features from the input box-type transformer operating parameters and environmental indicator data using residual convolution operations. Specifically, the residual convolution layer performs convolution operations on the box-type substation monitoring data through the sliding window of the convolution kernel, extracts features (such as correlation features that show the correlation between different parameters, data peaks, data trends, etc.) through weighted calculation and adding bias, and performs residual learning through jump connections, and transmits the obtained residual information back to the previous convolution layer, and continuously optimizes the feature extraction results through a dynamic feedback mechanism, thereby achieving better feature expression effects. Compared with ordinary convolution layers, the residual convolution block connects different convolution layers in a residual connection manner, and can add enough convolution layers to improve the feature extraction capability of the model without worrying about gradient disappearance or gradient explosion problems.
[0015] The convolution operation of the residual convolution layer is as follows
[0016]
[0017] Where:
[0018] *——Extended convolution operation;
[0019] U i ——is the output of the lth convolutional layer;
[0020] ——The input in the area where the i-th convolution kernel of the l-1th layer is located, that is, the i-th indicator of the data collected by the box-type substation or its features obtained after convolution;
[0021] ——The weight of the i-th convolution kernel in the l-th layer;
[0022] ——The bias of the i-th convolution kernel in the l-th layer;
[0023] f(·) — activation function.
[0024] Furthermore, the structure of the capsule layer in the above step 2 is divided into a main capsule layer and a digital capsule layer as follows:
[0025] The capsule layer is divided into the main capsule layer and the digital capsule layer; different from the "Scalar in, Scalar out" feature of the convolutional layer, the input and output of the capsule layer are expressed in the form of vectors, that is, the "Vector in, Vector out" strategy. When the capsule unit of the capsule layer performs feature detection, the detection probability is used as the length of the vector, and the feature state is used as the direction of the vector. That is, the direction of the capsule unit output vector represents the state encoding of various operating parameters and environmental indicators of the box-type substation, and the vector length represents the probability encoding of the box-type substation expressed by this input being in a certain fault state. The calculation process of the capsule layer is mainly divided into the following two steps:
[0026] Step 1: The output scalar of the residual convolution layer is nonlinearly converted into a prediction vector through vector weighted calculation, which is used as the input vector of the main capsule layer. The calculation formula is:
[0027] U ji =W ji U i
[0028] Where:
[0029] W ji ——The weight matrix of the main capsule layer is formed by generating random numbers using the standard normal initialization method;
[0030] U i ——The output scalar of the lth residual convolution layer;
[0031] U ji ——The jth output vector generated by the weighted nonlinear transformation of the i-th input feature scalar.
[0032] Step 2: By calculating the similarity between the prediction vector output of the main capsule layer and the prediction vector output of the digital capsule layer, the coupling coefficient between the features is adjusted, so as to selectively transmit the prediction vector of the main capsule layer to the digital capsule layer, i.e., the dynamic routing mechanism. The mathematical calculation formula is as follows:
[0033]
[0034] Where:
[0035] C ij ——The coupling coefficient between the i-th vector and the j-th vector in the main capsule layer;
[0036] p ij ——The bias of the coupling coefficient between the i-th vector and the j-th vector in the main capsule layer;
[0037] S j ——The total input vector of the jth layer is obtained by multiplying the prediction vector of the main capsule layer by the coupling coefficient.
[0038] When C ij When it is 1, all the output information of the main capsule layer is passed to the digital capsule layer. If it is 0, the capsule is discarded.
[0039] In the above two steps, the capsule network continuously updates the coupling coefficient and bias of the digital capsule layer through the iterative execution of the dynamic routing algorithm. In each iteration, the dynamic router calculates the instance-level weights based on the output vector of the main capsule layer and the similarity, overlap and other indicators of all capsules in the digital capsule layer. These instance-level weights are used to adjust the coupling coefficient and bias of the digital capsule layer to more accurately describe the input data. The entire iterative process will continue until the optimal digital capsule weight is found, which is coupled with the main capsule in the best way to achieve better signal transmission and accuracy improvement, thereby improving the accuracy of fault diagnosis of box-type substations.
[0040] Since the capsule layer outputs in the form of vectors, the capsule network does not use the Sigmoid function, SoftMax function or tanh function commonly used in traditional convolutional neural networks as the activation function, but uses a nonlinear vector activation function - Squash to receive the vector output by the capsule layer, and compress its length to less than 1 without changing its direction, thus obtaining the fault classification result. The formula of the Squash function is:
[0041]
[0042] Where:
[0043] ——Input vector S j The reduction factor of
[0044] ——Input vector S j The unit vector of .
[0045] Furthermore, the steps of inputting the training set and the validation set into the residual-based improved capsule network fault diagnosis model for training in the above step 3 are as follows:
[0046] Step 3.1, set hyperparameters: find the best network structure and model hyperparameters on the training set and test set, including the number of convolutional layers of the residual convolutional network, the size of the convolution kernel, the learning rate of the model, the training batch size, and the maximum number of training iterations. Due to the large number of parameter combinations, manual parameter adjustment requires repeated adjustment and training, which is inefficient and difficult to find the global optimal parameter combination. The differential evolution algorithm (DE) is used to automatically optimize the hyperparameters of the residual improved capsule network model;
[0047] Step 3.2, Initialize the model: Initialize the trainable parameters in the model, such as convolution kernel weights, Dropout ratio, etc. It should be noted that this model is a deep learning model, and the all-zero initialization strategy should be avoided to prevent the parameters of each layer of the model from remaining consistent after the back-propagation algorithm is updated, resulting in the model being unable to mine truly valuable information;
[0048] Step 3.3, set the early stopping mechanism. Under the maximum number of iterations (obtained by hyperparameter tuning), if the accuracy of the validation set is improved by less than 0.05% after 30 consecutive iterations of the model, stop training early to save time.
[0049] Step 3.4, input the training set and validation set into the model for training, use the back propagation algorithm to train the residual capsule network, and continuously optimize the parameters in the model through forward excitation propagation and reverse weight update. The input data set is passed to the output layer through the residual convolution layer and the capsule layer, and the loss function value is calculated by the diagnosis result and the actual result; when the loss function reaches convergence, the training is completed; and the trained box-type substation fault diagnosis model is obtained.
[0050] Furthermore, the process of automatically optimizing the hyperparameters of the residual improved capsule network model using the differential evolution algorithm in the above step 3.1 is as follows: determine the differential evolution algorithm control parameters and fitness function, the control parameters include population size, scaling factor and hybridization probability; randomly generate an initial population; evaluate the initial population, that is, calculate the fitness of individuals in the population; determine whether each individual meets the final optimization conditions or whether the number of evolutionary iterations reaches the maximum, if so, the individual is output as the optimal solution; if not, continue; perform "mutation" and "crossover" operations to obtain an intermediate population; select individuals from the original population and the intermediate population to obtain a new generation of original population; the number of evolutions is +1, and the process is repeated.
[0051] Beneficial effects of the present invention: Compared with the prior art, the present invention introduces a residual block structure into the convolution layer of the capsule network to form a capsule network fault diagnosis model based on residual improvement, and uses the residual principle of the residual block to deepen the depth of the capsule network convolution layer, avoiding the degradation of model performance caused by network deepening. The convolution layer with the residual structure has a strong feature extraction capability, which can reduce the difficulty of subsequent calculations and effectively solve the problems of gradient disappearance and gradient explosion. The capsule network effectively constructs the correlation between the box-type substation data, improves the capsule network based on the residual block, and makes full use of the advantages of the two to establish a capsule network fault diagnosis model based on residual improvement, and mines fault information from the box-type substation detection data to perform fault diagnosis. The differential evolution algorithm is used to perform hyperparameter tuning on the model to improve the model training efficiency and performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] Figure 1This is the basic structure diagram of the capsule network;
[0053] Figure 2 It is the residual block structure; Figure 2 (a) and (b) are the structural diagrams of the residual block (two-layer structure) and the residual block (three-layer structure) respectively;
[0054] Figure 3 This is the structure diagram of the improved capsule network based on residual blocks;
[0055] Figure 4 Schematic diagram of the ResNet18 model training process;
[0056] Figure 5 Schematic diagram of the CapsNet model training process;
[0057] Figure 6 Schematic diagram of the Res-Unet model training process;
[0058] Figure 7 Schematic diagram of the training process of the residual-improved capsule network fault diagnosis model (Residual-CapsNet model);
[0059] Figure 8 This is a comparison chart of the accuracy of each model;
[0060] Fig. 9 The following is a comparison chart of the F1 scores of each model. DETAILED DESCRIPTION
[0061] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings of the embodiments of the present invention.
[0062] A capsule network box-type substation fault diagnosis method based on residual improvement includes the following steps:
[0063] Step 1: Process the data set, that is, divide the data set (into training set, validation set, and test set) and perform unique hot encoding on the fault types of the box-type substation for subsequent model training and performance evaluation;
[0064] The data set processing includes the following steps:
[0065] Step 1.1: Divide the input data set into a training set and a validation set in a ratio of 7:3, and use the original data set that has not been derived from data as the test set. The training set is used to train the model; the validation set is used to verify the generalization performance indicators of the model and serve as a basis for judging hyperparameter tuning; the test set is used as the data basis for the final evaluation of the trained model;
[0066] Step 1.2, the one-hot encoding of the fault type of the box-type substation means that when training the residual capsule network diagnosis model, in order to facilitate the calculation of the loss function, the fault type of the box-type substation needs to be processed in the form of one-hot encoding. The results of the one-hot encoding processing are shown in Table 1.
[0067] Table 1 Fault types and codes of box-type substations
[0068] Box-type substation status One-hot encoding Normal operation 1000000 High voltage circuit breaker failure 0100000 High voltage arrester failure 0010000 Dry-type transformer failure 0001000 Low voltage incoming circuit breaker failure 0000100 Low voltage outgoing circuit breaker failure 0000010 Capacitor arrester failure 0000001
[0069] Step 2: Construct a capsule network based on residual improvement; the capsule network based on residual improvement is composed of residual convolutional layers and capsule layers in sequence, and the capsule layers are divided into main capsule layers and digital capsule layers.
[0070] The basic structure of the capsule network mentioned above is as follows Figure 1 As shown in the figure, compared with the convolutional neural network, the capsule network only performs convolution to extract features at the beginning of the network, and then replaces the pooling layer and the fully connected layer by constructing local capsules. These capsules calculate the features extracted by the convolution layer and encapsulate the calculation results into small vector outputs representing the probability and properties of the entity. The dynamic routing mechanism is used to avoid the loss of feature information caused by the pooling operation. Specifically, the dynamic routing algorithm transmits the input vector obtained by the lower layer to the upper layer capsule. For each higher layer capsule that can be routed to, the lower layer capsule unit performs weighted calculation on its own output to obtain the prediction vector. If the prediction vector output by the lower layer capsule has a large scalar product with the output of the upper layer capsule (i.e., a high similarity), the guidance of the upper layer capsule to the lower layer capsule is strengthened through top-down inverse rendering feedback, that is, the coupling coefficient of the capsule is increased, while the coupling coefficients of other capsules are reduced, so as to more effectively process spatial information, ensure that each capsule can learn and diagnose independently, and avoid information interference between capsules.
[0071] The above residual convolution principle is as follows:
[0072] Theoretically, the accuracy of a network model is directly proportional to its network depth. Generally speaking, the more network layers and the deeper the depth, the more versatile the model is and the higher the accuracy is. However, in practical applications, a network that is too deep may cause the accuracy of the model to decrease. This is because as the network continues to deepen, its features become more and more complex, making the model more and more difficult to train. The more typical problems are gradient vanishing and gradient exploding. The gradient is directly related to the training process and generalization of the model. Gradient vanishing or exploding will cause the model to be unable to backpropagate, making it impossible to train the model normally.
[0073] From the perspective of objective function fitting, it is simpler to fit F(x) = H(x-) than to directly fit the objective function H(x). That is, the fitting effect is better when a certain layer of the network is only responsible for fitting the residual work of its own layer. In order to complete the objective function fitting task, it is more effective to adopt the residual learning of H(x) = F(x) + x. The traditional method to achieve this residual learning is to train an identity mapping from the weight layer, which is relatively complicated. The residual block uses the structure of jump connection to achieve efficient residual learning. Jump connection is an identity mapping that crosses one or some layers. Finally, the result of the identity mapping is added to the output of these layers to avoid gradient disappearance or explosion, thereby effectively completing residual learning.
[0074] The residual learning method of the residual structure block skip connection allows the objective function H(x) = F(x) + x to skip one or several layers when the performance decreases, so that the network depth can be increased infinitely under the premise of hardware conditions without performance degradation. The structure of the residual block is as follows Figure 2 shown.
[0075] Figure 2 In the figure: x is the input of the residual block; F(x) is the residual mapping function; Weight Layer is the convolution layer; Relu (Rectified Linear Uni) is the nonlinear activation function, and its specific function form is:
[0076]
[0077] Figure 2 The input x is convolved and the hidden mapping function H(x) is used to calculate the difference to obtain the residual mapping function F(x). At the same time, x is mapped by identity and then added to the residual mapping function F(x) to obtain the output F(x)+x of the residual block. The residual block can convert the original output fitting problem into the residual mapping function F(x) through residual learning. When H(x) = x, the optimal mapping solution for one layer of the network is obtained. In this way, the model only needs to minimize the residual mapping function F(x) to approximate the actual mapping, making it easier to train deep networks.
[0078] The main function of the residual convolution layer in step 2 is to use the residual convolution operation to extract features from the input box-type substation operating parameters and environmental indicator data. Specifically, the residual convolution layer performs a convolution operation on the box-type substation monitoring data through the sliding window of the convolution kernel, extracts the features through weighted calculation and adding bias, and performs residual learning through jump connections. The obtained residual information is passed back to the previous convolution layer, and the feature extraction results are continuously optimized through a dynamic feedback mechanism, thereby achieving better feature expression effects. Compared with ordinary convolution layers, the residual convolution block connects different convolution layers in a residual connection manner, and enough convolution layers can be added to improve the feature extraction capability of the model without worrying about gradient vanishing or gradient exploding problems. The constructed residual-improved capsule network fault diagnosis model is shown in Figure 2. Figure 3 shown.
[0079] The convolution operation of the residual convolution layer is as follows
[0080]
[0081] Where:
[0082] *——Extended convolution operation;
[0083] U i ——is the output of the lth convolutional layer;
[0084] ——The input in the area where the i-th convolution kernel of the l-1th layer is located, that is, the i-th indicator of the data collected by the box-type substation or its features obtained after convolution;
[0085] ——The weight of the i-th convolution kernel in the l-th layer;
[0086] ——The bias of the i-th convolution kernel in the l-th layer;
[0087] f(·) — activation function.
[0088] The capsule layer is divided into the main capsule layer and the digital capsule layer. The structure is as follows:
[0089] The capsule layer is divided into the main capsule layer and the digital capsule layer. Different from the "Scalar in, Scalar out" feature of the convolutional layer, the input and output of the capsule layer are expressed in the form of vectors, that is, the "Vector in, Vector out" strategy. When the capsule unit performs feature detection, the detection probability is used as the length of the vector, and the feature state is used as the direction of the vector. That is, the direction of the capsule unit output vector represents the state encoding of various operating parameters and environmental indicators of the box-type substation, and the vector length represents the probability encoding of the box-type substation expressed by this input being in a certain fault state. The calculation process of the capsule layer is mainly divided into the following two stages:
[0090] The first stage: The output scalar of the residual convolution layer is nonlinearly converted into a prediction vector through vector weighted calculation, which is used as the input vector of the main capsule layer. The calculation formula is:
[0091] U ji =W ji U i
[0092] Where:
[0093] W ji ——The weight matrix of the main capsule layer is formed by generating random numbers using the standard normal initialization method;
[0094] U i ——The output scalar of the lth residual convolution layer;
[0095] U ji ——The jth output vector generated by the weighted nonlinear transformation of the i-th input feature scalar.
[0096] The second stage: Calculate the similarity between the prediction vector output of the main capsule layer and the prediction vector output of the digital capsule layer to selectively transfer the prediction vector of the main capsule layer to the digital capsule layer by adjusting the coupling coefficient between the features, that is, the dynamic routing mechanism. The mathematical calculation formula is as follows:
[0097]
[0098] Where:
[0099] C ij ——The coupling coefficient between the i-th vector and the j-th vector in the main capsule layer;
[0100] p ij ——The bias of the coupling coefficient between the i-th vector and the j-th vector in the main capsule layer;
[0101] S j ——The total input vector of the jth layer is obtained by multiplying the prediction vector of the main capsule layer by the coupling coefficient.
[0102] When C ij When it is 1, all the output information of the main capsule layer is passed to the digital capsule layer. If it is 0, the capsule is discarded.
[0103] In the above two stages, the capsule network continuously updates the coupling coefficient and bias of the digital capsule layer through the iterative execution of the dynamic routing algorithm. In each iteration, the dynamic router calculates the instance-level weights based on the output vector of the main capsule layer and the similarity, overlap and other indicators of all capsules in the digital capsule layer. These instance-level weights are used to adjust the coupling coefficient and bias of the digital capsule layer to more accurately describe the input data. The entire iterative process will continue until the optimal digital capsule weight is found, which is coupled with the main capsule in the best way to achieve better signal transmission and accuracy improvement, thereby improving the accuracy of fault diagnosis of box-type substations.
[0104] Since the capsule layer outputs in the form of vectors, the capsule network does not use the Sigmoid function, SoftMax function or tanh function commonly used in traditional convolutional neural networks as the activation function, but uses a nonlinear vector activation function - Squash to receive the vector output by the capsule layer, and compress its length to less than 1 without changing its direction, thus obtaining the fault classification result. The formula of the Squash function is:
[0105]
[0106] Where:
[0107] ——Input vector S j The reduction factor of
[0108] ——Input vector S j The unit vector of .
[0109] Step 3: Input the training set and validation set into the residual-based improved capsule network fault diagnosis model for training. Use the back propagation algorithm to train the residual capsule network. Through forward excitation propagation and reverse weight update, continuously optimize the number of convolutional layers, convolution kernel size, model learning rate, training batch size, and maximum number of training iterations of the residual convolutional network in the model to obtain a trained box-type substation fault diagnosis model.
[0110] Among them, the steps of inputting the training set and the validation set into the residual-based improved capsule network fault diagnosis model for training are as follows:
[0111] Step 3.1, set hyperparameters: find the best network structure and model hyperparameters on the training set and test set, including the number of convolutional layers of the residual convolutional network, the size of the convolution kernel, the learning rate of the model, the training batch size, and the maximum number of training iterations. Due to the large number of parameter combinations, manual parameter adjustment requires repeated adjustment and training, which is inefficient and difficult to find the global optimal parameter combination. The differential evolution algorithm (DE) is used to automatically optimize the hyperparameters of the residual improved capsule network model;
[0112] Step 3.2, Initialize the model: Initialize the trainable parameters in the model, such as convolution kernel weights, Dropout ratio, etc. It should be noted that this model is a deep learning model, and the all-zero initialization strategy should be avoided to prevent the parameters of each layer of the model from remaining consistent after the back-propagation algorithm is updated, resulting in the model being unable to mine truly valuable information;
[0113] Step 3.3, set the early stopping mechanism. Under the maximum number of iterations (obtained by hyperparameter tuning), if the accuracy of the validation set is improved by less than 0.05% after 30 consecutive iterations of the model, stop training early to save time.
[0114] Step 3.4, input the training set and validation set into the model for training, use the back propagation algorithm to train the residual capsule network, and continuously optimize the parameters in the model through forward excitation propagation and reverse weight update. The input data set is passed to the output layer through the residual convolution layer and the capsule layer, and the loss function value is calculated through the diagnosis results and actual results; when the loss function reaches convergence, the training is completed; and the trained box-type substation fault diagnosis model is obtained;
[0115] The method of automatically optimizing the hyperparameters of the residual improved capsule network model using the differential evolution algorithm is as follows: determine the control parameters and fitness function of the differential evolution algorithm, the control parameters include population size, scaling factor and hybridization probability; randomly generate an initial population; evaluate the initial population, that is, calculate the fitness of individuals in the population; determine whether each individual meets the final optimization conditions or whether the number of evolutionary iterations reaches the maximum, if so, the individual is output as the optimal solution; if not, continue; perform "mutation" and "crossover" operations to obtain an intermediate population; select individuals from the original population and the intermediate population to obtain a new generation of original population; the number of evolutions is +1, and the process is repeated.
[0116] Step 4: Input the test set into the trained box-type substation fault diagnosis model to obtain the box-type substation fault diagnosis method results.
[0117] The following is a specific implementation to further illustrate the box-type substation fault diagnosis method of the present invention. A data set of 1400 data derivatives is used, which is divided into a training set and a validation set in a ratio of 7:3. The original data set that has not been derived from the data is used as a test set to evaluate the model. In order to verify the fault diagnosis effect of the capsule network fault diagnosis model based on residual improvement, the ResNet18 fault diagnosis model, the CapsNet fault diagnosis model and the Res-Unet fault diagnosis model are established as comparison models. The above-divided data sets are used as the input of each model, and comparative experiments are performed to analyze the performance of each model.
[0118] In order to ensure that each model achieves the optimal training effect, the DE optimization algorithm is used to optimize the hyperparameters of each model. The model parameter range settings are shown in Table 2 (for the convenience of expression, "Residual-CapsNet" is used to represent the capsule network fault diagnosis model based on residual improvement):
[0119] Table 2 Hyperparameter range settings for four fault diagnosis models
[0120]
[0121]
[0122] The DE algorithm is used to automatically optimize hyperparameters, and the accuracy of the validation set is used as the evaluation criterion. The optimal hyperparameter combinations of each model are shown in Table 3.
[0123] Table 3 Optimal hyperparameter combinations of four fault diagnosis models
[0124]
[0125] Under the maximum number of iterations, an early stopping mechanism is set to automatically stop training when the model's performance improvement is small or not within a certain number of iterations, saving model training time and improving efficiency.
[0126] In order to quantitatively compare the effects of the four diagnostic models, it is necessary to use evaluation indicators to judge the effects of the models. The fault diagnosis model is essentially a classification model, and the box-type substation fault diagnosis is a multi-classification problem. Therefore, the multi-classification model evaluation indicators are used to judge the model training effect and the final diagnosis effect. For multi-classification models, the confusion matrix is an important tool. It is an n×n matrix, where n represents the number of categories, also known as the target attribute. The confusion matrix of the multi-classification model is shown in Table 4:
[0127] Table 4 Confusion matrix of multi-classification model
[0128]
[0129] Cell in tableij Indicates that the true category is D i , diagnostic category D j Number of instances
[0130] D k Taking the class as an example, calculate the values of TP (True Positive, positive value), FP (False Positive, negative value), TN (True Negative, positive value), and FN (False Negative, negative value):
[0131] TP: the number of samples whose actual type is consistent with the diagnosis result;
[0132] TP(D k )=cell kk
[0133] FP: the number of samples whose actual type is negative but whose diagnosis result is positive, that is, the sum of all corresponding columns except TP value;
[0134]
[0135] TN: The number of samples with the same meaning as the actual category and the diagnosis result; for category D k , all other categories are negative categories, so TN is all non-D k The sum of row and column values;
[0136] TN(D k )=Σcell ij (i≠k,j≠k)
[0137] FN: The number of samples whose actual category is negative and whose diagnosis result is also negative, calculated by the adjacent rows except the TP value.
[0138]
[0139] Based on the confusion matrix values, the following common indicators of classification models can be obtained:
[0140] (1) Precision: The ratio of samples with positive model diagnosis results and positive actual results to all samples with correct model diagnosis results:
[0141]
[0142] (2) Recall: The ratio of samples with positive diagnosis results and positive actual results to samples with positive actual results:
[0143]
[0144] (3) Accuracy: The proportion of correctly classified samples to all samples:
[0145]
[0146] (4) F1 score: It takes into account both precision and recall, and is the harmonic average of the model’s precision and recall:
[0147]
[0148] The comparative analysis of different models is as follows:
[0149] (1) ResNet18 model
[0150] After hyperparameter tuning, the training batch size of the ResNet18 fault diagnosis model is smaller and the learning rate is larger than that of other models. Therefore, the training of the ResNet18 model stops after only 181 iterations. Figure 4 As shown in Table 4, the evaluation results of the ResNet18 model using the above four classification model evaluation indicators are shown.
[0151] Table 4 ResNet18 model training results based on four evaluation indicators
[0152]
[0153] analyze Figure 4 As can be seen from Table 4, the model was volatile in the first 100 iterations, but quickly converged and stabilized, with an accuracy of 89.17% on the test set.
[0154] (2) CapsNet model
[0155] The actual performance of the CapsNet fault diagnosis model under the optimal hyperparameter combination is as follows Figure 5 As shown in Figure 5, the model automatically stopped training at the 198th generation and achieved the optimal training effect. The evaluation results of the CapsNet model using the above four classification model evaluation indicators are shown in Table 5.
[0156] Table 5 CapsNet model training results based on four evaluation indicators
[0157]
[0158] According to the performance of the CapsNet diagnostic model, it can be seen that the training performance in the training set, validation set and test set is slightly better than that of the ResNet18 diagnostic model, but the training process is relatively volatile.
[0159] (3) Res-Unet model
[0160] Since Unet itself adopts a dense connection method, the model structure is more complex than other models and the training is difficult. Therefore, the number of iterations of the Res-Unet model reached 268 times. The iteration process is as follows: Figure 6 The training results are shown in Table 6.
[0161] Table 6 Res-Unet model training results based on four evaluation indicators
[0162]
[0163] (4) Residual-CapsNet model
[0164] Relatively speaking, the calculation process of the dynamic routing mechanism of the capsule network is relatively complicated, so the number of iterations of the Residual-CapsNet model reached 276 times. Figure 7 As shown in the figure, the Residual-CapsNet fault diagnosis model is evaluated using the above four classification model evaluation indicators:
[0165] Table 7 Residual-CapsNet model training results based on four evaluation indicators
[0166]
[0167] from Figure 7 It can be seen that the Residual-CapsNet model converges earlier and has relatively small oscillations, but it takes more iterations to reach the optimal value. Like Res-Unet, the Residual-CapsNet model has a more complex structure than ResNet18 and CapsNet models, has more iterations, and has higher accuracy. Compared with the Res-Unet model, Residual-CapsNet not only has improved accuracy, but also has closer performance in the training set, validation set, and test set, and is more versatile.
[0168] The accuracy and F1 score indicators of the four models are summarized and visualized to more intuitively observe the performance of different models. Figure 8 and Fig. 9. Analyzing the performance of each model, the structure of the ResNet18 model is relatively simple, mainly relying on 17 convolutional layers of residual connections to extract features, and finally obtaining the diagnosis results through the fully connected layer. The accuracy is low, but the training difficulty is small; the internal calculation of the capsule of the CapsNet model is complex, the training difficulty is large, and the accuracy is not ideal; compared with the ResNet18 and CapsNet models, the performance of the Res-Unet model is superior, but due to its relatively complex structure, the training is more time-consuming; the Residual-CapsNet model proposed in the present invention is slightly oscillating in the first two hundred iterations, the convergence speed is relatively fast, the number of iterations is large, but the accuracy is the highest, and it is also the best in terms of the comprehensive performance of the training set, test set and validation set.
[0169] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims
1. A capsule network box-type substation fault diagnosis method based on residual improvement, characterized in that: The method comprises the following steps: Step 1: Process the data set, that is, divide the data set and uniquely encode the fault types of the box-type substation for subsequent model training and performance evaluation. The divided data sets include training set, validation set, and test set; Step 2: constructing a capsule network fault diagnosis model based on residual improvement, wherein the capsule network fault diagnosis model based on residual improvement comprises a residual convolution layer and a capsule layer in sequence, and the capsule layer is divided into a main capsule layer and a digital capsule layer; Step 3: Input the training set and validation set into the residual-based improved capsule network fault diagnosis model for training. Use the back propagation algorithm to train the residual capsule network. Through forward excitation propagation and reverse weight update, continuously optimize the number of convolutional layers, convolution kernel size, model learning rate, training batch size, and maximum number of training iterations of the residual convolutional network in the model to obtain a trained box-type substation fault diagnosis model. Step 4: Input the test set into the trained box-type substation fault diagnosis model to obtain the box-type substation fault diagnosis result.
2. According to the residual improved capsule network box-type substation fault diagnosis method according to claim 1, it is characterized in that: The data set preprocessing method performed in step 1 includes the following specific steps: Step 1.1: Divide the input data set into a training set and a validation set in a ratio of 7:3, and use the original data set that has not been derived from data as the test set. The training set is used to train the model; the validation set is used to verify the generalization performance indicators of the model and serve as a basis for judging hyperparameter tuning; the test set is used as the data basis for the final evaluation of the trained model; Step 1.2, the one-hot encoding of the fault type of the box-type substation means that when training the residual capsule network diagnosis model, the fault type of the box-type substation is processed in the form of one-hot encoding.
3. According to the residual improved capsule network box-type substation fault diagnosis method of claim 1, it is characterized by: The specific method for constructing the capsule network fault diagnosis model based on residual improvement in step 2 is: introducing a residual module into the convolutional layer of the capsule network to establish a capsule network fault diagnosis model based on residual improvement.
4. According to the residual improved capsule network box-type substation fault diagnosis method of claim 1, it is characterized by: The residual convolution layer described in step 2 uses the residual convolution operation to extract features from the input box-type substation operating parameters and environmental indicator data, that is, the residual convolution layer performs a convolution operation on the box-type substation monitoring data through the sliding window of the convolution kernel, extracts the features through weighted calculation and adding bias, and performs residual learning through skip connections, and transmits the obtained residual information back to the previous convolution layer, and continuously optimizes the feature extraction results through a dynamic feedback mechanism; The convolution operation of the residual convolution layer is as follows: Where: *——Extended convolution operation; U i ——is the output of the lth convolutional layer; ——The input in the area where the i-th convolution kernel of the l-1th layer is located, that is, the i-th indicator of the data collected by the box-type substation or its features obtained after convolution; ——The weight of the i-th convolution kernel in the l-th layer; ——The bias of the i-th convolution kernel in the l-th layer; f(·) — activation function.
5. According to the residual improved capsule network box-type substation fault diagnosis method of claim 4, it is characterized by: The capsule layer described in step 2 is divided into a main capsule layer and a digital capsule layer, and the structure is as follows: The input and output of the capsule layer are expressed in the form of vectors. When the capsule unit in the capsule layer performs feature detection, the detection probability is used as the length of the vector, and the feature state is used as the direction of the vector. That is, the direction of the capsule unit output vector represents the state coding of various operating parameters and environmental indicators of the box-type substation, and the vector length represents the probability coding of the box-type substation expressed by this input being in a certain fault state. The calculation method of the capsule layer mainly includes the following two steps: Step 1: The output scalar of the residual convolution layer is nonlinearly converted into a prediction vector through vector weighted calculation, which is used as the input vector of the main capsule layer. The calculation formula is: U ji =W ji U i Where: W ji ——The weight matrix of the main capsule layer is formed by generating random numbers using the standard normal initialization method; U i ——The output scalar of the lth residual convolution layer; U ji ——The jth output vector generated by the weighted nonlinear transformation of the i-th input feature scalar; Step 2: By calculating the similarity between the prediction vector output of the main capsule layer and the prediction vector output of the digital capsule layer, the coupling coefficient between the features is adjusted, and the prediction vector of the main capsule layer is selectively transmitted to the digital capsule layer, that is, the dynamic routing mechanism. The mathematical calculation formula is as follows: Where: C ij ——The coupling coefficient between the i-th vector and the j-th vector in the main capsule layer; p ij ——The bias of the coupling coefficient between the i-th vector and the j-th vector in the main capsule layer; S j ——The total input vector of the jth layer is obtained by multiplying the prediction vector of the main capsule layer by the coupling coefficient; When C ij When it is 1, all the output information of the main capsule layer is passed to the digital capsule layer. If it is 0, the capsule is discarded. In steps 1 and 2, the capsule network continuously updates the coupling coefficient and bias of the digital capsule layer through the iterative execution of the dynamic routing algorithm. In each iteration, the dynamic router calculates the instance-level weight based on the output vector of the main capsule layer and the similarity and overlap indicators of all capsules in the digital capsule layer. The instance-level weight is used to adjust the coupling coefficient and bias of the digital capsule layer. The entire iterative process will continue until the optimal digital capsule weight is found and coupled with the main capsule in the best way. The capsule network uses a nonlinear vector activation function to receive the vector output by the capsule layer. The nonlinear vector activation function is the Squash function. Without changing its direction, its length is compressed to less than 1, and the fault classification result is obtained. The formula of the Squash function is: Where: ——Input vector S j The reduction factor of ——Input vector S j The unit vector of .
6. The method for fault diagnosis of capsule network box-type substation based on residual improvement according to claim 1 is characterized in that: The steps of inputting the training set and the validation set into the residual-based improved capsule network fault diagnosis model for training described in step 3 are as follows: Step 3.1, setting hyperparameters: Find the best network structure and model hyperparameters on the training set and test set, including the number of convolutional layers of the residual convolutional network, the convolution kernel size, the model learning rate, the training batch size, and the maximum number of training iterations; use the differential evolution algorithm to automatically optimize the hyperparameters of the residual-based improved capsule network fault diagnosis model; Step 3.2, initialize the model: initialize the training parameters in the model; Step 3.3, set the early stopping mechanism. Under the maximum number of iterations, if the accuracy of the validation set is improved by less than 0.05% after 30 consecutive iterations of the model, stop the training early. In step 3.4, the training set and validation set are input into the model for training. The residual capsule network is trained using the back-propagation algorithm. The parameters in the model are continuously optimized through forward excitation propagation and reverse weight update. The input data set is passed to the output layer through the residual convolution layer and the capsule layer. The loss function value is calculated through the diagnosis results and the actual results. When the loss function reaches convergence, the training is completed. The trained box-type substation fault diagnosis model is obtained.
7. The method for fault diagnosis of capsule network box-type substation based on residual improvement according to claim 6 is characterized in that: The method described in step 3.1 for automatically optimizing the hyperparameters of the residual-improved capsule network fault diagnosis model using the differential evolution algorithm is as follows: determine the control parameters and fitness function of the differential evolution algorithm, the control parameters include population size, scaling factor and hybridization probability; randomly generate an initial population; evaluate the initial population, that is, calculate the fitness of the individuals in the population; determine whether each individual meets the final optimization conditions or whether the number of evolutionary iterations reaches the maximum, if so, the individual is output as the optimal solution; if not, continue to automatically optimize; after optimizing, perform "mutation" and "crossover" operations to obtain an intermediate population; select individuals from the original population and the intermediate population to obtain a new generation of original population; the number of evolutions is +1, and the process is repeated until the model converges.
Citation Information
Cited By
Underwater propeller fault detection method and device and computer readable storage medium
CN120277431A
Monitoring method of intelligent rail vehicle hinge device system
CN120744669A
A monitoring method for a smart track vehicle articulation system
CN120744669B
