System fault diagnosis method based on attention mechanism and deep residual network

Abstract

The present invention discloses a system fault diagnosis method based on attention mechanism and deep residual network. In this method, the optimal number of network layers and convolution kernels are obtained by using artificial bee colony algorithm to design network structure, so as to give full play to the feature extraction capability of convolution operation for large industrial system time series, multi-dimensional and noise interference data. At the same time, the attention mechanism based on channel attention and spatial attention is added between residual blocks, so that the network can automatically learn the importance of different channels and the importance of different pixels of input data, so as to accelerate the model training convergence speed and fault diagnosis accuracy. In this method, the state variable time series data of the steam-water separation reheating system is input into the model, and a higher fault diagnosis accuracy is obtained under noise interference.

Description

Technical Field

[0001] The present invention relates to the technical field of fault diagnosis of nuclear power steam-water separation reheating systems, and in particular to a fault diagnosis algorithm based on an attention mechanism and a deep residual network. Background Art

[0002] The Moisture Separator and Reheater (MSR) system is the main subsystem of the nuclear power secondary circuit system. Its safe and efficient operation is of great significance to the safety and economy of nuclear power units.

[0003] However, due to the increase in the number of nuclear power units under construction and the large-scale grid connection of intermittent renewable energy, nuclear power units need to participate deeply in peak and frequency regulation, resulting in the steam-water separation and reheating system being prone to a variety of typical faults under time-varying operating conditions.

[0004] Therefore, accurate diagnosis of typical faults in the steam-water separation reheat system is of great significance to the safe and stable operation of nuclear power units under time-varying conditions.

[0005] Current fault diagnosis technologies are mainly divided into analytical model-based methods, empirical knowledge-based methods and data-driven methods.

[0006] After analyzing the process system based on the analytical model method, a functional relationship consisting of mathematical formulas that can describe the system function is abstracted, thereby obtaining the analytical redundancy of the process system. The analytical redundancy produces the difference between the measured value and the model estimate, that is, the physical residual. The key to fault detection is to analyze the residual characteristics to identify, locate and isolate the fault. However, the model-based method relies on the accuracy of the model. Only when the model is accurate can the fault be effectively identified. However, whether the model is established through analysis or experimental data, it often cannot accurately reflect the steady-state and dynamic characteristics of the system. Therefore, this method has great limitations.

[0007] The method based on empirical knowledge is suitable for complex industrial systems that people have a deep understanding of, especially fault identification in the nuclear power field. The classic method based on empirical knowledge is the expert system. However, for complex systems such as nuclear power plants, the knowledge base currently established is still insufficient to reflect the main accident characteristics, so the development of expert systems depends more on the development of system understanding.

[0008] With the rapid development of random computers and the Internet, the operating data of actual systems can be more easily saved and mined, so data-driven methods are widely used. However, when data-driven methods are applied to fault identification of complex industrial systems, there are some defects, such as large amount of calculation, poor real-time performance, and high requirements on the amount of data.

[0009] In recent years, data-driven methods represented by deep learning have developed rapidly thanks to the improvement of computer computing power, big data and advancement of computing models, and have broad prospects in fault diagnosis applications.

[0010] However, the fault diagnosis methods applied to complex industrial systems such as steam-water separation and reheating systems are subject to the following limitations in practical applications: noise interference, coupling of data time series dimension and variable dimension, which makes it difficult to implement the fault diagnosis model for steam-water separation and reheating systems. Summary of the invention

[0011] Technical problem to be solved by the present invention: The purpose of the present invention is to solve the problem that the data-driven fault diagnosis method is difficult to apply to the fault diagnosis of the actual steam-water separation reheating system due to noise interference and coupling of data time series dimension and variable dimension.

[0012] The present invention provides the following technical solution: The present invention discloses a system fault diagnosis method based on an attention mechanism and a deep residual network, comprising the following steps:

[0013] S1. Use the artificial bee colony heuristic optimization algorithm to optimize the number of network residual blocks and the number of convolution kernels in each internal layer, in order to obtain the Pareto optimal network structure that meets the maximum fault diagnosis accuracy and minimum network calculation parameters;

[0014] S2. Add channel attention mechanism and spatial attention mechanism between residual blocks and before and after to improve the network training convergence speed and fault diagnosis accuracy;

[0015] S3. Preprocess the data by performing sliding window, edge zero padding, and standardization operations;

[0016] S4. Input the two-dimensional image data into the network for training, and finally obtain a trained network model, and classify the state data of the steam-water separation and reheating system input into the network model, so as to diagnose the system state.

[0017] Furthermore, in step S1, the network structure is designed using an artificial bee colony heuristic optimization algorithm, including the following steps:

[0018] The number of network residual blocks and the number of convolution kernels in each layer of the network are listed as design variables;

[0019] The maximum fault diagnosis accuracy and the minimum total amount of network calculation parameters after 20 cycles of network training are listed as target variables;

[0020] The training cycle is set to 100 times, and then the artificial bee colony algorithm is used to automatically optimize the Pareto optimal solution set of fault diagnosis accuracy and the total number of network calculation parameters. The Pareto optimal solution set corresponds to the optimal number of network residual blocks and the number of convolution kernels in each layer of the network, that is, the optimal network structure.

[0021] Furthermore, in step S2, the channel attention mechanism includes the following steps:

[0022] Sa1. Compression, the compression formula is:

[0023]

[0024] Among them, u c (v,w) represents the pixel (v,w) in the feature map of channel c; z c Represents the cth global feature point; the input feature map size is H×W×C.

[0025] Sa2. Activation, the activation formula is:

[0026] s c =σ(W2δ(W1z c ))

[0027] Where σ represents the sigmoid activation function; δ represents the Relu activation function; W1 and W2 represent the weights of the fully connected layer; s c Represents the output value after activation;

[0028] Sa3. Product, the product formula is:

[0029]

[0030] In the formula, Represents the weighted feature map.

[0031] Furthermore, in step S2, the spatial attention mechanism includes the following steps:

[0032] Sb1. Pooling operation, that is, performing channel-based global maximum pooling and global average pooling on the feature map with an input size of H×W×C, thereby obtaining two feature planes with a size of H×W×1, and then channel-joining the two feature planes to finally obtain a feature map with a size of H×W×2;

[0033] Sb2. Dimensionality reduction operation, that is, reducing the feature map of size H×W×2 to a feature map of size H×W×1 through a 7×7 convolution kernel;

[0034] Sb3. Activation operation, that is, using the sigmoid activation function to generate weights at different positions, so that the model can distinguish the importance of different positions;

[0035] Sb4. Product operation, that is, assigning the weight coefficients of each position learned in the previous step to the initial feature map, so that the model has the ability to distinguish features at different positions;

[0036] Sb5. The spatial attention mechanism can be described as:

[0037] M s (F) = σ(f 7×7 ([AvgPool(F); MaxPool(F)]))

[0038] Where σ represents the sigmoid activation function; f 7×7 represents a 7×7 convolution operation; AvgPool(F) and MaxPool(F) represent global average pooling and global maximum pooling, respectively.

[0039] Furthermore, a sliding window method is used to collect two-dimensional image data of a fixed time length, and a sliding window operation is performed on the steam-water separation reheating system status data on the time axis, thereby converting the time series data into time series-variable two-dimensional image data, so that the system status data can be convolved.

[0040] Furthermore, the edge zero padding includes padding the edges of the time-variable two-dimensional image data with pixel points whose values ​​are 0, thereby obtaining time-variable two-dimensional image data of a fixed variable length, and finally saving it as two-dimensional image data with a resolution of 600×600, so that the data can be saved as input data with a unified format to facilitate fault diagnosis operations.

[0041] Furthermore, the standardization process includes standardizing the two-dimensional image data obtained by the edge zero-filling operation, thereby removing the influence of the dimension on different variables;

[0042] The standardization process includes expanding the input image data according to the variable dimension, calculating the mean μ and standard deviation σ of different variables at all time points, and subtracting the mean from the original input data X and dividing it by the standard deviation to obtain the standardized data X. * ;

[0043]

[0044] In the formula, μ is the mean of all sample data; σ is the standard deviation of all sample data.

[0045] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0046] 1. The present invention can effectively perform fault diagnosis on time and variable coupled input data containing noise interference, and ensure a high accuracy rate, and can be applied to the fault diagnosis of nuclear power steam-water separation reheating system; the artificial bee colony heuristic intelligent algorithm is used to optimize the network structure, and a network structure that satisfies the Pareto optimal solution set with higher network fault diagnosis accuracy and fewer network calculation parameters can be obtained, which can maximize the network performance and obtain the highest possible fault diagnosis accuracy rate under limited computing resources.

[0047] 2. The present invention adds a channel attention mechanism and a spatial attention mechanism between residual blocks, wherein the channel attention mechanism can speed up the model to extract effective channel features, and the spatial attention mechanism can speed up the model to extract effective spatial features of input image data.

[0048] 3. The present invention adopts data preprocessing operations such as sliding window, edge zero filling and standardization processing, which can obtain two-dimensional image input data in a unified data format, and can facilitate the conversion of actual steam-water separation and reheating system data into model input data. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is the overall flow chart of the present invention;

[0050] Figure 2 It is a flow chart of the artificial bee colony heuristic optimization algorithm in the present invention;

[0051] Figure 3 Schematic diagram of the channel attention mechanism in the present invention;

[0052] Figure 4 Schematic diagram of the spatial attention mechanism in the present invention;

[0053] Figure 5 It is a schematic diagram of the sliding window in the present invention;

[0054] Figure 6 Schematic diagram of edge zero padding in the present invention;

[0055] Figure 7 This is a diagram of the fault diagnosis results of the model in the present invention. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solutions and advantages of the present invention more clear, the technical solutions of the present invention will be described more clearly and completely below in conjunction with the accompanying drawings in the examples of the present invention.

[0057] like Figure 1 As shown in the figure, this paper proposes a fault diagnosis model based on attention mechanism and deep residual network, referred to as A-DRSN, which is a flow chart of the fault diagnosis method. The specific process of algorithm design mainly includes the following steps:

[0058] 1. Design network structure using artificial bee colony heuristic optimization algorithm

[0059] The number of network residual blocks and the number of convolution kernels in each layer of the network are listed as design variables;

[0060] The maximum fault diagnosis accuracy and the minimum total amount of network calculation parameters after 20 cycles of network training are listed as target variables;

[0061] The training cycle is set to 100 times, and then the artificial bee colony algorithm is used to automatically optimize the Pareto optimal solution set of fault diagnosis accuracy and the total number of network calculation parameters. The Pareto optimal solution set corresponds to the optimal number of network residual blocks and the number of convolution kernels in each layer of the network, that is, the optimal network structure.

[0062] 2. Add a channel attention mechanism and a spatial attention mechanism between the two residual blocks, and connect the channel attention mechanism and the spatial attention mechanism in series. The channel attention mechanism obtains the most important convolution kernel parameters in different convolution kernels through compression, activation and product steps, and accelerates the training speed and model accuracy; the spatial attention mechanism obtains the most important pixel point parameters in the feature path corresponding to the same convolution kernel through pooling, dimensionality reduction, activation and convolution, thereby accelerating the network to extract important spatial features, so as to accelerate network training and improve model accuracy.

[0063] 2.1 Adding Attention Mechanism

[0064] Channel attention mechanism such as Figure 3 As shown in Figure 1, the channel attention mechanism is divided into three steps: compression, activation, and product. The compression operation can be described as:

[0065]

[0066] where u c (v,w) represents the pixel (v,w) in the feature map of channel c; z c Represents the cth global feature point; the input feature map size is H×W×C.

[0067] The activation operation can be described as:

[0068] s c =σ(W2δ(W1z c ))

[0069] Where σ represents the sigmoid activation function; δ represents the Relu activation function; W1 and W2 represent the weights of the fully connected layer; s c Indicates the output value after activation.

[0070] The product operation can be described as:

[0071]

[0072] In the formula, Represents the weighted feature map.

[0073] 2.2 Spatial Attention Mechanism Figure 4 As shown, it is divided into four operations: pooling, dimensionality reduction, activation and product;

[0074] The first is the pooling operation, which is to perform channel-based global maximum pooling and global average pooling on the feature map with an input size of H×W×C to obtain two feature planes with a size of H×W×1. These two feature planes are then channel-joined to finally obtain a feature map with a size of H×W×2.

[0075] The second step is the dimensionality reduction operation, which is to reduce the feature map of size H×W×2 to a feature map of size H×W×1 through a 7×7 convolution kernel.

[0076] The third step is the activation operation, which uses the sigmoid activation function to generate weights for different positions, so that the model can distinguish the importance of different positions.

[0077] The fourth step is the product operation, which assigns the weight coefficients of each position learned in the previous step to the initial feature map, so that the model has the ability to distinguish features at different positions.

[0078] Finally, the spatial attention mechanism can be described as:

[0079] M s (F) = σ(f 7×7 ([AvgPool(F); MaxPool(F)]))

[0080] Where σ represents the sigmoid activation function; f 7×7 represents a 7×7 convolution operation; AvgPool(F) and MaxPool(F) represent global average pooling and global maximum pooling, respectively.

[0081] 3. Preprocess the data by performing sliding window, edge zero padding, and standardization operations;

[0082] 3.1 Use the sliding window method to collect two-dimensional image data of a fixed time length. The sliding window is as follows: Figure 5 As shown; the steam-water separation and reheating system status data is subjected to a sliding window operation on the time axis, thereby converting the time series data into time series-variable two-dimensional image data, so that the system status data can be convolved.

[0083] 3.2 Use edge zero padding to convert the input data size into a unified data format of 600×600, such as Figure 6 As shown; edge zero padding includes padding the edge of the time series-variable two-dimensional image data with a pixel value of 0, thereby obtaining time series-variable two-dimensional image data of a fixed variable length, and finally saving it as a two-dimensional image data with a resolution of 600×600, so that the data can be saved as input data with a unified format to facilitate fault diagnosis operations

[0084] The data is standardized to eliminate the dimension effect and speed up the network convergence. The standardization operation is as follows:

[0085]

[0086] In the formula, μ is the mean of all sample data; σ is the standard deviation of all sample data.

[0087] 4. Use the preprocessed data as the input training data of the deep residual network with the attention mechanism added, use Pytorch to build the network model, build a network model after artificial bee colony optimization and adding the attention mechanism, and input the preprocessed training data into the network model to train the parameters of the network model; and observe the changes in the network fault diagnosis accuracy and loss value at the same time to prevent overfitting due to too many training times. When the accuracy of the network training set and test set meets the requirements, the network training can be stopped.

[0088] During the training process, the Adam optimization algorithm is used to update the network parameters, and the softmax classifier is used to output the fault diagnosis results.

[0089] 5. The actual state data of the steam-water separation and reheating system is used as input data and input into the trained network model to test the performance of the algorithm.

[0090] The deep network fault diagnosis model based on the attention mechanism proposed in the present invention has a high accuracy rate for faults in the steam-water separation and reheating system. In normal noise-free data, the accuracy rate reaches more than 99%. At the same time, this method also has good recognition accuracy for data containing noise signals.

[0091] from Figure 7 It can be seen that in the case of low noise, the deep residual network DRSN without adding the attention mechanism, the traditional convolutional neural network VGG, and the multi-layer perceptron MLP all have good accuracy. However, with the increase of noise signals in the data, the accuracy of the other methods has dropped significantly, while the method proposed in the present invention still has an accuracy of more than 90%. Therefore, it can be concluded that the method proposed in the present invention has a very good diagnostic accuracy for high-noise data.

[0092] The above specific implementation methods are only for illustrating the technical concept and structural features of the present invention, and the purpose is to enable relevant persons familiar with this technology to implement it. However, the above content does not limit the protection scope of the present invention. Any equivalent changes or modifications made according to the spirit of the present invention should fall within the protection scope of the present invention.

Claims

1. A system fault diagnosis method based on attention mechanism and deep residual network, characterized by: The following steps are included: S1. Use the artificial bee colony heuristic optimization algorithm to optimize the number of network residual blocks and the number of convolution kernels in each internal layer to obtain the Pareto optimal network structure that meets the maximum fault diagnosis accuracy and minimum network calculation parameters; S2. Add channel attention mechanism and spatial attention mechanism between residual blocks and before and after to improve the network training convergence speed and fault diagnosis accuracy; S3. Preprocess the data by performing sliding window, edge zero padding, and standardization operations; S4. Input the two-dimensional image data into the network for training, and finally obtain the trained network model, and classify the state data of the steam-water separation reheating system input into the network model, so as to diagnose the system state; In step S1, the network structure is designed using an artificial bee colony heuristic optimization algorithm, including the following steps: The number of network residual blocks and the number of convolution kernels in each layer of the network are listed as design variables; The maximum fault diagnosis accuracy and the minimum total amount of network calculation parameters after 20 cycles of network training are listed as target variables; The training cycle is set to 100 times, and then the artificial bee colony algorithm is used to automatically optimize the Pareto optimal solution set of fault diagnosis accuracy and the total number of network calculation parameters. The Pareto optimal solution set corresponds to the optimal number of network residual blocks and the number of convolution kernels in each layer of the network, that is, the optimal network structure. In step S2, the channel attention mechanism includes the following steps: Sa1. Compression, the compression formula is: Among them, u c (v,w) represents the pixel (v,w) in the feature map of channel c; z c represents the cth global feature point; the input feature map size is H×W×C; Sa2. Activation, the activation formula is: s c =σ(W2δ(W1z c )) Where σ represents the sigmoid activation function; δ represents the Relu activation function; W1 and W2 represent the weights of the fully connected layer; s c Represents the output value after activation; Sa3. Product, the product formula is: In the formula, Represents the weighted feature map; In step S2, the spatial attention mechanism includes the following steps: Sb1. Pooling operation, that is, performing channel-based global maximum pooling and global average pooling on the feature map with an input size of H×W×C, thereby obtaining two feature planes with a size of H×W×1, and then channel-joining the two feature planes to finally obtain a feature map with a size of H×W×2; Sb2. Dimensionality reduction operation, that is, reducing the feature map of size H×W×2 to a feature map of size H×W×1 through a 7×7 convolution kernel; Sb3. Activation operation, that is, using the sigmoid activation function to generate weights at different positions, so that the model can distinguish the importance of different positions; Sb4. Product operation, that is, assigning the weight coefficients of each position learned in the previous step to the initial feature map, so that the model has the ability to distinguish features at different positions; Sb5. The spatial attention mechanism is described as: M s (F)=σ(f 7×7 ([AvgPool(F);MaxPool(F)])) Where σ represents the sigmoid activation function; f 7×7 represents a 7×7 convolution operation; AvgPool(F) and MaxPool(F) represent global average pooling and global maximum pooling, respectively.

2. The system fault diagnosis method based on attention mechanism and deep residual network according to claim 1, characterized in that: The sliding window method is used to collect two-dimensional image data of a fixed time length, and the sliding window operation on the time axis is performed on the steam-water separation reheating system state data, thereby converting the time series data into time series-variable two-dimensional image data, so that the system state data can be convolved.

3. The system fault diagnosis method based on attention mechanism and deep residual network according to claim 2, characterized in that: The edge zero padding includes padding the edges of the time-variable two-dimensional image data with pixel points whose values ​​are 0, thereby obtaining time-variable two-dimensional image data of a fixed variable length, and finally saving it as two-dimensional image data with a resolution of 600×600, so that the data can be saved as input data with a unified format to facilitate fault diagnosis operations.

4. The system fault diagnosis method based on attention mechanism and deep residual network according to claim 3, characterized in that: The standardization process includes standardizing the two-dimensional image data obtained by the edge zero-filling operation, thereby removing the influence of the dimension on different variables; The standardization process includes expanding the input image data according to the variable dimension, calculating the mean μ and standard deviation σ of different variables at all time points, and subtracting the mean from the original input data X and dividing it by the standard deviation to obtain the standardized data X. * ; In the formula, μ is the mean of all sample data; σ is the standard deviation of all sample data.

Images