CNN-Transformer-based fuel cell fault diagnosis method

Abstract

The invention discloses a fuel cell fault diagnosis method based on a CNN (convolutional neural network)-Transform. The method comprises the following steps: collecting original data; performing data preprocessing and feature importance evaluation selection on the original data; dividing the data samples selected through feature importance evaluation into a training set and a test set; respectively inputting the training set and the test set into a CNN-Transform model for feature extraction, and outputting a new training set and a new test set; constructing double full connection layers, and outputting three operation states of fuel cell normality, membrane dryness and hydrogen leakage predicted by the corresponding CNN-Transformer model to obtain a prediction score of each group of data; the optimal parameter configuration of the CNN-Transform model is obtained, and the optimal parameter configuration of the CNN-Transform model is obtained; and substituting the new test set into the CNN-Transform model after parameter optimization, and outputting a fuel cell fault diagnosis result. Through the CNN-Transformer algorithm provided by the invention, the three operation states of the fuel cell, i.e., the normal state, the membrane drying state and the hydrogen leakage state, can be better distinguished, and the problems of insufficient robustness and poor generalization ability of a single model are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of fuel cell fault diagnosis, and particularly to a fuel cell fault diagnosis method based on CNN-Transformer. Background Art

[0002] As a green and efficient energy conversion technology, fuel cells play an important role in the implementation of the dual-carbon strategy. Proton Exchange Membrane Fuel Cell (PEMFC), as one of the most widely studied and mature types of fuel cell technology, exhibits great application potential in multiple fields including transportation and energy supply due to its advantages such as high energy conversion efficiency, superior environmental protection performance, strong power characteristics, and good adaptability to various environments.

[0003] However, as a multi-physical field coupled non-linear complex system, fuel cells involve multi-physical coupling phenomena such as heat, electricity, and fluid during operation. Component aging, environmental changes, and improper control of the battery may all lead to system failures. If the failures cannot be detected and cleared in time, it will further cause system performance degradation, shortened service life, and even more serious safety accidents. Fault diagnosis can accurately and quickly identify faults and play an important role in preventing fault spread and alleviating the impact of faults.

[0004] However, in traditional data-driven fault diagnosis research, most algorithms show good diagnostic effects for the single fault studied. But when studying extended fault types, the accuracy rate of a single algorithm is low, the reliability is poor, and problems of insufficient robustness and weak generalization ability occur. Summary of the Invention

[0005] Aiming at the deficiencies of the prior art, the present invention proposes a fuel cell fault diagnosis method based on CNN-Transformer, which not only overcomes the problems of insufficient robustness and weak generalization ability of a single model, but also can better distinguish the three operating states of normal, membrane drying, and hydrogen leakage of fuel cells, and effectively judge fuel cell faults.

[0006] To achieve the above object, the present invention designs a fuel cell fault diagnosis method based on CNN-Transformer, which is characterized in that it includes the following steps:

[0007] S1) Collect original data, where the original data is the state variables related to time characteristics during the operation of the fuel cell;

[0008] The state variables include the anode / cathode inlet and outlet pressures of the stack, the cathode inlet and outlet temperatures, the cathode air excess coefficient, the inlet water pressure, the inlet water flow rate, the load current, the cooling water inlet temperature, and the anode / cathode inlet air flow rates;

[0009] S2) Perform data preprocessing on the original data. The data preprocessing includes normalizing the original data and then evaluating and selecting the feature importance of the normalized data, and taking the selected important feature variables as the fuel cell fault diagnosis variables;

[0010] S3) Divide the data samples selected through feature importance evaluation into a training set and a test set;

[0011] S4) Input the training set and the test set into the CNN-Transformer model respectively for feature extraction; the CNN model transforms the data from the original state into a form that is concentrated and contains multi-dimensional features; the Transformer model models the global dependence relationship of the output results of the CNN model through the self-attention mechanism, captures long-distance interaction information and complex context structures, and outputs a new training set and a new test set;

[0012] S5) Construct a double fully-connected layer to enhance the non-linear expression ability of the CNN-Transformer model, map the multi-dimensional vectors output by the Transformer model into three-dimensional outputs, corresponding to the three operating states of normal, membrane drying, and hydrogen leakage predicted by the CNN-Transformer model for the fuel cell, and obtain the prediction scores for each group of data;

[0013] S6) Measure the error between the prediction scores of the CNN-Transformer model and the true labels by calculating the loss function, use the backpropagation algorithm to calculate the gradient of the loss with respect to the parameters of the CNN-Transformer model, indicate the direction and magnitude of the parameter adjustment through the gradient, and gradually reduce the loss function value through continuous iterative optimization to achieve the optimal parameter configuration;

[0014] S7) Input the new test set into the CNN-Transformer model with optimized parameters, and output the fuel cell fault diagnosis result, that is, the fuel cell is in a normal state, or a membrane drying state, or a hydrogen leakage state.

[0015] Further, in S2), the following formula is used for normalizing the original data

[0016]

[0017] In the formula,

[0018] x i *is the value after the data sample value has been normalized,

[0019] x i is the data sample value,

[0020] x imax is the maximum value of the sample value x i and

[0021] x imin is the minimum value of the sample value x i and

[0022] Furthermore, in S2), the Gini impurity in the random forest algorithm is used as the feature importance evaluation, and the calculation formula of the Gini impurity is as follows

[0023]

[0024] where,

[0025] G m is the Gini impurity,

[0026] K is the category included in the data sample,

[0027] p m,k is the probability that the data sample at node m belongs to the k-th category.

[0028] Further, in S3), the division ratio of the training set and the test set is set to 7:3.

[0029] Furthermore, in S4), the CNN model includes 3 convolutional layers and 2 pooling layers, and the Transformer model includes two encoder layers, and each encoder layer includes an attention layer, 2 residual connections and normalization layers, and a feed-forward network layer.

[0030] Furthermore, in S4), the specific steps for the CNN model to transform the data from the original state into a form that is concentrated and contains multi-dimensional features are as follows,

[0031] S51) First, after preprocessing and reshaping the data, it is input with a single channel by the first convolutional layer, 32 feature channels are extracted, and after being activated by the RELU function, it is then pooled by the pooling layer;

[0032] S52) Then, repeat the convolution and pooling operations in step S51) once to further expand the number of feature channels to 64 channels;

[0033] S53) Finally, the third convolutional layer generates an output of 128 channels based on 64 channels.

[0034] Furthermore, in S4), the specific steps for the Transformer model to output the new training set and the new test set are as follows:

[0035] S54) First, flatten the output obtained by the CNN model to get data in the input form suitable for the Transformer model, and then perform positional encoding on the data.

[0036] S55) Input the data after positional encoding into the attention layer to extract global features. After evaluating the data scale, set the number of attention heads in the attention layer to 1.

[0037] S56) Perform residual connection and layer normalization on the output obtained by the attention layer. Residual connection is used to alleviate gradient disappearance and feature enhancement, and layer normalization is used to keep the input distribution of each layer in the model relatively stable, reduce the risk of gradient disappearance or gradient explosion, and accelerate the convergence speed of the model.

[0038] S57) Input the data after residual connection and layer normalization into the feed-forward network layer for feature extraction and dimensionality transformation, and capture the non-linear associations between input features.

[0039] S58) Perform another residual connection and layer normalization operation on the output of the feed-forward network layer to obtain the output of the first encoder layer.

[0040] S59) Overall, repeat the process from step S54) to step S58) to finally obtain the new training set and the new test set output by the Transformer layer.

[0041] Furthermore, in S5), the fully-connected layer includes a first fully-connected layer and a second fully-connected layer. The first fully-connected layer maps the 128-dimensional vector output by the Transformer model to a 100-dimensional vector and enhances the non-linear expression ability of the model through the ReLU activation function. The second fully-connected layer maps the 100-dimensional vector output by the first fully-connected layer to a 3-dimensional output, corresponding to the three operating states of normal fuel cell, membrane drying, and hydrogen leakage predicted respectively, to obtain the prediction scores for each group of data.

[0042] The advantages of the present invention are as follows:

[0043] 1. The present invention first uses the random forest algorithm to select the feature importance of the data set, divides the data after feature importance selection into a training set and a test set according to a certain ratio, performs feature extraction using the CNN-Transformer model and then classifies. After verification, this method has high accuracy and reliability, and can provide a new idea for fuel cell fault diagnosis.

[0044] 2. The proposed CNN-Transformer algorithm of the present invention makes full use of the local feature extraction ability of CNN and the global feature modeling advantage of Transformer, significantly improving the ability of the fault diagnosis model to distinguish the operating states of fuel cells (normal, membrane drying, and hydrogen leakage); at the same time, the model can overcome the problems of insufficient robustness and weak generalization ability of a single model and can effectively judge fuel cell faults.

[0045] The fuel cell fault diagnosis method based on CNN-Transformer of the present invention, through the proposed CNN-Transformer algorithm, can not only better distinguish the three operating states of normal, membrane drying, and hydrogen leakage of fuel cells, effectively judge fuel cell faults, but also overcome the problems of insufficient robustness and weak generalization ability of a single model. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is a flowchart of the present invention;

[0047] Figure 2 is the feature importance evaluation result in the embodiment of the present invention;

[0048] Figure 3 is the CNN model structure diagram in the present invention;

[0049] Figure 4 is the Transformer model structure diagram in the present invention;

[0050] Figure 5a is the fault classification result in the embodiment of the present invention;

[0051] Figure 5b is the confusion matrix in the embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0052] The following further describes the present invention in detail with reference to the accompanying drawings and specific embodiments.

[0053] As Figure 1 shown, a fuel cell fault diagnosis method based on CNN-Transformer of the present invention includes the following steps:

[0054] S1) Collect raw data, where the raw data is the state variables of the fuel cell during operation with respect to time features.

[0055] Specifically, the state variables include the anode / cathode inlet and outlet pressures of the stack, the cathode inlet and outlet temperatures, the cathode air excess coefficient, the inlet water pressure, the inlet water flow rate, the load current, the cooling water inlet temperature, and the anode / cathode inlet air flow rates.

[0056] The original data collected in this embodiment is from a fuel cell system with a dual-stack structure. The original data collected contains a total of 20 state variables, as shown in Table 1 specifically.

[0057] Table 1 PEMFC Sensor Monitoring Variables

[0058]

[0059] S2) Perform data preprocessing on the original data. The data preprocessing includes normalizing the original data and then evaluating and selecting the feature importance of the normalized data, and using the selected important feature variables as fuel cell fault diagnosis variables.

[0060] Using Min-Max normalization can transform features with different dimensions into the same order of magnitude, reduce the differences between features, accelerate the convergence process of the model, and at the same time enable the model to balance the contributions of each feature and improve the performance of the model. Specifically, the following formula is used for normalizing the original data

[0061]

[0062] In the formula,

[0063] x i * is the value of the data sample after normalization processing,

[0064] x i is the data sample value,

[0065] x imax is the maximum value of the sample value x i ;

[0066] x imin is the minimum value of the sample value x i .

[0067] Use the random forest algorithm (Random forest, RF) to evaluate the feature importance of the original data and select the fault diagnosis variables.

[0068] Preferably, use the Gini Impurity in the random forest algorithm as the feature importance evaluation. The selection criterion of the Gini Impurity is that each child node reaches the highest purity. If the Gini Impurity after feature partitioning is smaller, the purity of the divided feature set is larger, indicating that the feature set is more important. On the contrary, the larger the Gini Impurity, the greater the uncertainty, indicating that the importance of the feature set is lower. Therefore, the Gini Impurity can be used to evaluate the feature importance and thus perform feature selection.

[0069] The formula for the Gini Impurity is as follows

[0070]

[0071] Wherein,

[0072] G m is the Gini impurity,

[0073] K is the category included in the data sample,

[0074] p m,k is the probability that the data sample at node m belongs to the k-th category.

[0075] The feature importance evaluation results in this embodiment are as Figure 2 shown. The higher the feature importance, the richer the information contained in the feature and the greater the impact on the diagnosis result. In order to select the best input feature dimension of the CNN-Transformer model, different data dimensions are selected as the input of the feature extraction model according to the feature importance. The final feature selection situation in this embodiment is shown in Table 2.

[0076] Table 2 Feature Selection Results

[0077] Input model Feature number CNN-Transformer 5,6,7,8,12,14,17

[0078] S3) Divide the data samples selected through feature importance evaluation into a training set and a test set.

[0079] Specifically, the division ratio of the training set and the test set is set to 7:3.

[0080] S4) Input the training set and the test set into the CNN-Transformer model for feature extraction respectively; the CNN (Convolutional Neural Network) model transforms the data from the original state into a form that is concentrated and contains multi-dimensional features; the Transformer model models the global dependence relationship through the self-attention mechanism, captures long-distance interaction information and complex context structures, and outputs a new training set and a new test set.

[0081] CNN is good at extracting local features, efficiently capturing the detailed features of data using local receptive fields and convolutional operations, and at the same time having strong robustness to local changes, while Transformer models the global dependence relationship through the self-attention mechanism and can capture long-distance interaction information and complex context structures.

[0082] Specifically, as Figure 3As shown, the CNN model includes 3 convolutional layers and 2 pooling layers, and the Transformer model includes two encoder layers. Each encoder layer includes an attention layer, 2 residual connection and normalization layers, and a feed-forward network layer.

[0083] The specific steps for the CNN model to transform data from its original state into a form that is concentrated and contains multi-dimensional features are as follows.

[0084] S51) First, after preprocessing and reshaping the data, it is input into the first convolutional layer with a single channel, extracting 32 feature channels. After activation by the RELU function, it is then pooled by the pooling layer.

[0085] S52) Then repeat the convolutional and pooling operations in step S51) once to further expand the number of feature channels to 64 channels.

[0086] S53) Finally, the third convolutional layer generates an output of 128 channels based on the 64 channels.

[0087] After a series of convolutional and pooling operations, the data is transformed from its original state into a form that is concentrated and contains multi-dimensional features.

[0088] The structure of the Transformer model is as Figure 4 shown, which contains two encoder layers. The specific steps for the Transformer model to output a new training set and a new test set are as follows.

[0089] S54) First, flatten the output obtained from the CNN model to get data in a form that conforms to the input of the Transformer model, and then perform positional encoding on the data. The relevant formulas are as follows:

[0090] PE (pos,2i) = sin(pos / 10000 2i / d model )

[0091] PE (pos,2i+1) = cos(pos / 10000 2i / d model )

[0092] In the formula,

[0093] pos is the position of the word vector in the input sequence,

[0094] d model is the dimension of the word vector,

[0095] i is the column index used to map to the positional encoding vector,

[0096] PE (pos,2i)is the element value at an even position in the position encoding vector of the represented word vector,

[0097] PE (pos,2i+1) is the element value at an odd position in the position encoding vector of the represented word vector.

[0098] S55) Input the data after position encoding into the attention layer for global feature extraction. After evaluating the data scale, set the number of attention heads in the attention layer to 1; the involved operation formula is as follows

[0099] Q = input × Q W

[0100] K = input × K W

[0101] V = input × V W

[0102]

[0103] In the formula,

[0104] input is the input vector,

[0105] Q W , K W , V W are matrices automatically generated by the model initially,

[0106] Q, K, and V are the corresponding query matrix, key matrix, and value matrix respectively,

[0107] d k is the dimension of the key matrix K,

[0108] Attention(Q, K, V) is the attention value obtained after the operation.

[0109] S56) Perform residual connection and layer normalization on the output obtained from the attention layer. Residual connection is used to alleviate gradient vanishing and feature enhancement, and layer normalization is used to keep the input distribution of each layer in the model relatively stable, reduce the risk of gradient vanishing or gradient explosion, and accelerate the convergence speed of the model.

[0110] The main idea of residual connection is to directly add the input of a certain layer to the output of that layer, mainly playing the role of alleviating gradient vanishing and feature enhancement. The calculation method is:

[0111] output = sublayer(input) + input

[0112] In the formula,

[0113] The input is the input of a certain operation layer,

[0114] Sublayer(input) is the output of a certain operation layer,

[0115] The output is the obtained residual value.

[0116] Layer normalization can keep the input distribution of each layer in the model relatively stable, reduce the risk of gradient vanishing or gradient explosion. At the same time, it can accelerate the convergence speed of the model and improve the generalization ability of the model. The formula used is as follows:

[0117]

[0118] In the formula,

[0119] x i is the input value,

[0120] μ is the mean,

[0121] δ 2 is the variance,

[0122] γ and β are learnable parameters. γ is used to scale the normalized result, and β is used to shift the normalized result.

[0123] ε is a very small number used to prevent the case where the variance is 0.

[0124] S57) Input the data after residual connection and layer normalization into the feed-forward network layer for feature extraction and dimensionality transformation, and capture the non-linear associations between input features.

[0125] The feed-forward network layer (Feed-Forward Network, FFN) mainly includes two fully connected layers and a RELU activation function. It is mainly used for feature extraction and dimensionality transformation. With the structure of two linear transformation layers and the RELU activation function, it deeply processes the features. Secondly, since the RELU activation function in the feed-forward network layer can effectively capture the non-linear associations between input features, the expressive ability of the model is significantly enhanced. The formula used is as follows:

[0126] FFN(x) = RELU(0, xW 1 + b 1 )W 2 + b 2

[0127] In the formula,

[0128] x is the input matrix,

[0129] W 1 、W 2 are weight matrices respectively,

[0130] b 1 and b 2 are the bias vectors respectively,

[0131] FFN(x) is the output of the feed - forward network layer.

[0132] S58) Perform another residual connection and layer normalization operation on the output of the feed - forward network layer to obtain the output of the first - layer encoder.

[0133] S59) Repeat the process from step S54) to step S58) as a whole. Finally, obtain the new training set and new test set output by the Transformer layer, that is, the new training set and new test set of the fault diagnosis model.

[0134] S5) Construct a double fully - connected layer to enhance the non - linear expression ability of the CNN - Transformer model. Map the multi - dimensional vector output by the Transformer model to a three - dimensional output, which respectively correspond to the three operating states of normal, membrane drying, and hydrogen leakage of the fuel cell predicted by the CNN - Transformer model, and obtain the prediction scores for each group of data.

[0135] Specifically, the double fully - connected layer includes a first fully - connected layer and a second fully - connected layer. The first fully - connected layer maps the 128 - dimensional vector output by the Transformer model to a 100 - dimensional vector, and enhances the non - linear expression ability of the model through the ReLU activation function. This layer not only plays a role in dimensionality reduction but also enables the model to better abstract features. The second fully - connected layer maps the 100 - dimensional vector output by the first fully - connected layer to a 3 - dimensional output, which respectively correspond to the three operating states of normal, membrane drying, and hydrogen leakage of the predicted fuel cell, and obtains the prediction scores for each group of data.

[0136] S6) Measure the error between the prediction scores of the CNN - Transformer model and the true labels by calculating the loss function. Use the backpropagation algorithm to calculate the gradient of the loss with respect to the parameters of the CNN - Transformer model. The gradient indicates the direction and magnitude in which the parameters need to be adjusted, and through continuous iterative optimization, gradually reduce the value of the loss function to achieve the optimal parameter configuration.

[0137] The error between the prediction scores and the true labels. The smaller the loss value, the better the performance of the model. In addition, by clearing the gradients, it is ensured that each gradient update will not be interfered by the previous calculation results, and the impact of gradient accumulation on the training effect is avoided. The gradient indicates the direction and magnitude in which the parameters need to be adjusted, serving as the basis for the optimizer to update the model parameters. Through continuous iterative optimization, the model gradually reduces the value of the loss function, thereby improving the prediction performance and finally achieving the optimal parameter configuration.

[0138] S7) Bring the new test set into the CNN-Transformer model with optimized parameters, and output the fuel cell fault diagnosis results, that is, the fuel cell is in a normal state, or a membrane dry state, or a hydrogen leakage state, and perform performance verification.

[0139] As Figures 5a - 5b shown, it is the fault diagnosis accuracy rate and confusion matrix of the specific embodiment of the present invention, including normal, membrane dry and hydrogen leakage. The overall classification accuracy rate of the three types of data is 98.7778%.

[0140] The fuel cell fault diagnosis method based on CNN-Transformer of the present invention can not only better distinguish the three operating states of normal, membrane dry and hydrogen leakage of the fuel cell through the proposed CNN-Transformer algorithm, effectively judge the fuel cell faults, but also overcome the problems of insufficient robustness and weak generalization ability of a single model.

[0141] The above embodiments are the preferred embodiments of the present invention, but the embodiments of the present invention are not limited by the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications made without departing from the spirit and principle of the present invention shall be equivalent replacement methods and are all included in the protection scope of the present invention.

Claims

1. A fuel cell fault diagnosis method based on CNN-Transformer, characterized in that: The steps include: S1) collecting raw data, wherein the raw data is a state variable with respect to time characteristics during the operation of the fuel cell; The state variables include the anode / cathode inlet and outlet pressures of the stack, the cathode inlet and outlet temperatures, the cathode air excess coefficient, the water inlet pressure, the water inlet flow rate, the load current, the cooling water inlet temperature, and the anode / cathode air inlet flow rate; S2) performing data preprocessing on the original data, wherein the data preprocessing includes normalizing the original data, and then evaluating and selecting feature importance of the normalized data, and selecting important feature variables as fuel cell fault diagnosis variables; S3) dividing the data samples selected by feature importance evaluation into a training set and a test set; S4) The training set and the test set are respectively input into the CNN-Transformer model for feature extraction; the CNN model transforms the data from the original state into a condensed form containing multiple dimensional features; The Transformer model uses the self-attention mechanism to model global dependencies of the output results of the CNN model, capture long-distance interaction information and complex context structures, and output new training sets and new test sets. S5) constructing a double fully connected layer to enhance the nonlinear expression ability of the CNN-Transformer model, mapping the multidimensional vector output by the Transformer model into a three-dimensional output, which respectively corresponds to the three operating states of the fuel cell predicted by the CNN-Transformer model, namely, normal, membrane dry, and hydrogen leakage, and obtaining the prediction score for each set of data; S6) Measure the error between the predicted score of the CNN-Transformer model and the true label by calculating the loss function, use the back propagation algorithm to calculate the gradient of the loss to the parameters of the CNN-Transformer model, and use the gradient to indicate the direction and magnitude of the parameter adjustment. And through continuous iterative optimization, gradually reduce the loss function value to achieve the optimal parameter configuration; S7) The new test set is introduced into the CNN-Transformer model after parameter optimization, and the fuel cell fault diagnosis result is output, that is, the fuel cell is in a normal state, or a membrane dry state, or a hydrogen leakage state.

2. The fuel cell fault diagnosis method based on CNN-Transformer according to claim 1, characterized in that: S2), the original data is normalized using the following formula In the formula, x i * is the value of the data sample after normalization. x i is the data sample value, x imax is the sample value x i The maximum value of x imin is the sample value x i The minimum value of .

3. The fuel cell fault diagnosis method based on CNN-Transformer according to claim 2 is characterized in that: In S2), the Gini impurity in the random forest algorithm is used as the feature importance assessment. The Gini impurity calculation formula is as follows In the formula, G m is the Gini impurity, K is the number of categories contained in the data sample. p m,k is the probability that the data sample at node m belongs to the kth class.

4. The fuel cell fault diagnosis method based on ensemble learning according to claim 1 is characterized in that: In S3), the division ratio of the training set and the test set is set to 7:

3.

5. The fuel cell fault diagnosis method based on CNN-Transformer according to claim 4 is characterized in that: In S4), the CNN model includes 3 convolutional layers and 2 pooling layers, and the Transformer model includes two encoder layers, each of which includes an attention layer, 2 residual connections and normalization layers, and a feedforward network layer.

6. The fuel cell fault diagnosis method based on CNN-Transformer according to claim 5 is characterized in that: In S4), the specific steps of the CNN model to transform the data from the original state into a condensed form containing multiple dimensional features are as follows: S51) First, after preprocessing and reshaping the data, the first convolutional layer uses a single channel as input to extract 32 feature channels, which are activated by the RELU function and then pooled by the pooling layer; S52) repeating the convolution and pooling operations in step S51) to further expand the number of feature channels to 64 channels; S53) Finally, the third convolutional layer generates a 128-channel output on top of the 64-channel one.

7. The fuel cell fault diagnosis method based on CNN-Transformer according to claim 6 is characterized in that: In S4), the specific steps for the Transformer model to output new training sets and new test sets are as follows: S54) first, flatten the output obtained by the CNN model to obtain data in the form of Transformer model input, and then perform position encoding on the data; S55) inputting the position-encoded data into the attention layer to extract global features, and after evaluating the data scale, setting the number of attention heads in the attention layer to 1; S56) performing residual connection and layer normalization on the output obtained by the attention layer, wherein the residual connection is used to alleviate gradient vanishing and feature enhancement, and the layer normalization is used to keep the input distribution of each layer in the model relatively stable, reduce the risk of gradient vanishing or gradient explosion, and accelerate the convergence speed of the model; S57) inputting the residual connection and layer normalized data into the feedforward network layer to perform feature extraction and dimension transformation, and capture the nonlinear correlation between input features; S58) performing a residual connection and layer normalization operation on the output of the feedforward network layer again to obtain the output of the first layer encoder; S59) repeats the process from step S54) to step S58) as a whole, and finally obtains a new training set and a new test set output by the Transformer layer.

8. The fuel cell fault diagnosis method based on CNN-Transformer according to claim 7 is characterized in that: S5), the dual fully connected layer includes a first fully connected layer and a second fully connected layer, the first fully connected layer maps the 128-dimensional vector output by the Transformer model to a 100-dimensional vector, and enhances the nonlinear expression ability of the model through the ReLU activation function; the second fully connected layer maps the 100-dimensional vector output by the first fully connected layer to a 3-dimensional output, which respectively corresponds to the predicted three operating states of the fuel cell: normal, membrane dry and hydrogen leakage, to obtain a prediction score for each group of data.

Images