A hydrogen fuel cell early fault diagnosis method based on multi-modal data fusion

Abstract

The present application relates to fuel cell operation and maintenance and fault diagnosis technical field, specifically to a kind of hydrogen fuel cell early fault diagnosis method based on multi-modal data fusion, comprising the following steps: data acquisition and preprocessing, obtain the standardized input data for evaluation;Multi-modal data fusion deep learning network model is constructed and trained;Model training and online evaluation;Beneficial effects are: two-dimensional magnetic field space distribution diagram constructed by dense magnetic sensor array is first carried out multi-modal fusion with one-dimensional time series voltage signal, and the magnetic field space snapshot of starting, middle, end three time points is extracted in a time window and spliced into three-channel image along channel dimension, while retaining complete voltage time sequence waveform.This configuration makes magnetic field information from "point measurement" upgrade "image characterization", with voltage signal to supplement global time dynamic, realize the synchronous capture of fault local space change and global time evolution, fundamentally make up the observation blind area of single mode.

Description

Technical Field

[0001] This invention relates to the field of fuel cell operation and maintenance and fault diagnosis, specifically to a method for early fault diagnosis of hydrogen fuel cells based on multimodal data fusion. Background Technology

[0002] Currently, hydrogen fuel cell systems typically diagnose faults by monitoring changes in parameters such as cell voltage, current distribution, and gas supply pressure. These methods have the advantage of not requiring precise physical models, but suffer from low signal dimensionality and an inability to capture the spatial distribution characteristics of faults. Furthermore, methods based on electrochemical impedance spectroscopy have clear physical meaning but demanding hardware requirements; non-invasive methods based on external magnetic field measurements can reflect internal current distribution, but traditional approaches neglect cutting-edge approaches such as spatial information and multi-source information fusion.

[0003] In existing technologies, data-driven methods for fuel cell fault diagnosis include the following two technical solutions: 1. A hydrogen fuel cell fault detection device and method employing a data-driven approach. This scheme (as described in patent CN121790448A) obtains training data for a deep learning model for hydrogen fuel cell fault diagnosis through sensors; trains the model using the aforementioned data; collects test data through sensors and inputs it into the trained deep learning model for hydrogen fuel cell fault diagnosis to calculate the output voltage of the hydrogen fuel cell; compares the output voltage obtained from sensor testing with the output voltage predicted by the model; if the deviation exceeds a user-preset threshold, the fault state of the hydrogen fuel cell is displayed. This invention processes a series of sensor data that significantly affect the output characteristics of the hydrogen fuel cell to establish a predictive model for the output voltage, thereby determining whether the battery characteristics deviate from the normal operating state and obtaining the final detection result of whether the battery is in a fault state.

[0004] 2. A deep learning-based fault diagnosis method for proton exchange membrane fuel cells (PEMFCs), comprising (as described in patent CN121642032A) constructing a semi-mechanistic, semi-empirical model of a PEMFC system, collecting time-series data from multiple sensors; preprocessing the time-series data, and converting the one-dimensional time-series data into a two-dimensional image using Grammy angle field technology; using the two-dimensional image as input, employing a pre-trained ResNet50 convolutional neural network combined with transfer learning for feature extraction and fault classification, training, validating, and testing the neural network, and outputting fault classification results. By constructing a semi-mechanistic, semi-empirical model of the fuel cell system and preliminarily verifying its accuracy, the application of ResNet50 and related knowledge of transfer learning to PEMFC fault diagnosis results in higher fault diagnosis accuracy.

[0005] The common feature of the two fault diagnosis schemes mentioned above is that they use data-driven methods to accurately identify the fault status of hydrogen fuel cells and solve the problem of online monitoring of the operating status of hydrogen fuel cells.

[0006] Although the aforementioned data-driven existing technical solutions have made progress in fault diagnosis, they still have the following significant technical shortcomings compared with the proposal of this application. These shortcomings are precisely the core technical problems that this application aims to solve: 1. Insufficient utilization of magnetic field data and wasted spatial information. While recent diagnostic methods based on external magnetic fields can capture spatial information about internal current distribution, most approaches simply concatenate magnetic field data from multiple sensors into a one-dimensional time-series vector, or extract only statistics (such as mean and variance), discarding the spatial topological relationships between the sensors. In reality, magnetic field sensor arrays naturally possess a two-dimensional or three-dimensional spatial layout, and changes in magnetic field strength at different locations directly correspond to changes in current density in different regions within the fuel cell. Existing technologies generally "reduce the dimensionality" of magnetic field data, essentially still using one-dimensional methods to process high-dimensional data, failing to truly leverage the spatial advantages of magnetic field imaging.

[0007] 2. Early-stage fault features are weak and difficult to capture using traditional methods. Existing fault diagnosis methods generally treat faults as a static category, focusing on the state identification after the fault has fully manifested, while ignoring the dynamic evolution of a fault from its inception to its deterioration. Most methods do not differentiate between fault stages, treating early, middle, and late-stage samples uniformly. This leads to models tending to learn patterns with obvious and easily categorized late-stage fault features during training and diagnosis, while lacking sensitivity to early-stage fault features characterized by weak amplitude, low signal-to-noise ratio, and blurred boundaries with normal states. Specifically, in the early stage where voltage drop is less than 5%, the diagnostic accuracy of existing methods drops significantly, and often requires accumulating long windows of data exceeding 100 seconds to extract distinguishable fault features. This indiscriminate approach to fault stages makes it difficult for diagnostic systems to issue timely warnings when faults first appear, missing the optimal intervention window.

[0008] 3. Limited modal selection and information dimensions. Existing technologies mainly rely on time-series signals such as voltage and current for diagnosis. Although these signals can reflect the overall performance changes of the fuel cell, they are essentially one-dimensional time series with zero spatial resolution. Faults (such as local flooding, local membrane dryness, and uneven gas distribution) first manifest as spatial heterogeneity of internal current density. This heterogeneity is often averaged in the macroscopic voltage signal, causing early minor fault characteristics to be submerged or coupled with other faults, making them difficult to distinguish. Summary of the Invention

[0009] The purpose of this invention is to provide an early fault diagnosis method for hydrogen fuel cells based on multimodal data fusion, so as to solve the technical problems of low early diagnosis accuracy and slow response speed caused by the inability of existing single-modal diagnosis methods to simultaneously obtain the temporal and spatial characteristics of faults.

[0010] To achieve the above objectives, the present invention provides the following technical solution: a method for early fault diagnosis of hydrogen fuel cells based on multimodal data fusion, the method comprising the following steps: S1: Data acquisition and preprocessing, obtaining standardized input data for evaluation; S2: Construct and train a multimodal data fusion deep learning network model. The network model's structural design includes magnetic field image feature extraction, voltage time series feature extraction, cross-modal attention fusion module, and classification layer. S3: Model training and online evaluation.

[0011] Preferably, the data acquisition includes the following steps: The timing voltage signal is collected, and the output voltage data under the operation of the hydrogen fuel cell is obtained by using a high-precision electronic load. An M×N fluxgate sensor array is arranged outside the PEMFC bipolar plate to obtain a two-dimensional magnetic field spatial distribution map at each moment, where M and N are both greater than 1 and M×N is not less than 4.

[0012] Preferably, the data preprocessing includes the following steps: External magnetic field data collected in advance under 0A load conditions of hydrogen fuel cells are compared with external magnetic field data under operating conditions to eliminate interference from geomagnetic and environmental magnetic fields, and finally ΔB is obtained; the collected voltage data are processed by sliding window to obtain voltage sequence signals of a specific time window length.

[0013] Preferably, the data preprocessing further includes the following steps: ΔB is normalized and mapped to pixel grayscale values ​​according to its element values ​​to generate a two-dimensional grayscale image of M×N pixels. Three magnetic field space snapshots at the start, middle and end times of the time window are extracted and stitched along the channel dimension to form a three-channel external magnetic field image that represents the spatiotemporal evolution of the magnetic field. .

[0014] Preferably, the magnetic field image feature extraction includes the following steps: The three-channel magnetic field image generated in step S1 B As input, the path contains a three-layer convolutional module and a three-layer channel-region attention module for extracting external magnetic field image features; The channel-region attention module works in parallel after the convolutional module: Channel attention: Weighting the characteristics of different time channels to capture the evolution of the magnetic field over time; Region attention: Different spatial regions of the feature map are weighted, focusing on specific regions related to the fault. Complementary attention is calculated using channel and region attention modules to address the issues of feature "temporal variation" and "regional variation" respectively. The final calculation yields the enhanced magnetic field image features. .

[0015] Preferably, the voltage timing feature extraction includes the following steps: Voltage time-series signals are input into a Long Short-Term Memory (LSTM) network for modeling. A temporal attention module is introduced to dynamically calculate the weights at different time steps, enabling the model to focus on the most discriminative key time points in the early stages of a fault. The voltage features extracted by the LSTM are weighted with the temporal attention weights to obtain the key temporal features. .

[0016] Preferably, the cross-modal attention fusion module is used to deeply fuse features from different modalities of the image and voltage signal, and its workflow is as follows: Voltage feature vector Projected into a query vector through linear transformation; Features of magnetic field images After being reshaped into a feature sequence, it is projected into a key vector and a value vector through linear transformations respectively; The scaled dot product of the query vector and all key vectors is calculated, and the attention weights are obtained after normalization using the Softmax function. This process enables the network to adjust its attention weights according to the current voltage state. Dynamically monitor magnetic field image features The region most relevant to it; The value vector is weighted and summed using the attention weights obtained above, and a joint feature representation incorporating the electro-magnetic correlation is finally output. .

[0017] Preferably, the classification layer includes: Fault state classification network: fusing feature vectors After flattening, a probability distribution is output through a fully connected layer and a Softmax activation function, which represents the probability that the hydrogen fuel cell belongs to multiple predefined abnormal state categories.

[0018] Preferably, the model training includes the following steps: The multimodal data fusion deep learning network was trained end-to-end using a labeled dataset containing multiple anomaly types, and its parameters were optimized. The labeled dataset containing multiple anomaly types was obtained by introducing different types of faults during the operation of the hydrogen fuel cell and capturing the time period when the voltage drop was less than 5%.

[0019] Preferably, the online assessment includes the following steps: The real-time differential magnetic field image obtained by processing the operating data of the hydrogen fuel cell under test in step S1, along with the synchronously acquired and normalized voltage parameters, are input into the trained model. The model provides the probability distribution information of the abnormal state of the hydrogen fuel cell at this time, which is used by maintenance personnel for decision-making.

[0020] Compared with the prior art, the beneficial effects of the present invention are: This invention proposes a multimodal data fusion-based early fault diagnosis method for hydrogen fuel cells. For the first time, it integrates a two-dimensional magnetic field spatial distribution map constructed from a dense magnetic sensor array with a one-dimensional time-series voltage signal. Within a single time window, it extracts three snapshots of the magnetic field space at the start, middle, and end points, stitching them together along the channel dimension to form a three-channel image while preserving the complete voltage time-series waveform. This configuration upgrades magnetic field information from "point measurement" to "image-based representation," truly leveraging the spatial advantages of magnetic field imaging. Simultaneously, it supplements the global temporal dynamics with voltage signals, achieving synchronous capture of local spatial changes and global temporal evolution of the fault, fundamentally compensating for the blind spots of single-mode observation.

[0021] A TCZ-MFN multimodal fusion network was designed to address early-stage faults. It utilizes a channel-partition attention mechanism to focus on fault-related local spatial regions (such as gas inlet, outlet, and central region) in the magnetic field image, a temporal attention mechanism to pinpoint key time points in the voltage signal reflecting early-stage trends, and cross-modal cross-attention to establish deep interactive correlations between the two types of features. This effectively enhances the extraction capability of early-stage weak fault features. Experimental results show that in the early fault stage (voltage drop less than 5%), the diagnostic accuracy of the proposed method can reach over 95%, a performance significantly superior to existing technologies.

[0022] By employing multimodal information complementarity and a dedicated attention mechanism, magnetic field and voltage signals are enhanced during the data preprocessing stage. This allows the model to extract sufficiently discriminative fault features from weak signals within a short observation window of only 3 seconds, with a total system response time of approximately 3.03 seconds (including 30 milliseconds of inference time). This provides ample intervention time for the control system, meeting the requirements for rapid fault response in practical engineering applications, and is particularly suitable for rapidly deteriorating scenarios such as gas starvation and air starvation. Attached Figure Description

[0023] Figure 1 This is a diagram showing the overall method steps of this invention patent; Figure 2 This is a diagram of the channel-region attention structure of the present invention. Figure 3 This is a diagram of the cross-modal attention module of this invention patent; Figure 4 This is a magnetic field monitoring diagram for a hydrogen fuel cell, as per the present invention patent. Figure 5 This is a diagram showing the alignment of the magnetic field and voltage in this patent application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the present invention clear and complete, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only some, not all, embodiments of the present invention, and are merely illustrative of the embodiments of the present invention. They are not intended to limit the embodiments of the present invention. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] Example 1: The present invention provides a technical solution: performance evaluation of early fault datasets based on 4x4 sensor arrays and voltage data.

[0026] This embodiment evaluates a proton exchange membrane hydrogen fuel cell.

[0027] Hardware configuration and data acquisition: 1. A 4x4 fluxgate sensor array is used, such as... Figure 4 As shown, it is fixed at a uniform spacing about 0.5 cm outside the bipolar plate of the hydrogen fuel cell.

[0028] 2. Simultaneously acquire 16 channels of magnetic field data and record the total voltage and total current of the hydrogen fuel cell electronic load.

[0029] 3. Baseline Establishment: Under 0A load conditions, collect 20 minutes of magnetic field data and calculate the average value as the baseline magnetic field matrix B_ref.

[0030] 4. Simulating anomalies: In the hydrogen fuel cell system, hydrogen starvation faults caused by factors such as hydrogen leakage and pipeline blockage are simulated by reducing the hydrogen supply in actual situations; air starvation faults are simulated by setting experimental conditions based on the same principle; and water management faults are simulated by increasing or decreasing the intake air humidity in actual situations.

[0031] 5. Image generation: The magnetic field data B_test and B_ref collected by the magnetic array during the experiment are differentially and normalized to generate a grayscale image of 16 pixels. The image is then enlarged to a differential magnetic field image of 224x224 pixels by bilinear interpolation. The magnetic field images at three time points are then stitched together as three channels to obtain a three-channel magnetic field image.

[0032] 6. Align the three-channel magnetic field image with the time-series voltage data, such as... Figure 5 As shown, a dataset is constructed as fault data.

[0033] Model building and training: 1. Magnetic field image path: Feature extraction is performed using a CNN extractor. The extracted features are then weighted using the channel-region attention module of the fault diagnosis model to enhance key features. Multiple fault extraction and enhancement processes are then performed to finally obtain the modal fault features of the magnetic field image. .

[0034] 2. Time-series voltage path: The normalized input voltage vector is captured by an LSTM time encoder to obtain the dynamic dependence and temporal evolution of fault features in the time series. This is then weighted with a time-series attention module to obtain voltage mode fault features. .

[0035] 3. Cross-modal attention fusion: As a query, The reshaped sequences are used as keys and values, fused together by a Transformer decoder layer, and the fused features are output. .

[0036] 4. Output layer: The fault category probability of the current hydrogen fuel cell is output after passing through a fully connected layer and a Softmax activation function (corresponding to four typical faults: hydrogen starvation, air starvation, flooding, and membrane dryness).

[0037] After training on an early fault dataset with a voltage drop of less than 5%, the method described in this embodiment achieves a classification accuracy of 96.2% for different faults. In comparison, a control method using a single modality (inputting only images or voltage data) achieves classification accuracies of 90.1% and 85.2%, respectively. This fully demonstrates the decisive role of multimodal data fusion in improving the accuracy of fault diagnosis.

[0038] Example 2: Extended applications for fault data of different degrees.

[0039] This embodiment demonstrates the ability of the method of the present invention to diagnose faults in hydrogen fuel cells under different fault severity levels, highlighting its applicability in practical engineering.

[0040] 1. On the hydrogen fuel cell experimental platform, in addition to the four typical standard fault states of hydrogen starvation, air starvation, membrane dryness, and water flooding set in Example 1, multiple severity levels were further set for each type of fault. For example, by adjusting the anode stoichiometry ratio from the normal value of 1.5 to 0.9, 0.7, and 0.6, multiple gradients from slight hydrogen starvation to severe hydrogen starvation were simulated; by reducing the cathode stoichiometry ratio from 3.5 to 1.5, 1.3, and 1.1, different degrees of air starvation were simulated; by increasing the humidifier temperature from 60°C to 70°C, 75°C, and 80°C, mild, moderate, and severe water flooding were simulated; and by introducing unhumidified gas for different durations, membrane dryness faults of different degrees of dryness were simulated.

[0041] 2. For each fault level, a two-dimensional spatial distribution map of the external magnetic field and a time-series voltage signal were collected. The data preprocessing and image generation methods were exactly the same as in Example 1, and the network parameters remained consistent with those in Example 1. No retraining or adjustment for each fault level was required. Experimental results show that the accuracy rate remained above 90%.

[0042] 3. This embodiment verifies the generalization ability of the method of the present invention for faults across severity levels: even when trained using fault data of only a specific severity level, the model can still effectively identify similar faults of other severity levels. This is due to the cross-scale consistency of the physical fault features extracted by the multimodal data fusion framework. In summary, Embodiment 2 demonstrates that the method of the present invention can adapt to complex scenarios in actual engineering where the severity of faults is unknown, and possesses good scalability and industrial application value.

[0043] Test results show that even in the early stages of a hydrogen fuel cell failure where the voltage drop is less than 5%, the method proposed in this application still achieves a fault diagnosis accuracy of over 96%. This demonstrates the powerful feature extraction and pattern recognition capabilities of this method. Through the correlation and fusion learning of "voltage sequence-magnetic field image," the model can effectively extract feature vectors related to different fault states, achieving accurate diagnosis of early faults, which is of great significance for ensuring the stable operation of hydrogen fuel cell systems.

[0044] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for early fault diagnosis of hydrogen fuel cells based on multimodal data fusion, characterized in that: The method includes the following steps: S1: Data acquisition and preprocessing, obtaining standardized input data for evaluation; S2: Construct and train a multimodal data fusion deep learning network model. The network model's structural design includes magnetic field image feature extraction, voltage time series feature extraction, cross-modal attention fusion module, and classification layer. S3: Model training and online evaluation.

2. The method for early fault diagnosis of hydrogen fuel cells based on multimodal data fusion according to claim 1, characterized in that: The data collection includes the following steps: The timing voltage signal is collected, and the output voltage data under the operation of the hydrogen fuel cell is obtained by using a high-precision electronic load. An M×N fluxgate sensor array is arranged outside the PEMFC bipolar plate to obtain a two-dimensional magnetic field spatial distribution map at each moment, where M and N are both greater than 1 and M×N is not less than 4.

3. The method for early fault diagnosis of hydrogen fuel cells based on multimodal data fusion according to claim 1, characterized in that: The data preprocessing includes the following steps: External magnetic field data collected in advance under 0A load conditions of hydrogen fuel cells are compared with external magnetic field data under operating conditions to eliminate interference from geomagnetic and environmental magnetic fields, and finally ΔB is obtained; the collected voltage data are processed by sliding window to obtain voltage sequence signals of a specific time window length.

4. The method for early fault diagnosis of hydrogen fuel cells based on multimodal data fusion according to claim 3, characterized in that: The data preprocessing also includes the following steps: ΔB is normalized and mapped to pixel grayscale values ​​according to its element values ​​to generate a two-dimensional grayscale image of M×N pixels. Three magnetic field space snapshots at the start, middle and end times of the time window are extracted and stitched along the channel dimension to form a three-channel external magnetic field image that represents the spatiotemporal evolution of the magnetic field. 。 5. The method for early fault diagnosis of hydrogen fuel cells based on multimodal data fusion according to claim 4, characterized in that: The magnetic field image feature extraction includes the following steps: The three-channel magnetic field image generated in step S1 B As input, the path contains a three-layer convolutional module and a three-layer channel-region attention module for extracting external magnetic field image features; The channel-region attention module works in parallel after the convolutional module: Channel attention: Weighting the characteristics of different time channels to capture the evolution of the magnetic field over time; Region attention: Different spatial regions of the feature map are weighted, focusing on specific regions related to the fault. Complementary attention is calculated using channel and region attention modules to address the issues of feature "temporal variation" and "regional variation" respectively. The final calculation yields the enhanced magnetic field image features. .

6. The method for early fault diagnosis of hydrogen fuel cells based on multimodal data fusion according to claim 5, characterized in that: The voltage timing feature extraction includes the following steps: Voltage time-series signals are input into a Long Short-Term Memory (LSTM) network for modeling. A temporal attention module is introduced to dynamically calculate the weights at different time steps, enabling the model to focus on the most discriminative key time points in the early stages of a fault. The voltage features extracted by the LSTM are weighted with the temporal attention weights to obtain the key temporal features. .

7. The method for early fault diagnosis of hydrogen fuel cells based on multimodal data fusion according to claim 6, characterized in that: The cross-modal attention fusion module is used to deeply fuse features from different modalities of images and voltage signals, and its workflow is as follows: Voltage feature vector Projected into a query vector through linear transformation; Features of magnetic field images After being reshaped into a feature sequence, it is projected into a key vector and a value vector through linear transformations respectively; The scaled dot product of the query vector and all key vectors is calculated, and the attention weights are obtained after normalization using the Softmax function. This process enables the network to adjust its attention weights according to the current voltage state. Dynamically monitor magnetic field image features The region most relevant to it; The value vector is weighted and summed using the attention weights obtained above, and a joint feature representation incorporating the electro-magnetic correlation is finally output. .

8. The method for early fault diagnosis of hydrogen fuel cells based on multimodal data fusion according to claim 7, characterized in that: The classification layer includes: Fault state classification network: fusing feature vectors After flattening, a probability distribution is output through a fully connected layer and a Softmax activation function, which represents the probability that the hydrogen fuel cell belongs to multiple predefined abnormal state categories.

9. The method for early fault diagnosis of hydrogen fuel cells based on multimodal data fusion according to claim 1, characterized in that: The model training includes the following steps: The multimodal data fusion deep learning network was trained end-to-end using a labeled dataset containing multiple anomaly types, and its parameters were optimized. The labeled dataset containing multiple anomaly types was obtained by introducing different types of faults during the operation of the hydrogen fuel cell and capturing the time period when the voltage drop was less than 5%.

10. The method for early fault diagnosis of hydrogen fuel cells based on multimodal data fusion according to claim 9, characterized in that: The online assessment includes the following steps: The real-time differential magnetic field image obtained by processing the operating data of the hydrogen fuel cell under test in step S1, along with the synchronously acquired and normalized voltage parameters, are input into the trained model. The model provides the probability distribution information of the abnormal state of the hydrogen fuel cell at this time, which is used by maintenance personnel for decision-making.

Images