Adaptive working condition aware fuel cell hybrid tram layered management method

Abstract

The application discloses a fuel cell hybrid railcar layered energy management method based on adaptive working condition sensing; in the identification layer, a sliding window mechanism is used to extract load working condition time domain and frequency domain features, feature data is clustered based on a spectral clustering algorithm driven by a deep auto-encoder, a data set with a category label is obtained, and a deep dynamic learning vector quantization neural network classifier is trained; in the strategy layer, a double-delay deep deterministic policy gradient reinforcement learning algorithm is used to construct a reward function, a lithium battery SOC fluctuation penalty term limit parameter in the reward function is adaptively adjusted according to a real-time load working condition category output by the identification layer, and an optimal power distribution scheme between multiple fuel cell power generation systems and lithium batteries is obtained by training a reinforcement learning intelligent agent; according to the performance degradation degree of different fuel cell stacks, a distributed collaborative control strategy considering performance differences is used to distribute the output power of each stack, so that the coordinated control of the operating states of the multiple fuel cell power generation systems is realized.

Description

Technical Field

[0001] This invention belongs to the field of fuel cell hybrid tram technology, and in particular relates to a hierarchical management method for fuel cell hybrid trams with adaptive operating condition perception. Background Technology

[0002] Against the backdrop of dual carbon emissions, and with the continuous expansion of my country's rail transit system, the requirements for clean and sustainable development are becoming increasingly stringent. Compared to traditional rail transit systems powered by overhead contact lines or conventional internal combustion engines, new overhead contactless rail transit systems are better suited to the needs of modern urban development. Hydrogen energy, as a clean, efficient, safe, and sustainable new energy source, stands out due to its absolute zero emissions in the context of decarbonization. Proton exchange membrane fuel cells, as devices that directly convert hydrogen energy into electrical energy, feature high power generation efficiency, low operating noise, and clean, pollution-free operation, and are widely used in rail transit, aerospace, distributed power generation, and new energy vehicles.

[0003] In high-power applications, proton exchange membrane fuel cells (PEMFCs) often struggle to meet load power demands as a single stack due to inherent limitations such as soft output characteristics, slow dynamic response, and inability to recover braking energy. To overcome these shortcomings, multiple fuel cell stacks are typically used collaboratively to form a fuel cell power generation system, integrating energy storage devices to create a hybrid power system that collectively meets load requirements. Energy storage devices can not only recover excess braking energy but also quickly respond to power shortages, adapting to instantaneous changes in load demand and thus improving the overall dynamic performance of the system. However, traditional power control strategies for multi-stack fuel cells often fail to adequately consider the performance differences between stacks. These differences lead to further reductions in the energy conversion efficiency of weaker stacks during operation, accelerating their degradation and consequently limiting the service life and operational stability of the entire system. Therefore, for multi-stack fuel cell power generation systems, it is necessary to adopt a collaborative control method that considers performance differences, optimizing the power allocation among stacks to comprehensively improve system efficiency and extend its service life.

[0004] Energy management strategy is the core of the entire hybrid power system control and is crucial for achieving optimal power distribution. However, current energy management methods for fuel cell hybrid trams, whether rule-based or optimization-based, are mostly geared towards fixed operating conditions. In actual operation, however, due to the presence of random factors, load conditions can change randomly. Summary of the Invention

[0005] To address the aforementioned issues, this invention proposes an adaptive operating condition-aware hierarchical management method for fuel cell hybrid trams. This method can make targeted decisions online in response to different complex operating conditions, thereby achieving the optimal online power distribution control law for the hybrid power system.

[0006] To achieve the above objectives, the technical solution adopted by this invention is: a hierarchical management method for fuel cell hybrid trams based on adaptive operating condition perception, comprising the following steps:

[0007] S100 configures parameters for the fuel cell hybrid system based on the feasible domain of vehicle parameter matching, determines the topology of the hybrid system, and divides the energy management strategy into an identification layer and a strategy layer according to the control mode.

[0008] In the recognition layer, the S200 uses a sliding window mechanism to extract the time and frequency domain features of the load condition data within the window based on multiple segments of actual load condition data. It trains a deep autoencoder to unsupervisedly compress and reconstruct the extracted features and use them as input to the spectral clustering algorithm. Through cluster analysis, it automatically determines the category of load condition, which includes: base power state, equal power state, and peak power state.

[0009] Based on the feature reconstruction results of the deep autoencoder and the category classification results of spectral clustering in step S200, S300 preprocesses them to form a training dataset, and trains a deep dynamic learning vector quantization neural network classifier offline to achieve fast real-time category classification under complex random conditions.

[0010] In the policy layer, S400 uses the category label output by the deep dynamic learning vector quantization classifier trained in step S300 as one of the input states of the dual-delay deep deterministic policy gradient reinforcement learning agent. It adaptively adjusts the parameters of the lithium battery SOC fluctuation penalty term in the reward function in real time according to the working condition category, and trains the agent using a priority experience replay mechanism to achieve real-time optimal power allocation for the entire hybrid system.

[0011] The S500 assessment characterizes the performance degradation of each fuel cell stack. Based on the degradation status of each stack, it calculates the allocation ratio of its output power increment and adopts a distributed cooperative control strategy that takes into account performance differences to rationally allocate the required power to each fuel cell stack.

[0012] Furthermore, in step S200, a sliding window mechanism is used to extract time-domain and frequency-domain features such as mean, peak value, main frequency component, and frequency domain energy of the data within the window. A deep autoencoder is then trained to compress the high-dimensional feature data into low-dimensional data, which is then used as input to the spectral clustering algorithm.

[0013] Furthermore, the deep autoencoder-driven spectral clustering algorithm includes the following steps:

[0014] (1) Training the deep autoencoder: The input to the deep autoencoder is determined to be the extracted high-dimensional feature data, i.e., X = {x1, x2, ..., x...} n};

[0015] The encoding process of a deep autoencoder can be represented as follows:

[0016]

[0017] Where z is the output of the encoder, Let L be the weight matrix of the encoder layer L. Let f be the bias vector of the Lth layer of the encoder, σ be the activation function, θ be the encoder parameters, and f be the bias vector of the Lth layer of the encoder. θ For encoding functions;

[0018] The decoding process of a deep autoencoder is represented as follows:

[0019]

[0020] in, This is the output of the decoder, i.e., the reconstructed feature vector; Let L be the weight matrix of the decoder's Lth layer. Let σ' be the bias vector of the Lth layer of the decoder, σ′ be the activation function, and φ be the decoder parameters. This is the decoding function;

[0021] The loss function is expressed as:

[0022]

[0023] The first term is the reconstruction error, the second term is the regularization term, and N w X is the number of training samples. i For the i-th input sample, Let be the i-th reconstructed sample, λ be the regularization coefficient, l be the layer index of the deep autoencoder, ranging from 1 to L, and F be the Frobenius norm;

[0024] The gradient descent algorithm is used to iteratively update the model parameters until convergence.

[0025] (2) Construct a similarity matrix: Calculate the similarity between data using the Gaussian kernel function;

[0026] (3) Construct the degree matrix: The degree matrix is ​​a diagonal matrix used to represent the sum of the similarities between each data point and other points, reflecting the connection strength of each node;

[0027] (4) Calculate the Laplacian matrix: Calculate the normalized Laplacian matrix based on the similarity matrix and degree matrix. Its eigenvalues ​​and eigenvectors reveal the grouping structure of the data.

[0028] (5) Eigendecomposition: Perform eigendecomposition on the Laplacian matrix to extract its eigenvalues ​​and eigenvectors, and select the eigenvectors corresponding to the k smallest non-zero eigenvalues.

[0029] The matrix expression formed by k eigenvectors is:

[0030] U = [v1, v3, ..., v k ];

[0031] Where U is the eigenvector matrix, [v1, v3, ..., v k [] represents the eigenvectors corresponding to the first k smallest non-zero eigenvalues;

[0032] (6) Dimensionality reduction and clustering: Each row of the feature vector matrix U is regarded as the representation of data points in a low-dimensional space. Traditional clustering algorithms are applied to group the data points and obtain the final category labels.

[0033] Furthermore, in step S300, based on the clustering results of step S200, the labeled feature vectors are normalized to form a training dataset for training a deep dynamic learning vector quantization classifier. The training steps are as follows:

[0034] (1) Network initialization: Initialize the weight vector and learning rate between the input layer and the competition layer with small random numbers, and define the classification error and prototype frequency threshold to dynamically adjust the prototype vector;

[0035] (2) Recent Prototype Identification: Randomly select one sample from each label set to form a prototype vector ω = (ω1, ω2, ω3). T Calculate the input vector x i Find the prototype vector ω that is closest to the input vector by calculating the Euclidean distance from all prototype vectors. c ;

[0036] (3) Dynamic update: If ω c The category and the true category label y of the i-th training sample i If the samples are identical, i.e., correctly classified, then the prototype vector is updated to move closer to the sample, enhancing representativeness; if ω c Category and y i If a sample is misclassified, the prototype vector is updated to reduce misclassification. If a sample is misclassified and the classification error exceeds a preset threshold, a prototype vector is dynamically added near the sample. If a prototype vector is used less frequently than the threshold in multiple iterations, the vector is dynamically deleted.

[0037] (4) Learning rate decay: Update the learning rate η(t) according to the preset decay formula;

[0038] (5) Repeat the above steps until convergence.

[0039] Furthermore, in step S400, an adaptive dual-delay deep deterministic strategy gradient reinforcement learning algorithm is used to realize the power allocation between the multi-stack fuel cell power generation system and the lithium battery. The reward function of the agent is set to include system hydrogen consumption, performance degradation of fuel cells and lithium batteries, fuel cell efficiency, and lithium battery SOC fluctuation penalty term.

[0040] Taking into account the synergistic effect of different power sources' lifespans, that is, comprehensively considering the performance degradation of fuel cells and lithium batteries as well as their mutual influence, the fuel cell performance degradation evaluation formula is as follows:

[0041]

[0042] Among them, D FC For the overall performance degradation of multi-stack fuel cell systems, D FC,j Let N be the number of fuel cells and D be the number of fuel cells. low,j For the performance degradation caused by the j-th fuel cell idling, D high,j D represents the performance degradation caused by the j-th fuel cell operating at full load. chg,j For the performance degradation caused by the j-th fuel cell variable load operation, D on / off,j The performance degradation caused by the start-up and shutdown of the j-th fuel cell;

[0043] The formula for evaluating the performance degradation of lithium batteries is:

[0044]

[0045] Among them, D bat This refers to the performance degradation of lithium batteries, where t is the operating time and T is the battery's operating temperature. As a factor accelerating the decline, P bat For lithium battery charging and discharging power, I bat E is the current of the lithium battery. bat For battery capacity, This represents the equivalent lifetime cycle count.

[0046] The cooperation expression for both is:

[0047] D sys =[w FC D FC p +w bat D bat p ] 1 / p ;

[0048] Among them, D sys For system performance degradation, w FCand w bat These are weighting factors reflecting the synergistic effect of the two declines, where p is the power exponent, controlling the sensitivity to larger declines during aggregation.

[0049] Fuel cell efficiency is characterized by a smoothing exponential switching of an index consisting of its optimal operating range and the power at its maximum efficiency point. The evaluation formula is as follows:

[0050]

[0051] in,

[0052] Among them, E FC To constrain the efficiency range of fuel cells, P FC_MEP The maximum efficiency point power of the fuel cell, P2 and P1 are the upper and lower limits of the optimal efficiency operating range, respectively, σ(P) is the power smoothing coefficient, k is the smoothing switching coefficient, P is the fuel cell power, and d is the smoothing offset threshold.

[0053] The evaluation formula for the SOC fluctuation penalty of lithium batteries is as follows:

[0054]

[0055] Where, α k β is the SOC deviation penalty coefficient corresponding to the k-th working condition category. k The SOC change rate penalty coefficient is the SOC-related factor for the kth operating condition category. tag,k The target SOC value corresponding to the k-th working condition is... For the rate of change of SOC, this term serves as an additional penalty term in the reward function;

[0056] After normalization, the reward function is expressed as:

[0057]

[0058] in, K represents the total hydrogen consumption of the system, and k1, k2, and k3 are normalization coefficients.

[0059] Furthermore, based on the aforementioned reward function, the Pontryagin minimum principle optimization algorithm is used to optimize the power distribution of the system under three operating conditions offline, obtaining the optimal lithium battery SOC fluctuation penalty coefficient α under the three different operating conditions. k SOC change rate penalty coefficient β k and target value SOC tag,k During subsequent training, the above-mentioned limiting parameters will be adaptively and dynamically adjusted based on the real-time classification results of the load conditions, thereby adjusting the reward function in real time to further improve system performance.

[0060] Furthermore, during the training of the reinforcement learning agent, real-time operating condition categories Cat and load demand power P are defined. load Lithium-ion battery SOC and fuel cell output power P FC As an intelligent agent, the change in fuel cell output power ΔP is perceived from the state space of the environment. FC The action space is the output of the intelligent agent.

[0061] Furthermore, the state space is represented as:

[0062] S = {Cat, P} load SOC, P FC};

[0063] The action space is represented as follows:

[0064] A={ΔP FC |ΔP FC ∈[ΔP FCmin ,ΔP FCmax ]}.

[0065] Where, ΔP FCmin ΔP is the lower limit of the power variation of the fuel cell. FCmax This represents the upper limit of the power variation in the fuel cell.

[0066] Furthermore, during reinforcement learning training, an actor main network, two critic main networks, and a corresponding target network are constructed. The dual critic main network takes the minimum value of Q and works with the actor main network to reduce value overestimation. The target actor network adds smooth noise to the target action and works with the dual critic target network to further improve the policy generalization ability. The main network and the target network are decoupled to reduce the maximization of bias. At the same time, delayed policy updates are adopted to stabilize training.

[0067] In addition, a priority experience replay mechanism is adopted for the samples (s) in the experience pool. t ,a t ,s t+1 ,r t Prioritization is assigned by adding a small positive number to the absolute value of the TD error of each experience, and dynamically updating the TD error and priority of each experience as the model is trained.

[0068] With the synergistic effect of various networks, the gradient descent algorithm is iterated multiple times based on the Bellman equation until convergence.

[0069] Furthermore, in step S500, based on the power allocation results of the multi-stack fuel cell power generation system, and according to the performance differences between different stacks, a distributed cooperative control strategy that takes into account performance differences is adopted to dynamically allocate power to each fuel cell unit, and the following function is established to realize dynamic power allocation:

[0070]

[0071] Among them, P dc1(q) and P dc2(q) Let P be the reference output power of the two fuel cells at time q. dc1(0) and P dc2(0) The initial output power is typically set to 0, ΔP dc1(m) and ΔP dc2(m) Let ΔP be the power increment of the two fuel cells at time m. load(m) K represents the load power increment at time m. div(m) Let m be the power increment allocation ratio at time m.

[0072] The expression is as follows:

[0073]

[0074] Where, ΔP dc1(m) and ΔP dc2(m) Let m be the power increment of the two fuel cells at time m. and Let γ represent the health state of the two fuel cells at time m, and let γ be the adjustment factor.

[0075] The beneficial effects of adopting this technical solution are:

[0076] In the identification layer, this invention utilizes a deep dynamic learning vector quantization neural network classifier to classify load conditions in real time based on feature data extracted by a deep autoencoder, in order to cope with various complex and random operating conditions. In the policy layer, based on the real-time classification results from the identification layer, an adaptive dual-delay deep deterministic policy gradient reinforcement learning agent is trained to autonomously make decisions according to changes in environmental conditions, thereby realizing online dynamic power allocation of multi-stack fuel cell hybrid power systems. This allows for online adaptive and targeted decision-making for different complex operating conditions, achieving the optimal online power allocation control law for the hybrid power system.

[0077] This invention utilizes a deep autoencoder to autonomously extract and reduce the dimensionality of operating conditions, and combines this with spectral clustering analysis to automatically capture nonlinear features and complex patterns in the data, identify nonlinear clusters, and improve the accuracy and efficiency of clustering. This method significantly enhances the robustness and stability of subsequent neural network training, while effectively improving the model's convergence speed through feature compression.

[0078] In practical applications, this invention employs a deep dynamic learning vector quantization classifier to achieve real-time classification of load conditions. The dynamic supervised learning mechanism of this classifier further improves classification accuracy and real-time performance by dynamically adding or subtracting prototype vectors and iteratively optimizing prototype positions, thus significantly enhancing the dynamic response capability of the system.

[0079] This invention constructs a gradient reinforcement learning algorithm based on a dual-delay deep deterministic policy, which reduces value overestimation while incorporating a priority experience replay mechanism, making the interaction between the agent and the environment more stable and efficient. The agent's reward function comprehensively considers system hydrogen consumption, fuel cell efficiency, and fuel cell performance degradation, while also fully taking into account the constraints of lithium battery SOC under various operating conditions. This allows the agent to adaptively and dynamically adjust the constraint parameters of the lithium battery SOC penalty term according to changes in the operating condition category. While optimizing the aforementioned comprehensive system indicators, this also more rationally coordinates the power distribution between the lithium battery and the multi-stack fuel cell system, significantly improving the agent's generalization ability and further enhancing the overall operational efficiency of the hybrid power system. Attached Figure Description

[0080] Figure 1 This is a schematic diagram of the adaptive operating condition sensing hierarchical management method for fuel cell hybrid trams according to the present invention.

[0081] Figure 2 This is a schematic diagram of a fuel cell hybrid power system for a tram in an embodiment of the present invention;

[0082] Figure 3 This is a schematic diagram of the framework for hierarchical energy management of fuel cell hybrid trams with adaptive operating condition perception in an embodiment of the present invention. Detailed Implementation

[0083] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described below with reference to the accompanying drawings.

[0084] In this embodiment, as Figure 2 As shown, the fuel cell hybrid power system for trams includes a multi-stack fuel cell power generation system, a lithium battery system, a DC bus, a DC / DC converter, and an energy management unit and control system. The multi-stack fuel cell power generation system consists of two fuel cell stacks, each connected to a unidirectional DC / DC converter in parallel to the DC bus. The lithium battery is connected to a bidirectional DC / DC converter connected to the DC bus. The DSP-based energy management unit and control system collect information such as bus voltage, output current and voltage of the fuel cells and lithium battery, and output current and voltage of the converter. Based on this information, the system generates corresponding control sequence signals through a loaded program to achieve optimal power distribution.

[0085] See Figure 1 As shown, this invention proposes a hierarchical management method for fuel cell hybrid trams based on adaptive operating condition perception, including the following steps:

[0086] S100 configures parameters for the fuel cell hybrid system based on the feasible domain of vehicle parameter matching, determines the topology of the hybrid system, and divides the energy management strategy into an identification layer and a strategy layer according to the control mode.

[0087] In the recognition layer, the S200 uses a sliding window mechanism to extract the time and frequency domain features of the load condition data within the window based on multiple segments of actual load condition data. It trains a deep autoencoder to unsupervisedly compress and reconstruct the extracted features and use them as input to the spectral clustering algorithm. Through cluster analysis, it automatically determines the category of load condition, which includes: base power state, equal power state, and peak power state.

[0088] Based on the feature reconstruction results of the deep autoencoder and the category classification results of spectral clustering in step S200, S300 preprocesses them to form a training dataset, and trains a deep dynamic learning vector quantization neural network classifier offline to achieve fast real-time category classification under complex random conditions.

[0089] In the policy layer, S400 uses the category label output by the deep dynamic learning vector quantization classifier trained in step S300 as one of the input states of the dual-delay deep deterministic policy gradient reinforcement learning agent. It adaptively adjusts the parameters of the lithium battery SOC fluctuation penalty term in the reward function in real time according to the working condition category, and trains the agent using a priority experience replay mechanism to achieve real-time optimal power allocation for the entire hybrid system.

[0090] The S500 assessment characterizes the performance degradation of each fuel cell stack. Based on the degradation status of each stack, it calculates the allocation ratio of its output power increment and adopts a distributed cooperative control strategy that takes into account performance differences to rationally allocate the required power to each fuel cell stack.

[0091] The first step involves determining the parameter configuration, topology, and energy management strategy hierarchy of the hybrid power system, as detailed below:

[0092] S101, Establish a vehicle dynamics model. Based on vehicle dynamics analysis, obtain the maximum traction load power demand under conditions such as maximum acceleration, maximum operating speed, and maximum gradeability. Based on the constraints of the hybrid power system, obtain the feasible region for tram parameter matching, thereby completing the parameter configuration of the hybrid power system and determining the topology of the entire system, as follows: Figure 2 As shown;

[0093] S102 In this embodiment, the energy management strategy of the hybrid power system is divided into an identification layer and a strategy layer according to the control mode.

[0094] The second step involves offline training of a deep dynamic learning vector quantization classifier based on a deep autoencoder-driven spectral clustering algorithm in the recognition layer for real-time load condition classification, as detailed below:

[0095] In step S200, a sliding window mechanism is used to extract time-domain and frequency-domain features such as mean, peak value, main frequency component, and frequency domain energy of the data within the window. A deep autoencoder is trained to compress the high-dimensional feature data into low-dimensional data, which is then used as input for the spectral clustering algorithm.

[0096] The deep autoencoder-driven spectral clustering algorithm includes the following steps:

[0097] (1) Training the deep autoencoder: The input to the deep autoencoder is determined to be the extracted high-dimensional feature data, i.e., X = {x1, x2, ..., x...} n};

[0098] The encoding process of a deep autoencoder can be represented as follows:

[0099]

[0100] Where z is the output of the encoder, Let L be the weight matrix of the encoder layer L. Let f be the bias vector of the Lth layer of the encoder, σ be the activation function, θ be the encoder parameters, and f be the bias vector of the Lth layer of the encoder. θ For encoding functions;

[0101] The decoding process of a deep autoencoder is represented as follows:

[0102]

[0103] in, This is the output of the decoder, i.e., the reconstructed feature vector; Let L be the weight matrix of the decoder's Lth layer. Let σ' be the bias vector of the Lth layer of the decoder, σ′ be the activation function, and φ be the decoder parameters. This is the decoding function;

[0104] The loss function is expressed as:

[0105]

[0106] The first term is the reconstruction error, the second term is the regularization term, and N w X is the number of training samples. i For the i-th input sample, Let be the i-th reconstructed sample, λ be the regularization coefficient, l be the layer index of the deep autoencoder, ranging from 1 to L, and F be the Frobenius norm;

[0107] The gradient descent algorithm is used to iteratively update the model parameters until convergence.

[0108] (2) Constructing a similarity matrix: The similarity between data is calculated using a Gaussian kernel function; the expression for the similarity matrix is:

[0109]

[0110] Among them, S ab The elements of the similarity matrix, Let σ be the Euclidean distance between data points, and σ be the bandwidth parameter of the Gaussian kernel.

[0111] (3) Construct the degree matrix: The degree matrix is ​​a diagonal matrix used to represent the sum of the similarities between each data point and other points, reflecting the connection strength of each node;

[0112] (4) Calculate the Laplacian matrix: Calculate the normalized Laplacian matrix based on the similarity matrix and degree matrix. Its eigenvalues ​​and eigenvectors reveal the grouping structure of the data.

[0113] The expression for the Laplace matrix is:

[0114] L sym =ID -1 / 2 SD -1 / 2 ;

[0115] Among them, L sym Let I be the normalized Laplace matrix, and D be the identity matrix. -1 / 2 It is the square root of the inverse of the degree matrix.

[0116] (5) Eigendecomposition: Perform eigendecomposition on the Laplacian matrix, extract its eigenvalues ​​and eigenvectors, and select the eigenvectors corresponding to the k smallest non-zero eigenvalues, where k is the expected number of clusters (3).

[0117] The matrix expression formed by k eigenvectors is:

[0118] U = [v1, v3, ..., v k ];

[0119] Where U is the eigenvector matrix, [v1, v3, ..., v k [] represents the eigenvectors corresponding to the first k smallest non-zero eigenvalues;

[0120] (6) Dimensionality reduction and clustering: Each row of the feature vector matrix U is regarded as the representation of data points in a low-dimensional space. Traditional clustering algorithms are applied to group the data points and obtain the final category labels.

[0121] In step S300, based on the clustering results of step S200, the labeled feature vectors are normalized to form a training dataset for training a deep dynamic learning vector quantization classifier. The training steps are as follows:

[0122] (1) Network initialization: Initialize the weight vector and learning rate between the input layer and the competition layer with small random numbers, and define the classification error and prototype frequency threshold to dynamically adjust the prototype vector;

[0123] (2) Recent Prototype Identification: Randomly select one sample from each label set to form a prototype vector ω = (ω1, ω2, ω3). T Calculate the input vector x i Find the prototype vector ω that is closest to the input vector by calculating the Euclidean distance from all prototype vectors. c ;

[0124] (3) Dynamic update: If ω c The category and the true category label y of the i-th training sample i If the samples are identical, i.e., correctly classified, then the prototype vector is updated to move closer to the sample, enhancing representativeness; if ω c Category and y i If a sample is misclassified, the prototype vector is updated to reduce misclassification. If a sample is misclassified and the classification error exceeds a preset threshold, a prototype vector is dynamically added near the sample. If a prototype vector is used less frequently than the threshold in multiple iterations, the vector is dynamically deleted.

[0125] (4) Learning rate decay: Update the learning rate η(t) according to the preset decay formula;

[0126] (5) Repeat the above steps until convergence.

[0127] The third step involves using an adaptive dual-delay deep deterministic gradient reinforcement learning algorithm to determine the optimal output power allocation scheme between the multi-stack fuel cell power generation system and the lithium battery, based on the real-time category of the load condition. A distributed cooperative control strategy, taking into account performance differences, is then employed to allocate power to each fuel cell stack, as detailed below:

[0128] In step S400, an adaptive dual-delay deep deterministic strategy gradient reinforcement learning algorithm is used to realize the power allocation between the multi-stack fuel cell power generation system and the lithium battery. The reward function of the agent is set to include system hydrogen consumption, performance degradation of fuel cells and lithium batteries, fuel cell efficiency, and lithium battery SOC fluctuation penalty term.

[0129] Taking into account the synergistic effect of different power sources' lifespans, that is, comprehensively considering the performance degradation of fuel cells and lithium batteries as well as their mutual influence, the fuel cell performance degradation evaluation formula is as follows:

[0130]

[0131] Among them, D FC For the overall performance degradation of multi-stack fuel cell systems, D FC,j Let N be the number of fuel cells and D be the number of fuel cells. low,j For the performance degradation caused by the j-th fuel cell idling, D high,j D represents the performance degradation caused by the j-th fuel cell operating at full load. chg,j For the performance degradation caused by the j-th fuel cell variable load operation, D on / off,j This represents the performance degradation caused by the start-up and shutdown of the j-th fuel cell.

[0132] The formula for evaluating the performance degradation of lithium batteries is:

[0133]

[0134] Among them, D bat This refers to the performance degradation of lithium batteries, where t is the operating time and T is the battery's operating temperature. As a factor accelerating the decline, P bat For lithium battery charging and discharging power, I bat E is the current of the lithium battery. bat For battery capacity, This represents the equivalent lifetime cycle count.

[0135] The cooperation expression for both is:

[0136] D sys =[w FC D FC p +w bat D bat p ] 1 / p ;

[0137] Among them, D sys For system performance degradation, w FC and w bat These are weighting factors reflecting the synergistic effect of the two declines, where p is a power exponent that controls the sensitivity to larger declines during aggregation.

[0138] Fuel cell efficiency is characterized by a smoothing exponential switching of an index consisting of its optimal operating range and the power at its maximum efficiency point. The evaluation formula is as follows:

[0139]

[0140] in,

[0141] Among them, E FC To constrain the efficiency range of fuel cells, P FC_MEP The maximum efficiency point power of the fuel cell, P2 and P1 are the upper and lower limits of the optimal efficiency operating range, respectively, σ(P) is the power smoothing coefficient, k is the smoothing switching coefficient, P is the fuel cell power, and d is the smoothing offset threshold.

[0142] The evaluation formula for the SOC fluctuation penalty of lithium batteries is as follows:

[0143]

[0144] Where, α k β is the SOC deviation penalty coefficient corresponding to the k-th working condition category. k The SOC change rate penalty coefficient is the SOC-related factor for the kth operating condition category. tag,k The target SOC value corresponding to the k-th working condition is... For the rate of change of SOC, this term serves as an additional penalty term in the reward function.

[0145] After normalization, the reward function is expressed as:

[0146]

[0147] in, K represents the total hydrogen consumption of the system, and k1, k2, and k3 are normalization coefficients.

[0148] The following constraints are set for the powertrain system:

[0149]

[0150] Among them, P FC_j Let P be the output power of the j-th fuel cell. FC_min and P FC_max ΔP represents the lower and upper limits of the fuel cell output power. FC P represents the rate of change of fuel cell power. bat This refers to the power of the lithium battery.

[0151] Based on the aforementioned reward function, the Pontryagin minimum principle optimization algorithm is used to optimize the power distribution of the system under three operating conditions offline, obtaining the optimal lithium battery SOC fluctuation penalty coefficient α under the three different operating conditions. k SOC change rate penalty coefficient β k and target value SOC tag,kDuring subsequent training, the above-mentioned limiting parameters will be adaptively and dynamically adjusted based on the real-time classification results of the load conditions, thereby adjusting the reward function in real time to further improve system performance.

[0152] During the training of a reinforcement learning agent, the real-time operating condition category Cat and the load demand power P are defined. load Lithium-ion battery SOC and fuel cell output power P FC As an intelligent agent, the change in fuel cell output power ΔP is perceived from the state space of the environment. FC The action space is the output of the intelligent agent.

[0153] The state space is represented as:

[0154] S = {Cat, P} load SOC, P FC};

[0155] The action space is represented as:

[0156] A={ΔP FC |ΔP FC ∈[ΔP FCmin ,ΔP FCmax ]}.

[0157] Where, ΔP FCmin ΔP is the lower limit of the power variation of the fuel cell. FCmax This represents the upper limit of the power variation in the fuel cell.

[0158] During reinforcement learning training, an actor main network, two critic main networks, and a corresponding target network are constructed. The dual critic main network takes the minimum value of Q and works with the actor main network to reduce value overestimation. The target actor network adds smooth noise to the target action and works with the dual critic target network to further improve the policy generalization ability. The main network and the target network are decoupled to reduce the maximization of bias. At the same time, delayed policy updates are adopted to stabilize training.

[0159] In addition, a priority experience replay mechanism is adopted for the samples (s) in the experience pool. t ,a t ,s t+1 ,r t Priorities are assigned by adding a small positive number to the absolute value of the TD error of each experience. As the model is trained, the TD error and priority of each experience are dynamically updated.

[0160] The expression for TD error is as follows:

[0161] δ=r+αmax a′ Q(s′,a′)-Q(s,a);

[0162] Where r is the reward, α is the discount factor, Q(s,a) is the Q-value of the current state-action pair, and max a′ Q(s′,a′) is the maximum Q value of the next state.

[0163] The priority definition formula is as follows:

[0164] p t =|δ t |+ò;

[0165] Where, δ t Let t be the TD error of the t-th experience, and ò be a small constant.

[0166] Through the collaborative efforts of various networks, the agent can learn key experiences more efficiently, significantly improving learning efficiency and stability. Furthermore, by iterating through multiple gradient descent algorithms according to the Bellman equation until convergence, the agent's performance can be continuously optimized.

[0167] In step S500, based on the power allocation results of the multi-stack fuel cell power generation system, and according to the performance differences between different stacks, a distributed cooperative control strategy that takes into account performance differences is adopted to dynamically allocate power to each fuel cell unit. The following function is established to realize dynamic power allocation:

[0168]

[0169] Among them, P dc1(q) and P dc2(q) Let P be the reference output power of the two fuel cells at time q. dc1(0) and P dc2(0) The initial output power is typically set to 0, ΔP dc1(m) and ΔP dc2(m) Let ΔP be the power increment of the two fuel cells at time m. load(m) K represents the load power increment at time m. div(m) Let m be the power increment allocation ratio at time m.

[0170] The expression is as follows:

[0171]

[0172] Where, ΔP dc1(m) and ΔP dc2(m) Let m be the power increment of the two fuel cells at time m. and Let γ represent the health state of the two fuel cells at time m, and let γ be the adjustment factor.

[0173] Based on this, the lower-performance fuel cell stacks output less power, while the higher-performance fuel cell stacks output more power, thus balancing the performance differences among the various fuel cell stacks within the system and extending the service life of the entire system.

[0174] In this embodiment, the architecture of the entire energy management strategy, as summarized above, is as follows: Figure 3 As shown, it is divided into an identification layer and a strategy layer. The core of the identification layer is the training of a learning vector quantization classifier, and the core of the strategy layer is the reinforcement learning algorithm. The trained classifier is applied online to the strategy layer. The optimal SOC control parameters under various operating conditions are optimized in advance by applying the minimum principle. The corresponding parameters are adaptively adjusted according to the real-time operating condition category to realize the online dynamic power allocation of the system.

[0175] This invention fully considers the impact of complex and uncertain operating conditions on tram energy management. At the recognition layer, it reshapes operating condition features through spectral clustering driven by a deep autoencoder and assigns category labels. A deep dynamic learning vector quantization classifier is used to classify load conditions in real time into three categories: base power state, balanced power state, and peak power state, and targeted responses are made for each category. At the policy layer, a dual-delay deep deterministic policy gradient reinforcement learning algorithm is employed. Based on the output category of the recognition layer, the parameters of the SOC penalty term in the reward function are adaptively and dynamically adjusted. Combined with a priority experience replay mechanism, the reinforcement learning agent is trained through continuous interaction with the environment to determine the optimal power allocation scheme for the system, achieving optimal power allocation between the multi-stacking fuel cell power generation system and the lithium battery. A distributed cooperative control strategy considering performance differences is adopted. Taking into account the performance differences between different fuel cell stacks, power is adaptively allocated according to the performance degradation degree of each stack, ensuring that the operating performance of each stack remains as consistent as possible and extending the service life of the entire system. This method significantly improves the adaptability of tram energy management strategies under different load conditions by optimizing the power flow allocation of the entire hybrid power system.

[0176] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A hierarchical management method for fuel cell hybrid trams with adaptive operating condition perception, characterized in that, Including the following steps: S100 configures parameters for the fuel cell hybrid system based on the feasible domain of vehicle parameter matching, determines the topology of the hybrid system, and divides the energy management strategy into an identification layer and a strategy layer according to the control mode. In the recognition layer, the S200 uses a sliding window mechanism to extract the time and frequency domain features of the load condition data within the window based on multiple segments of actual load condition data. It trains a deep autoencoder to unsupervisedly compress and reconstruct the extracted features and use them as input to the spectral clustering algorithm. Through cluster analysis, it automatically determines the category of load condition, which includes: base power state, equal power state, and peak power state. Based on the feature reconstruction results of the deep autoencoder and the category classification results of spectral clustering in step S200, S300 preprocesses them to form a training dataset, and trains a deep dynamic learning vector quantization neural network classifier offline to achieve fast real-time category classification under complex random conditions. In the policy layer, S400 uses the category label output by the deep dynamic learning vector quantization classifier trained in step S300 as one of the input states of the dual-delay deep deterministic policy gradient reinforcement learning agent. It adaptively adjusts the parameters of the lithium battery SOC fluctuation penalty term in the reward function in real time according to the working condition category, and trains the agent using a priority experience replay mechanism to achieve real-time optimal power allocation for the entire hybrid system. An adaptive dual-delay deep deterministic gradient reinforcement learning algorithm is adopted to realize the power allocation between multi-stack fuel cell power generation system and lithium battery. The reward function of the agent is set to include system hydrogen consumption, performance degradation of fuel cell and lithium battery, fuel cell efficiency and lithium battery SOC fluctuation penalty term. Taking into account the synergistic effect of different power sources' lifespans, that is, comprehensively considering the performance degradation of fuel cells and lithium batteries as well as their mutual influence, the fuel cell performance degradation evaluation formula is as follows: ； Among them, D FC For the overall performance degradation of multi-stack fuel cell systems, D FC,j Let N be the number of fuel cells and D be the number of fuel cells. low,j For the performance degradation caused by the j-th fuel cell idling, D high,j D represents the performance degradation caused by the j-th fuel cell operating at full load. chg,j For the performance degradation caused by the j-th fuel cell variable load operation, D on / off,j The performance degradation caused by the start-up and shutdown of the j-th fuel cell; The formula for evaluating the performance degradation of lithium batteries is: ； Among them, D bat This refers to the performance degradation of lithium batteries, where t is the operating time and T is the battery's operating temperature. As a factor accelerating the decline, P bat For lithium battery charging and discharging power, I bat E is the current of the lithium battery. bat For battery capacity, This represents the equivalent lifetime cycle count. The cooperation expression for both is: ； Among them, D sys For system performance degradation, and These are weighting factors reflecting the synergistic effect of the two declines, where p is the power exponent, controlling the sensitivity to larger declines during aggregation. Fuel cell efficiency is characterized by a smoothing exponential switching of an index consisting of its optimal operating range and the power at its maximum efficiency point. The evaluation formula is as follows: ； in, ; Among them, E FC To constrain the efficiency range of fuel cells, P FC_MEP The maximum efficiency point power of the fuel cell, P2 and P1 are the upper and lower limits of the optimal efficiency operating range, respectively, σ(P) is the power smoothing coefficient, k is the smoothing switching coefficient, P is the fuel cell power, and d is the smoothing offset threshold. The evaluation formula for the SOC fluctuation penalty of lithium batteries is as follows: ； Where, α k β is the SOC deviation penalty coefficient corresponding to the k-th working condition category. k The SOC change rate penalty coefficient is the SOC-related factor for the kth operating condition category. tag,k The target SOC value corresponding to the k-th working condition is... For the rate of change of SOC, this term serves as an additional penalty term in the reward function; After normalization, the reward function is expressed as: ； in, The total hydrogen consumption of the system is given by k1, k2, and k3, which are normalization coefficients. The S500 assessment characterizes the performance degradation of each fuel cell stack. Based on the degradation status of each stack, it calculates the allocation ratio of its output power increment and adopts a distributed cooperative control strategy that takes into account performance differences to rationally allocate the required power to each fuel cell stack.

2. The adaptive operating condition sensing hierarchical management method for fuel cell hybrid trams according to claim 1, characterized in that, In step S200, a sliding window mechanism is used in the recognition layer to extract the mean, peak value, main frequency component, and frequency domain energy of the data within the window. The deep autoencoder is trained to compress the high-dimensional feature data into low-dimensional data, which is then used as the input to the spectral clustering algorithm.

3. The adaptive operating condition sensing hierarchical management method for fuel cell hybrid trams according to claim 2, characterized in that, The deep autoencoder-driven spectral clustering algorithm includes the following steps: (1) Training the deep autoencoder: The input to the deep autoencoder is determined to be the extracted high-dimensional feature data, i.e. ; The encoding process of a deep autoencoder can be represented as follows: ； Where z is the output of the encoder, Let L be the weight matrix of the encoder layer L. Let σ be the bias vector of the Lth layer of the encoder, σ be the activation function, and θ be the encoder parameters. For encoding functions; The decoding process of a deep autoencoder is represented as follows: ； in, This is the output of the decoder, i.e., the reconstructed feature vector; Let L be the weight matrix of the decoder's Lth layer. Let L be the bias vector of the decoder's Lth layer. For activation function, For decoder parameters, This is the decoding function; The loss function is expressed as: ； The first term is the reconstruction error, the second term is the regularization term, and N w X is the number of training samples. i For the i-th input sample, Let be the i-th reconstructed sample, λ be the regularization coefficient, l be the layer index of the deep autoencoder, ranging from 1 to L, and F be the Frobenius norm; The gradient descent algorithm is used to iteratively update the model parameters until convergence. (2) Construct a similarity matrix: Calculate the similarity between data using the Gaussian kernel function; (3) Construct the degree matrix: The degree matrix is ​​a diagonal matrix used to represent the sum of the similarities between each data point and other points, reflecting the connection strength of each node; (4) Calculate the Laplace matrix: Calculate the normalized Laplace matrix based on the similarity matrix and degree matrix. Its eigenvalues ​​and eigenvectors reveal the grouping structure of the data. (5) Eigenvalue decomposition: Perform eigenvalue decomposition on the Laplacian matrix, extract its eigenvalues ​​and eigenvectors, and select the eigenvectors corresponding to the k smallest non-zero eigenvalues; The matrix expression formed by k eigenvectors is: ； Where U is the eigenvector matrix. The eigenvectors corresponding to the first k smallest non-zero eigenvalues; (6) Dimensionality reduction and clustering: Each row of the feature vector matrix U is regarded as the representation of the data points in the low-dimensional space. Traditional clustering algorithms are applied to group the data points and obtain the final category labels.

4. The adaptive operating condition sensing hierarchical management method for fuel cell hybrid trams according to claim 1, characterized in that, In step S300, based on the clustering results of step S200, the labeled feature vectors are normalized to form a training dataset for training a deep dynamic learning vector quantization classifier. The training steps are as follows: (1) Network initialization: Initialize the weight vector and learning rate between the input layer and the competition layer with small random numbers, and define the classification error and prototype frequency threshold to dynamically adjust the prototype vector; (2) Prototype identification: Randomly select a sample from each label set to form a prototype vector. Calculate the input vector x i Find the prototype vector ω that is closest to the input vector by calculating the Euclidean distance from all prototype vectors. c ; (3) Dynamic update: If ω c The category and the true category label y of the i-th training sample i If the samples are identical, i.e., correctly classified, then the prototype vector is updated to move closer to the sample, enhancing representativeness; if ω c Category and y i If the classification is incorrect, then the prototype vector should be updated to reduce misclassification. If a sample is misclassified and the classification error exceeds a preset threshold, a prototype vector is dynamically added near the sample; if a prototype vector is used less frequently than the threshold in multiple iterations, the vector is dynamically deleted. (4) Learning rate decay: Update the learning rate η(t) according to the preset decay formula; (5) Repeat the above steps until convergence.

5. The adaptive operating condition sensing hierarchical management method for fuel cell hybrid trams according to claim 1, characterized in that, Based on the aforementioned reward function, the Pontryagin minimum principle optimization algorithm is used to optimize the power distribution of the system under three operating conditions offline, obtaining the optimal lithium battery SOC fluctuation penalty coefficient α under the three different operating conditions. k SOC change rate penalty coefficient β k and target value SOC tag,k During subsequent training, the aforementioned limiting parameter α will be adaptively and dynamically adjusted based on the real-time classification results of the load conditions. k β k and SOC tag,k This allows for real-time adjustment of the reward function to further improve system performance.

6. The adaptive operating condition sensing hierarchical management method for fuel cell hybrid trams according to claim 5, characterized in that, During the training of a reinforcement learning agent, the real-time operating condition category Cat and the load demand power P are defined. load Lithium-ion battery SOC and fuel cell output power P fc As an intelligent agent, the change in the output power of a fuel cell is perceived from its state space. The action space is the output of the intelligent agent.

7. The adaptive operating condition sensing hierarchical management method for fuel cell hybrid trams according to claim 6, characterized in that, The state space is represented as follows: ； The action space is represented as follows: ； in, This represents the lower limit of the power variation in the fuel cell. This represents the upper limit of the power variation in the fuel cell.

8. The adaptive operating condition sensing hierarchical management method for fuel cell hybrid trams according to claim 6, characterized in that, During reinforcement learning training, an actor main network, two critic main networks, and a corresponding target network are constructed. The dual critic main network takes the minimum value of Q and works with the actor main network to reduce value overestimation. The target actor network adds smooth noise to the target action and works with the dual critic target network to further improve the policy generalization ability. The main network and the target network are decoupled to reduce the maximization of bias. At the same time, delayed policy updates are adopted to stabilize training. In addition, a priority experience replay mechanism is adopted for samples in the experience pool. Prioritization involves adding a small positive number to the absolute value of the TD error of each experience, and dynamically updating the TD error and priority of each experience as the model is trained. With the synergistic effect of various networks, the gradient descent algorithm is iterated multiple times based on the Bellman equation until convergence.

9. The adaptive operating condition sensing hierarchical management method for fuel cell hybrid trams according to claim 1, characterized in that, In step S500, based on the power allocation results of the multi-stack fuel cell power generation system, and according to the performance differences between different stacks, a distributed cooperative control strategy that takes into account performance differences is adopted to dynamically allocate power to each fuel cell unit. The following function is established to realize dynamic power allocation: ； ； Among them, P dc1(q) and P dc2(q) Let P be the reference output power of the two fuel cells at time q. dc1(0) and P dc2(0) This is the initial output power, which is typically set to 0. and Let m be the power increment of the two fuel cells at time m. K represents the load power increment at time m. div(m) Let m be the power increment allocation ratio at time m. The expression is as follows: ； in, and Let m be the power increment of the two fuel cells at time m. and Let m be the health status of the two fuel cells. It is a regulating factor.

Images