Intelligent energy-saving method of fuel cell vehicle for adaptively correcting black box model behavior

Abstract

The invention provides a fuel cell vehicle intelligent energy-saving method for adaptively correcting black box model behaviors, which comprises the following steps: constructing a typical vehicle reinforcement learning control model, and constructing a traditional fuel cell hybrid electric vehicle energy management strategy; extracting model driving behavior characteristics of the control model through an LSTM self-encoding network, and constructing a novel model state space by fusing an energy management strategy; based on the characteristics of real-time energy management, establishing a strategy framework fusing optimization and reinforcement learning, nesting an ECMS strategy into a reinforcement learning algorithm, establishing an integrated hierarchical energy management architecture, and reducing the scale of a decision model; an SAC algorithm is used as a decision model, a novel model state space is fused, an energy management strategy is used as a reference reward, a novel reward function is constructed, and the relation between energy consumption of a typical vehicle reinforcement learning control model in an ecological driving strategy and an action decision is accurately captured. The energy consumption of the fuel cell hybrid electric vehicle is reduced, and the deployment efficiency of the ecological driving strategy is improved.

Description

Technical Field

[0001] This invention relates to the field of vehicle energy management technology, and specifically to an intelligent energy-saving method for fuel cell vehicles that adaptively corrects the behavior of a black-box model. Background Technology

[0002] As a major source of global energy consumption and carbon emissions, the transportation industry is facing increasingly severe energy and environmental challenges. Statistics show that road traffic accounts for over 20% of global energy consumption and contributes approximately 23% of carbon dioxide emissions. Against this backdrop, fuel cell vehicles (FCVs), with their advantages of zero emissions and renewable fuels, are considered a key technological path to achieving a green transportation transformation. However, despite their superior environmental performance, the lack of economic viability remains a core bottleneck restricting their large-scale commercialization. Research indicates that the manufacturing cost and operating energy consumption of FCVs are significantly higher than those of traditional gasoline-powered vehicles and pure electric vehicles. The inefficiency of energy management strategies directly impacts the overall vehicle energy efficiency and powertrain lifespan, thereby increasing the total lifecycle cost.

[0003] With the development of connected technologies, current vehicle energy management technology is no longer limited to optimizing the energy consumption of the vehicle itself. Layered energy management has become the main way to deal with the relationship between driving strategies and energy consumption. Among them, reinforcement learning is widely used in this field. The formulation of ecological driving strategies is no longer limited to the control method of traditional black box models. How to reasonably reduce the model deployment and enhance the interpretability of ecological driving strategies has become a key factor in improving FCV energy management. Summary of the Invention

[0004] This invention proposes an intelligent energy-saving method for fuel cell vehicles that adaptively corrects the behavior of a black-box model. It designs a nested reinforcement learning framework focused on a novel state space to handle real-time energy management strategies, constructing an integrated architecture for the collaborative optimization of ecological driving and energy management. Based on the SAC algorithm, a novel state space is designed. Based on the control characteristics of different vehicle decisions, model driving behavior features are extracted, enhancing the interpretability of the reinforcement learning state-action-reward mapping relationship. A novel reward function is introduced, using energy management of typical driving strategies as a benchmark, effectively capturing the relationship between model energy consumption and action decisions.

[0005] In a first aspect, this invention provides an intelligent energy-saving method for fuel cell vehicles that adaptively corrects the behavior of a black-box model, comprising the following steps: S1, constructing a typical vehicle reinforcement learning control model by building a state space, an action space, and a reward function, while simultaneously constructing a traditional fuel cell hybrid vehicle energy management strategy; S2, based on the decision-making and control characteristics of different vehicles, extracting the model driving behavior features of the typical vehicle reinforcement learning control model through an LSTM autoencoder network, and fusing the model driving behavior features with the traditional fuel cell hybrid vehicle energy management strategy to construct a new model state space, thereby enhancing the interpretability of the reinforcement learning state-action-reward mapping relationship; S3, based on the characteristics of real-time energy management, establishing a strategy framework that integrates optimization and reinforcement learning, nesting the ECMS strategy with reinforcement learning algorithms, and establishing an integrated hierarchical energy management architecture to reduce the size of the decision model; S4, based on the strategy framework that integrates optimization and reinforcement learning and the integrated hierarchical energy management architecture, using the SAC algorithm as the decision model, fusing the new model state space and using the traditional fuel cell hybrid vehicle energy management strategy as the benchmark reward, constructing a new reward function to accurately capture the relationship between energy consumption and action decisions of the typical vehicle reinforcement learning control model in the ecological driving strategy.

[0006] Optionally, the state space of the energy management strategy for the conventional fuel cell hybrid electric vehicle includes the characteristics of the vehicle's powertrain, the action is the output power of the fuel cell, and the reward function includes the degradation cost of the powertrain, expressed as:

[0007] S ego =[v,a,SOC,P motor ,P FCE ]

[0008] a eco =[-4,-3,-2,-1,0,1,2,3,4]

[0009]

[0010] Where v is the vehicle's speed, a is the acceleration, SOC is the current state of charge (SOC) value, and P... motor P is the instantaneous power demand of the vehicle motor. FCE For the instantaneous power demand of the vehicle's fuel cell, a eco R represents the output power of the fuel cell. cost (s k ,a k ) represents the instantaneous equivalent hydrogen consumption cost of a vehicle, r soc (s k ,a k) represents the offset penalty of the vehicle's instantaneous SOC relative to the reference SOC value, and α1 and α2 represent the weight values ​​of the two costs, respectively.

[0011] Optionally, the network structure of the LSTM autoencoder network consists of a forget gate, an input gate, a cell state update gate, and an output gate, and the network structure is represented as follows:

[0012] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0013] i t =σ(W i ·[h t-1 ,x t ]+b i )

[0014]

[0015] O t =v(W o ·[h t-1 ,x t ]+b o )

[0016] h t =O t ⊙tanh(C t )

[0017] Among them, f t The input to the forget gate is represented by W and b, where W and b represent the weight matrix and bias term, respectively, and σ and tanh represent the activation functions, respectively. t-1 x represents the hidden state at the previous time step. t For the input at the current time step, i t Indicates the input gate output. Representing candidate memories, C t Indicates the updated cell state, O t Indicates the output gate;

[0018] The latent vector is concatenated with the vehicle dynamics feature vector to form a new state space, represented as:

[0019] S ECO =[v,a,Soc,P motor ,P FCE ,h t ]

[0020] Where v is the vehicle speed, a is the vehicle acceleration, Soc is the vehicle's state of charge, and P... motor P is the power required by the motor.FCE For the output power of the fuel cell, h t State variables that characterize the driving behavior features of the model.

[0021] Optionally, the LSTM autoencoder network is updated using a reconstruction loss function, which is expressed as:

[0022]

[0023] Where, x i Indicates input data, This indicates the reconstruction of data.

[0024] Optionally, the LSTM autoencoder network compresses high-dimensional matrix information into low-dimensional latent vectors and performs clustering verification using the K-means clustering method, which is expressed as follows:

[0025]

[0026] Where K is the preset number of clusters, C i For all data points in the i-th cluster, μ i Let x be the center point of the i-th cluster, and let x be the low-dimensional latent vector encoded by LSTM.

[0027] Optionally, the decision-making behavior of the ECMS strategy is represented as follows:

[0028]

[0029] Where, m fce (P fce For fuel cells, the fuel consumption rate, P bat Where λ is the battery power, η is the equivalent factor, and η is the equivalence factor. bat For battery charging and discharging efficiency, This refers to the calorific value of hydrogen.

[0030] The equivalent factor λ is dynamically adjusted to regulate the economic weights of electricity and hydrogen energy consumption, expressed as follows:

[0031] λ(t)=λ0+k×(SOC(t)-SOC target )

[0032] Where λ0 is the basic equivalent factor, k is the feedback gain coefficient, and the adjustment strength controls the SOC when it deviates from the target. target The target power level.

[0033] Optionally, the decision-making behavior of the ECMS strategy is represented as follows:

[0034] The ECMS strategy is embedded in a hierarchical energy management architecture as a lower-level strategy controller for real-time energy management. This provides real-time reward function feedback for the upper-level reinforcement learning-based driving action control. The driving behavior characteristics of the typical vehicle reinforcement learning control model are integrated into the state dimension and combined with the vehicle's power system characteristics to form the reinforcement learning state space. The output action, the output power of the fuel cell, can be expressed as:

[0035] S ego =[v,a,SOC,P motor ,P FCE ,h t ]

[0036] Where v is the vehicle speed, a is the vehicle acceleration, SOC is the vehicle state of charge, and P motor P is the power required by the motor. FCE For the output power of the fuel cell, h t To characterize the state variables of the model's driving behavior features;

[0037] Eco-driving output is the vehicle's acceleration (m / s²). 2 This can be represented as:

[0038] a eco = [-1.5, -1.0, -0.5, 0, 0.5, 1.0, 1.5]

[0039] The motion is discrete in seven dimensions, with an interval of 0.5 m / s. 2 Limited to [-1.5, 1.5] m / s 2 between;

[0040] The reward function for eco-driving can be expressed as:

[0041]

[0042] Where, r cost (s k ,a k ) represents the instantaneous equivalent hydrogen consumption cost of a vehicle, r soc (s k ,a k ) represents the offset penalty between the vehicle's instantaneous SOC and the reference SOC value, r gap (s k ,a k The value represents the difference between the minimum equivalent hydrogen consumption and the equivalent hydrogen consumption under typical operating conditions.

[0043] Optionally, the eco-driving strategy combines the classic reward mechanism of equivalent hydrogen consumption and battery degradation, integrating the difference between hydrogen consumption and the hydrogen consumption incurred by the expert strategy to form the novel reward function to accelerate the optimization of the eco-driving strategy, which can be expressed as:

[0044]

[0045] r cost (s k ,a k ) = J fce +J bat

[0046] r soc (s k ,a k )=|SOC(k)-SOC ref |

[0047] r gap (s k ,a k ) = Cost ems -Cost eco

[0048] Among them, J bat J represents the equivalent hydrogen consumption cost during battery charging and discharging. fce The SOC(k) represents the hydrogen consumption cost of the fuel cell, and SOC(k) represents the current SOC value. ref For reference to SOC value, Cost ems Cost represents the hydrogen consumption value for a conventional energy management strategy. eco This represents the hydrogen consumption value for the eco-driving strategy.

[0049] In a second aspect, the present invention provides an electronic device, including a processor and a memory storing a program. The program includes instructions that, when executed by the processor, cause the processor to perform the steps performed by the method described in the first aspect.

[0050] Thirdly, the present invention provides a computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect above.

[0051] Compared with the prior art, the beneficial effects of the present invention include:

[0052] (1) This invention extracts driving behavior features of the model through LSTM autoencoder unsupervised learning, establishes a new state space, and enhances the interpretability of the state-action-reward mapping relationship of reinforcement learning.

[0053] (2) This invention establishes an integrated hierarchical energy management architecture by embedding the ECMS strategy into SAC reinforcement learning, thereby reducing the size of the decision model.

[0054] (3) This invention strengthens the mapping effect of model state-reward by using the energy management results of typical strategy models as the benchmark reward for adaptive correction of model strategies, effectively captures the relationship between energy consumption and model action decisions, realizes the synergistic optimization of ecological driving and energy management, and reduces FCB energy consumption and power system life degradation. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is a flowchart of the steps of the present invention;

[0057] Figure 2 To and Figure 1 The corresponding technical roadmap of this invention.

[0058] Figure 3 This is a roadmap for extracting driving behavior features according to the present invention. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] It should be noted that the brief descriptions of terminology in this invention are merely for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this invention. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0061] See Figure 1 The present invention provides an intelligent energy-saving method for fuel cell vehicles that adaptively corrects the behavior of a black-box model, comprising the following steps:

[0062] S1. By constructing the state space, action space, and reward function, a typical vehicle reinforcement learning control model is built, and a traditional fuel cell hybrid electric vehicle energy management strategy is also constructed.

[0063] S2. Based on the decision-making and control characteristics of different vehicles, the driving behavior features of the typical vehicle reinforcement learning control model are extracted through the LSTM autoencoder network, and the driving behavior features of the model are fused with the energy management strategy of the traditional fuel cell hybrid electric vehicle to construct a new model state space, so as to enhance the interpretability of the reinforcement learning state-action-reward mapping relationship.

[0064] S3. Based on the characteristics of real-time energy management, a policy framework integrating optimization and reinforcement learning is established. The ECMS policy is nested with reinforcement learning algorithms to establish an integrated hierarchical energy management architecture, thereby reducing the size of the decision model.

[0065] S4. Based on the strategy framework of fusion optimization and reinforcement learning and the integrated hierarchical energy management architecture, using the SAC algorithm as the decision model, the new model state space is fused and the traditional fuel cell hybrid electric vehicle energy management strategy is used as the benchmark reward to construct a new reward function, so as to accurately capture the relationship between energy consumption and action decision of the typical vehicle reinforcement learning control model in the ecological driving strategy.

[0066] Furthermore, in step S1, a typical vehicle reinforcement learning control model is established, the state and action space and reward function are constructed, and a traditional fuel cell hybrid electric vehicle energy management strategy is established.

[0067] By establishing three typical vehicle control strategies in the SUMO simulation software, the vehicle's state in the current scenario is recorded in real time, and the working conditions of the three typical strategies are spliced ​​together as a training dataset. The target model is optimized by performing longitudinal dynamic mathematical modeling using Python, and SAC is selected as the optimization algorithm. Energy management strategies are developed on the training dataset.

[0068] SAC (soft-actor-critic), an important branch of the Actor-Critic algorithm framework, is a relatively new algorithm developed in recent years. It introduces entropy regularization to encourage the policy model to explore, maximizing both cumulative reward and maximum entropy. Considering the vehicle's dynamic characteristics as state input, power allocation of the power system as state output, and balancing energy consumption and power system degradation as the objective reward, a target energy management policy is constructed.

[0069] The loss functions for the SAC value network, decision network, temperature parameter, and network weight update formulas are as follows:

[0070]

[0071]

[0072] θ′ i =τθ i +(1-τ)θ′ i i = 1, 2

[0073] Where, θ i For the current network, Let be the policy network, s′ be the state at the next time step, r be the reward for the current state, and y be the target Q value. Let α be the entropy of the action, and α be the temperature hyperparameter. To represent the action at the next time step, avoiding gradient non-differentiability caused by direct sampling, θ′ i For the target network, θ i τ represents the current network, and τ represents the update rate.

[0074] The SAC's state input needs to fully consider the vehicle's driving state, and can be represented as:

[0075] S ego =[v,a,SOC,P motor ,P FCE ]

[0076] Where v is the vehicle's speed, a is the acceleration, SOC is the current state of charge (SOC) value, and P... motor P is the instantaneous power demand of the vehicle motor. FCE This refers to the instantaneous power demand of the vehicle's fuel cell.

[0077] SAC uses the power distribution of the power system as its output, which is generally the discrete output power (kW) of the fuel cell:

[0078] a eco =[-4,-3,-2,-1,0,1,2,3,4]

[0079] Among them, a eco This represents the output power of the fuel cell, measured in kW.

[0080] The primary objective of energy management is to optimize economic losses at the operating level, including energy consumption and power system lifespan degradation. Therefore, the reward function can be expressed as:

[0081]

[0082] Where, r cost (s k ,a k ) represents the instantaneous equivalent hydrogen consumption cost of a vehicle, r soc (s k ,a k) represents the offset penalty of the vehicle's instantaneous SOC relative to the reference SOC value. α1 and α2 represent the weight values ​​of the two costs, which need to be adjusted according to the actual order of magnitude.

[0083] Furthermore, in step S2, based on the different vehicle decision-making and control characteristics, the driving behavior features of the model are extracted, and a new model state space is built by integrating traditional strategies to enhance the interpretability of the reinforcement learning state-action-reward mapping relationship.

[0084] This invention employs an LSTM autoencoder network to identify driving decision-making behavior. The principle of LSTM is to process time-series data through a gating mechanism, extract features and output encoded vectors through window sliding, thereby achieving feature recognition of driving modes. The composition of the LSTM network can be represented as follows:

[0085] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0086] ρ t =σ(W i ·[h t-1 ,x t ]+b i )

[0087]

[0088] O t =σ(W o ·[h t-1 ,x t ]+b o )

[0089] h t =O t ⊙tanh(C t )

[0090] Among them, f t The input to the forget gate is represented by W and b, where W and b represent the weight matrix and bias term, respectively, and σ and tanh represent the activation functions, respectively. t-1 x represents the hidden state at the previous time step. t For the input at the current time step, i t Indicates the input gate output. Representing candidate memories, C t Indicates the updated cell state, O t This indicates the output gate.

[0091] The weights of the LSTM network are updated by reconstructing the loss function using the mean squared error. The loss function can be expressed as:

[0092]

[0093] Where, x i Indicates input data, This indicates the reconstruction of data.

[0094] Transform high-dimensional input into low-dimensional latent features h t This, together with the eigenvectors of the dynamical system, forms a novel state space, which can be represented as follows:

[0095] S ECO =[v,a,Soc,P motor ,P FCE ,h t ]

[0096] Where v is the vehicle speed, a is the vehicle acceleration, Soc is the vehicle's state of charge, and P... motor P is the power required by the motor. FCE For the output power of the fuel cell, h t State variables are used to characterize the driving behavior features of the model.

[0097] The trained network is validated using training data, and the K-means method is used to reduce the dimensionality of the encoding. Then, a visual clustering output of driving style is generated through supervised learning in a regression-classification manner. The principle behind this can be described as follows:

[0098] x i ∈K j if||x i -μ j || 2 =min(x i ,K)

[0099]

[0100] Where, x i K represents the location of the data point. j, For x i The cluster it belongs to, μ j Let x be the mean of the j-th cluster, min(x) i ,K) is the minimum Euclidean distance of a single data point from the K clusters.

[0101] Where K is the preset number of clusters, C i For all data points in the i-th cluster, μ i Let x be the center point (mean vector) of the i-th cluster, and let x be the low-dimensional latent vector encoded by LSTM.

[0102] Furthermore, in step S3, based on the characteristics of real-time energy management, a policy framework integrating optimization and reinforcement learning is established. The ECMS policy is nested with reinforcement learning algorithms to establish an integrated hierarchical energy management architecture, thereby reducing the size of the decision model.

[0103] The ECMS strategy dynamically optimizes the power allocation between the engine and the electric motor to minimize overall energy consumption and quickly calculates the optimal solution in each control cycle. The decision-making behavior of ECMS can be represented as follows:

[0104]

[0105] λ(t)=λ0+k×(SOC(t)-SOC target )

[0106]

[0107] Where, m fce (P fce For fuel cells, the fuel consumption rate, P bat Where λ is the battery power, η is the equivalent factor, and η is the equivalence factor. bat For battery charging and discharging efficiency, The calorific value of hydrogen is given by λ0, which is the basic equivalent factor. k is the feedback gain coefficient, which controls the adjustment intensity when the State of Charge (SOC) deviates from the target. target The target power level.

[0108] The ECMS policy is embedded into a hierarchical energy management architecture as a lower-level policy controller for real-time energy management. The energy management architecture can be represented as follows:

[0109] S ego =[v,a,SOC,P motor ,P FCE ,h t ]

[0110] a eco = [-1.5, -1.0, -0.5, 0, 0.5, 1.0, 1.5]

[0111]

[0112] Where v is the vehicle speed, a is the vehicle acceleration, Soc is the vehicle's state of charge, and P... motor P is the power required by the motor. FCE For the output power of the fuel cell, h t To characterize the driving behavior features of the model, a eco Discretized into seven dimensions, with an interval of 0.5 m / s. 2 r cost (s k ,a k) represents the instantaneous equivalent hydrogen consumption cost of a vehicle, r soc (s k ,a k ) represents the offset penalty between the vehicle's instantaneous SOC and the reference SOC value, r gap (s k ,a k α1 represents the difference between the minimum equivalent hydrogen consumption and the equivalent hydrogen consumption under typical operating conditions. α2, α3 represent the weight values ​​of the three costs, which need to be adjusted according to the actual order of magnitude.

[0113] Furthermore, in step S4, the new model state space from step S2 is integrated, the SAC algorithm is used as the decision model, and the energy management strategy from step S1 is used as the benchmark reward to construct a new reward function, in order to better capture the relationship between energy consumption and model action decisions in the eco-driving strategy.

[0114] The construction of the reinforcement learning reward function should conform to the chain mapping relationship of "state-action-reward". This invention uses the energy management strategy of typical policies as the benchmark reward to construct a self-supervised expert experience reinforcement learning framework, builds an information bridge between the old and new policies, and uses the benchmark reward made by expert experience to guide the model's policy exploration, expand the model's action space, and enable the model to find the correct policy to correct the black box model.

[0115] The eco-driving model strategy combines the classic reward mechanism of equivalent hydrogen consumption and battery degradation, and integrates the difference between hydrogen consumption and the hydrogen consumption of actions performed by the expert strategy to form a new reward function to accelerate strategy optimization, which can be expressed as follows:

[0116]

[0117] r cost (s k ,a k ) = J fce +J bat

[0118]

[0119] Among them, J bat The cost of hydrogen consumption equivalent to that of a battery. The main formula for calculating equivalent hydrogen consumption is as follows: J is an adjustment term for the equivalent hydrogen consumption cost. fce This refers to the hydrogen consumption of the fuel cell.

[0120] Specifically, according to Figure 2 , Figure 3 The present invention will be described in further detail as follows:

[0121] Step S1: Establish a typical vehicle reinforcement learning control model, construct the state, action space and reward function, and establish a traditional fuel cell hybrid electric vehicle energy management strategy.

[0122] Using three typical decision-making models built into SUMO as a foundation, the dynamic characteristics of vehicle decision-making behavior are captured in real time as the input to the state space. Specifically, the speed conditions of "Krauss", "IDE" and "Wiedemann" in the specified scenario are spliced ​​together, and the vehicle acceleration 'a' is recorded in real time as a training dataset. Based on the SAC algorithm, a basic model of fuel cell hybrid electric vehicle is built in Python to train conventional energy management strategies.

[0123] SAC (soft-actor-critic), an important branch of the Actor-Critic algorithm framework, is a relatively new algorithm developed in recent years. It introduces entropy regularization to encourage policy model exploration, maximizing both cumulative reward and maximum entropy. Based on the classic EMS energy management strategy, it uses vehicle dynamics characteristics as state input and energy allocation of the power system as action output. The reward function must consider the balance between energy loss and power system degradation. The algorithm strategy is based on the characteristics of the Bellman formula, designing a reward function and iteratively optimizing a state-action value function. Upon convergence, the reward function is as shown in the equation:

[0124]

[0125] Where, θ j ' is the target network, θ i For the current network, The action to generate the next state for the policy network. Let be the policy network, s′ be the state at the next time step, r be the reward for the current state, and y be the target Q value.

[0126] By maximizing the Q-value and entropy, the randomness of the policy is encouraged and the policy is guided to select actions with high Q-values, thus updating the policy network parameters.

[0127]

[0128] in, Sampling of actions avoids gradient non-differentiability caused by direct sampling. The mean and standard deviation of the policy network output. Let α be the entropy of the action, and α be the temperature hyperparameter.

[0129] The temperature parameter α is used to adjust the reinforcement of entropy regularization, which is used to balance the exploration and equilibrium of the reinforcement learning algorithm. Its update formula is shown below:

[0130]

[0131] Finally, the update magnitudes of the value network and policy network are adjusted by the hyperparameter τ, and the update formula is as follows:

[0132] θ′ i =τθ i +(1-τ)θ′ i i = 1, 2

[0133] Where, θ′ i For the target network, θ i τ represents the current network, and τ represents the update rate.

[0134] The state input of EMS-SAC needs to fully consider the vehicle's effective state in the current scenario as the network input in order to better explore the state-reward mapping relationship, specifically expressed as follows:

[0135] S ego =[v,a,Soc,P motor ,P FCE ]

[0136] Where v is the vehicle's speed, a is the acceleration, Soc is the current state of charge (SOC) value, and P... motor P is the instantaneous power demand of the vehicle motor. FCE This refers to the instantaneous power demand of the vehicle's fuel cell.

[0137] The EMS-SAC's motion space output is the fuel cell's output power (kW). Based on real-world FCV operating conditions, upper and lower limits of acceleration and acceleration increments are designed, and a discrete motion space is designed:

[0138] a ems =[-4,-3,-2,-1,0,1,2,3,4]

[0139] The primary objective of energy management is to optimize economic losses at the operating level, including energy consumption and power system lifespan degradation. Therefore, the reward function can be expressed as:

[0140]

[0141] r cost (s k ,a k ) = r cost_fce (s k ,a k )+r cost_bat (s k ,ak )

[0142] r soc (s k ,a k )=|SOC(k)-SOC ref | 2

[0143] Where, r cost (s k ,a k ) represents the instantaneous equivalent hydrogen consumption cost of a vehicle, r soc (s k ,a k This represents the vehicle's instantaneous SOC relative to the reference SOC. ref The offset penalty for the value, α1 and α2 represent the weight values ​​of the two costs respectively, and need to be adjusted according to the actual order of magnitude.

[0144] Step S2: Based on the decision-making and control characteristics of different vehicles, extract the driving behavior features of the model, integrate traditional strategies to build a new model state space, and enhance the interpretability of the state-action-reward mapping relationship of reinforcement learning.

[0145] The time-series driving behavior data is transformed into low-dimensional feature vectors by using an LSTM autoencoder network to represent the temporal pattern and decision characteristics of the current driving strategy. The state-action sequences of the past five seconds are collected and the output encoded vectors are fused with the feature vectors of the power system to form a new state space, thereby enhancing the interpretability of the reinforcement learning state-to-reward mapping.

[0146] The principle of LSTM is to process time-series data through a gating mechanism. It utilizes the state vectors from the past five seconds of the driving environment, including the vehicle's policy output information and surrounding vehicle information, concatenated with the vehicle's output acceleration to form state-action pairs. Feature extraction and encoding vector output are then performed through window sliding to achieve feature recognition of driving modes. The composition of the LSTM autoencoder network can be represented as follows:

[0147] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0148] i t =σ(W i ·[h t-1 ,x t ]+b i )

[0149]

[0150] Ot =σ(W o ·[h t-1 ,x t ]+b o )

[0151] h t =O t ⊙tanh(C t )

[0152] Among them, f t The input to the forget gate is represented by W and b, where W and b represent the weight matrix and bias term, respectively, and σ and tanh represent the activation functions, respectively. t-1 x represents the hidden state at the previous time step. t For the input at the current time step, i t Indicates the input gate output. Representing candidate memories, C t Indicates the updated cell state, O t This indicates the output gate.

[0153] The LSTM autoencoder network is trained using the training set data. The high-dimensional matrix information is converted into latent feature vectors by encoding the training dataset to learn the main features. Then, the latent vectors are decoded to reconstruct the data. The weights of the LSTM autoencoder network are updated using a mean squared error reconstruction loss function, which can be expressed as follows:

[0154]

[0155] Where, x i Indicates input data, This indicates the reconstruction of data.

[0156] By recording data from the past five seconds and sampling it at a frequency of 10Hz, the input historical state-action pair can be represented as:

[0157] S LSTM

[0158] =[[v,d front ,d rear ,d traffic-light ,a action ]1,……,[v,d front ,d rear ,d traffic-light ,a action ] 50 ]

[0159] Where v is the vehicle speed, d front , where d is the distance to the vehicle in front. rear d represents the distance to the following vehicle. traffic-lighta is the distance to the nearest traffic light. action This refers to the acceleration of the vehicle.

[0160] The trained network is validated using training data, and the encoded vectors are visualized by dimensionality reduction. This allows for clear separation of the encodings for different driving styles in 2D space. The K-means method is used to reduce the dimensionality of the encodings, and a visual clustering output of driving styles is generated through supervised learning in a regression-classification manner. K-means initially classifies all data points by initializing the cluster positions, then calculates the mean of the data to obtain the latest cluster positions, and so on. K-means clustering achieves the clustering task by minimizing the sum of the squared distances from all data points to their cluster centers. Its principle can be described as follows:

[0161] x i ∈K j if||x i -μ j || 2 =min(x i ,K)

[0162]

[0163] Where, x i K represents the location of the data point. j , for x i The cluster it belongs to, μ j Let x be the mean of the j-th cluster, min(x) i ,K) is the minimum Euclidean distance of a single data point from the K clusters.

[0164] Where K is the preset number of clusters, C i For all data points in the i-th cluster, μ i Let x be the center point (mean vector) of the i-th cluster, and let x be the low-dimensional latent vector encoded by LSTM.

[0165] Finally, the trained LSTM autoencoder network is embedded into a reinforcement learning framework for real-time feature extraction of test data, transforming the high-dimensional input into low-dimensional latent features h. t This, together with the eigenvectors of the dynamical system, forms a novel state space, which can be represented as follows:

[0166] S ECO =[v,a,SOC,P motor ,P FCE ,h t ]

[0167] Where v is the vehicle speed, a is the vehicle acceleration, SOC is the vehicle state of charge, and P motor P is the power required by the motor. FCEFor the output power of the fuel cell, h t State variables are used to characterize the driving behavior features of the model.

[0168] Step S3: Based on the characteristics of real-time energy management, establish a policy framework that integrates optimization and reinforcement learning, nest reinforcement learning algorithms within the ECMS policy, establish an integrated hierarchical energy management architecture, and reduce the size of the decision model.

[0169] The core of the ECMS strategy is to dynamically optimize the power distribution between the engine and the electric motor by equivalently converting electrical energy consumption to fuel consumption, thereby minimizing overall energy consumption. It also rapidly calculates the optimal solution within each control cycle to balance the ratio of fuel to electricity usage. The decision-making behavior of ECMS can be represented as follows:

[0170]

[0171] Where, m fce (P fce For fuel cells, the fuel consumption rate, P bat Where λ is the battery power, η is the equivalent factor, and η is the equivalence factor. bat For battery charging and discharging efficiency, This represents the calorific value of hydrogen.

[0172] Based on the actual vehicle powertrain model, the output range of the powertrain is limited.

[0173]

[0174] SOC min ≤SOC≤SOC max

[0175] in, For the minimum and maximum output power of the fuel cell, The SOC (State of Charge) represents the battery's maximum charging power and maximum discharging power. min SOC max These are the boundary values ​​of SOC.

[0176] The dynamic adjustment of the equivalent factor λ to regulate the economic weights of electricity and hydrogen energy consumption can be expressed as:

[0177] λ(t)=λ0+k×(SOC(t)-SOC target )

[0178] Where λ0 is the basic equivalent factor, k is the feedback gain coefficient, and the adjustment strength controls the SOC when it deviates from the target. target The target power level.

[0179] The ECMS strategy is embedded into a hierarchical energy management architecture as a lower-level strategy controller for real-time energy management. This provides real-time reward function feedback for the upper-level reinforcement learning-based driving action control. The driving behavior features extracted in step S2 are integrated into the state dimension, forming the reinforcement learning state space together with the vehicle's powertrain features. This state space can be represented as:

[0180] S ego =[v,a,SOC,P motor ,P FCE ,h t ]

[0181] Where v is the vehicle speed, a is the vehicle acceleration, SOC is the vehicle state of charge, and P motor P is the power required by the motor. FCE For the output power of the fuel cell, h t State variables are used to characterize the driving behavior features of the model.

[0182] Eco-driving output is the vehicle's acceleration (m / s²). 2 Considering real-world operating conditions, the acceleration output value is limited to [-1.5, 1.5] m / s². 2 Inside can be represented as:

[0183] a eco = [-1.5, -1.0, -0.5, 0, 0.5, 1.0, 1.5]

[0184] The motion is discrete in seven dimensions, with an interval of 0.5 m / s. 2 .

[0185] The reward function for eco-driving uses the energy management output value under typical operating conditions as the benchmark reward to construct a new reward function, which can be expressed as:

[0186]

[0187] Where, r cost (s k ,a k ) represents the instantaneous equivalent hydrogen consumption cost of a vehicle, r soc (s k ,a k ) represents the offset penalty between the vehicle's instantaneous SOC and the reference SOC value, r gap (s k ,a k α1 represents the difference between the minimum equivalent hydrogen consumption and the equivalent hydrogen consumption under typical operating conditions. α2, α3 represent the weight values ​​of the three costs, which need to be adjusted according to the actual order of magnitude.

[0188] Step S4: Integrate the new model state space from Step S2, use the SAC algorithm as the decision model, and construct a new reward function using the energy management strategy from Step S1 as the benchmark reward, in order to better capture the relationship between energy consumption and model action decisions in the eco-driving strategy.

[0189] The construction of the reinforcement learning reward function should conform to the chain mapping relationship of "state-action-reward". This invention uses the energy management strategy of typical policies as the benchmark reward to construct a self-supervised expert experience reinforcement learning framework, builds an information bridge between the old and new policies, and uses the benchmark reward made by expert experience to guide the model's policy exploration, expand the model's action space, and enable the model to find the correct policy to correct the black box model.

[0190] This invention incorporates a reward function within a nested reinforcement learning-based ecological driving strategy framework. This function includes the vehicle's real-time equivalent hydrogen consumption to optimize actions that reduce energy consumption; and a SOC plasticity reward term to measure the SOC relative to a reference SOC. ref The distance guides the agent to move the battery's state of charge (SOC) closer to the SOC. ref Decision-making to avoid overcharging / discharging of the power battery; the difference between the hydrogen consumption of the eco-driving model strategy and the hydrogen consumption of the expert strategy to accelerate strategy optimization can be expressed as the following formula:

[0191]

[0192]

[0193] r cost (s k ,a k ) = J fce +J bat

[0194] r soc (s k ,a k )=|SOC(k)-SOC ref |

[0195] r gap (s k ,a k ) = Cost ems -Cost eco

[0196] Among them, P fce LHVH2 is the output power of the fuel cell, P is the calorific value of hydrogen, and LHVH2 is the calorific value of hydrogen. bat For battery output power, η bat For battery charge and discharge efficiency, η dcdc For DC-DC conversion efficiency, J fceFor fuel cell consumption costs, J bat The cost of hydrogen consumption equivalent to that of a battery. The main calculation formula is... This is the equivalent hydrogen cost correction term, where SOC(k) is the real-time SOC value. ref Cost is the SOC reference value. ems Cost represents the hydrogen consumption value for a conventional energy management strategy. eco This represents the hydrogen consumption value for the eco-driving strategy.

[0197] In summary, the present invention designs a nested real-time energy management network structure as an energy-saving guide for the eco-driving strategy, and combines it with a novel state space extraction method to achieve synergistic strategy optimization of energy efficiency and high generalization of fuel cell hybrid electric vehicles, thereby reducing the energy consumption of fuel cell hybrid electric vehicles and improving the deployment efficiency of the eco-driving strategy.

[0198] As another example, embodiments of the present invention also provide an electronic device, which is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0199] The electronic device may include a processor, a communications interface, memory, and a communications bus.

[0200] The processor, communication interface, and memory communicate with each other via a communication bus. The communication interface is used to communicate with other electronic devices or servers.

[0201] The processor is used to execute programs, specifically the relevant steps in the above method embodiments.

[0202] Specifically, the program may include program code, which includes computer operation instructions.

[0203] The processor may be a CPU, an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The one or more processors included in a smart device may be of the same type, such as one or more CPUs; or they may be of different types, such as one or more CPUs and one or more ASICs.

[0204] Memory is used to store programs. Memory may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device.

[0205] When executed by a processor, the program is used to enable electronic devices to perform an intelligent energy-saving method for fuel cell vehicles that adaptively corrects the behavior of a black-box model, in accordance with the present invention.

[0206] Furthermore, the specific implementation of each step in the program can be found in the corresponding descriptions of the steps and units in the above method embodiments, and will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices and modules described above can be referred to the corresponding process descriptions in the foregoing method embodiments, and will not be repeated here.

[0207] An exemplary embodiment of the present invention also provides a computer storage medium storing a computer program, wherein when the computer program is executed by a processor, it implements the methods of the various embodiments of the present invention. The corresponding process descriptions in the foregoing method embodiments can be referred to, and will not be repeated here.

[0208] The methods described above according to embodiments of the present invention can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code originally stored on a remote recording medium or a non-transitory machine-readable medium and subsequently stored on a local recording medium, downloaded via a network. Thus, the methods described herein can be processed by software stored on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., RAM, ROM, flash memory, etc.) capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods described herein. Furthermore, when a general-purpose computer accesses code used to implement the methods shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the methods shown herein.

[0209] Specific embodiments of the invention have now been described. Other embodiments are within the scope of the appended claims. In some cases, the actions described in the claims can be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing can be advantageous.

[0210] It should be understood that although this specification is described according to various embodiments, not every embodiment contains only one independent technical solution. This way of describing the specification is only for clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

[0211] Finally, it should be noted that the above embodiments are only used to illustrate the embodiments of the present invention, and are not intended to limit the embodiments of the present invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of the present invention. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of the present invention, and the patent protection scope of the embodiments of the present invention should be defined by the claims.

Claims

1. A smart energy-saving method for fuel cell vehicles that adaptively corrects the behavior of a black-box model, characterized in that, Includes the following steps: S1. By constructing the state space, action space, and reward function, a typical vehicle reinforcement learning control model is built, and a traditional fuel cell hybrid electric vehicle energy management strategy is also constructed. S2. Based on the decision-making and control characteristics of different vehicles, the driving behavior features of the typical vehicle reinforcement learning control model are extracted through the LSTM autoencoder network, and the driving behavior features of the model are fused with the energy management strategy of the traditional fuel cell hybrid electric vehicle to construct a new model state space, so as to enhance the interpretability of the reinforcement learning state-action-reward mapping relationship. S3. Based on the characteristics of real-time energy management, a policy framework integrating optimization and reinforcement learning is established. The ECMS policy is nested with reinforcement learning algorithms to establish an integrated hierarchical energy management architecture, thereby reducing the size of the decision model. S4. Based on the strategy framework of fusion optimization and reinforcement learning and the integrated hierarchical energy management architecture, using the SAC algorithm as the decision model, the new model state space is fused and the traditional fuel cell hybrid electric vehicle energy management strategy is used as the benchmark reward to construct a new reward function, so as to accurately capture the relationship between energy consumption and action decision of the typical vehicle reinforcement learning control model in the ecological driving strategy.

2. The method according to claim 1, characterized in that, The state space of the energy management strategy for the conventional fuel cell hybrid electric vehicle includes the characteristics of the vehicle's powertrain, the action is the output power of the fuel cell, and the reward function includes the degradation cost of the powertrain, expressed as: S ego =[v,a,SOC,P motor ,P FCE ] a eco ＝[-4,-3,-2,-1,0,1,2,3,4] Where v is the vehicle's speed, a is the acceleration, SOC is the current state of charge (SOC) value, and P... motor P is the instantaneous power demand of the vehicle motor. FCE For the instantaneous power demand of the vehicle's fuel cell, a eco R represents the output power of the fuel cell. cost (s k ,a k ) represents the instantaneous equivalent hydrogen consumption cost of a vehicle, r soc (s k ,a k ) represents the offset penalty of the vehicle's instantaneous SOC relative to the reference SOC value, and α1 and α2 represent the weight values ​​of the two costs, respectively.

3. The method according to claim 1, characterized in that, The LSTM autoencoder network structure consists of a forget gate, an input gate, a cell state update gate, and an output gate. The network structure is represented as follows: f t =σ(W f ·[h t-1 ,x t ]+b f ) i t =σ(W i ·[h t-1 ,x t ]+b i ) The t =σ(W o ·[h t-1 ,x t ]+b o ) h t =O t ⊙tanh(C t ) Among them, f t The input to the forget gate is represented by W and b, where W and b represent the weight matrix and bias term, respectively, and σ and tanh represent the activation functions, respectively. t-1 x represents the hidden state at the previous time step. t For the input at the current time step, i t Indicates the input gate output. Representing candidate memories, C t Indicates the updated cell state, O t Indicates the output gate; The latent vector is concatenated with the vehicle dynamics feature vector to form a new state space, represented as: S ECO =[v,a,Soc,P motor ,P FCE ,h t ] Where v is the vehicle speed, a is the vehicle acceleration, Soc is the vehicle's state of charge, and P... motor P is the power required by the motor. FCE For the output power of the fuel cell, h t State variables that characterize the driving behavior features of the model.

4. The method according to claim 1, characterized in that, The LSTM autoencoder network is updated using a reconstruction loss function, which is expressed as: Where, x i Indicates input data, This indicates the reconstruction of data.

5. The method according to claim 1, characterized in that, The LSTM autoencoder network compresses high-dimensional matrix information into low-dimensional latent vectors and performs clustering verification using the K-means clustering method, which is expressed as follows: Where K is the preset number of clusters, C i For all data points in the i-th cluster, μ i Let x be the center point of the i-th cluster, and let x be the low-dimensional latent vector encoded by LSTM.

6. The method according to claim 1, characterized in that, The decision-making behavior of the ECMS strategy is represented as follows: Where, m fce (P fce For fuel cells, the fuel consumption rate, P bat Where λ is the battery power, η is the equivalent factor, and η is the equivalence factor. bat For battery charging and discharging efficiency, This refers to the calorific value of hydrogen. The equivalent factor λ is dynamically adjusted to regulate the economic weights of electricity and hydrogen energy consumption, expressed as follows: λ(t)=λ0+k×(SOC(t)-SOC target ) Where λ0 is the basic equivalent factor, k is the feedback gain coefficient, and the adjustment strength controls the SOC when it deviates from the target. target The target power level.

7. The method according to claim 2, characterized in that, The decision-making behavior of the ECMS strategy is represented as follows: The ECMS strategy is embedded in a hierarchical energy management architecture as a lower-level strategy controller for real-time energy management. This provides real-time reward function feedback for the upper-level reinforcement learning-based driving action control. The driving behavior characteristics of the typical vehicle reinforcement learning control model are integrated into the state dimension and combined with the vehicle's power system characteristics to form the reinforcement learning state space. The output action, the output power of the fuel cell, can be expressed as: S ego =[v,a,SOC,P motor ,P FCE ,h t ] Where v is the vehicle speed, a is the vehicle acceleration, SOC is the vehicle state of charge, and P motor P is the power required by the motor. FCE For the output power of the fuel cell, h t To characterize the state variables of the model's driving behavior features; The output of eco-driving, namely the vehicle's acceleration, can be expressed as: a eco ＝[-1.5,-1.0,-0.5,0,0.5,1.0,1.5] The motion is discrete in seven dimensions, with an interval of 0.5 m / s. 2 Limited to [-1.5, 1.5] m / s 2 between; The reward function for eco-driving can be expressed as: Where, r cost (s k ,a k ) represents the instantaneous equivalent hydrogen consumption cost of a vehicle, r soc (s k ,a k ) represents the offset penalty between the vehicle's instantaneous SOC and the reference SOC value, r gap (s k ,a k The value represents the difference between the minimum equivalent hydrogen consumption and the equivalent hydrogen consumption under typical operating conditions.

8. The method according to claim 1, characterized in that, The eco-driving strategy combines the classic reward mechanism of equivalent hydrogen consumption and battery degradation, and integrates the difference between hydrogen consumption and the hydrogen consumption of actions performed by the expert strategy to form the novel reward function to accelerate the optimization of the eco-driving strategy, which can be expressed as: r cost (s k ,a k )=J fce +J bat r foc (s k ,a k )=|SOC(k)-SOC ref | r gap (s k ,a k )=Cost ems -Cost eco Among them, J bat J represents the equivalent hydrogen consumption cost during battery charging and discharging. fce The SOC(k) represents the hydrogen consumption cost of the fuel cell, and SOC(k) represents the current SOC value. ref For reference to SOC value, Cost ems Cost represents the hydrogen consumption value for a conventional energy management strategy. eco This represents the hydrogen consumption value for the eco-driving strategy.

9. An electronic device, characterized in that, include: processor; Memory for stored programs; The program includes instructions that, when executed by the processor, cause the processor to perform the steps of the method as described in any one of claims 1-8.

10. A computer storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1-8.

Images