A hydrogen-electric hybrid system energy management method, system, device and storage medium based on dual-Q learning and real-time speed prediction

Abstract

The present invention belongs to the technical field of fuel cell hybrid system energy management, specifically relating to a hydrogen-electric hybrid system energy management method, system, device, and storage medium based on dual-Q learning and real-time speed prediction. The method comprises obtaining a classified driving pattern of a target vehicle; predicting the target vehicle's speed using a Markov chain; inputting the predicted speed into a hybrid system model of the target vehicle, optimizing a Q-learning structure based on the model output, updating the Q function within the Q-learning structure based on a predefined dual-Q learning strategy, and achieving real-time distribution of output power within the hybrid system based on the updated Q function. The present invention proposes a reward-value function that comprehensively considers system safety, economy, and fuel cell durability, enabling the optimal combination of multiple hybrid system indicators. The dual-Q learning strategy is used to update the Q-value function, which avoids the maximum deviation caused by the Q-learning algorithm's action selection.

Description

Technical Field

[0001] The present invention belongs to the technical field of fuel cell hybrid system energy management, and specifically relates to a hydrogen-electric hybrid system energy management method, system, device and storage medium based on dual-Q learning and real-time speed prediction. Background Art

[0002] Fuel cell vehicles (FCVs) have slow dynamic response and are unable to quickly track load changes, making them difficult to meet daily driving needs. Currently, hybrid systems, primarily combining fuel cells and batteries, are the primary configuration for research and development and application. The core issue with hybrid systems lies in energy management strategies.

[0003] There are three types of energy management strategies for hydrogen-electric hybrid systems: rule-based, optimization-based, and learning-based. Rule-based methods design a power allocation rule table based on operational knowledge and professional experience, allowing the fuel cell to operate within the most efficient range possible. Optimization-based energy management strategies can comprehensively consider various constraints such as power component aging and load conditions, and can be divided into global optimization methods and instantaneous optimization methods. Global optimization methods can ensure optimal system economy, but these methods require advance information on road conditions, are computationally intensive, and lack real-time performance. Instantaneous optimization methods offer the advantages of low computational effort and high real-time performance, but suffer from lower economy and durability.

[0004] To address this problem, the present invention uses reinforcement learning to achieve energy management and studies the key parameters of reinforcement learning to reduce algorithm complexity and improve real-time performance. Summary of the Invention

[0005] The purpose of the present invention is to provide a hydrogen-electric hybrid system energy management method, system, device and storage medium based on dual Q learning and real-time speed prediction to solve the problems raised in the background technology.

[0006] The present invention achieves the above-mentioned purpose through the following technical solutions:

[0007] In a first aspect, the present invention proposes a hydrogen-electric hybrid system energy management method based on dual reinforcement learning and real-time speed prediction, the method comprising the following steps:

[0008] S1. Obtain the target vehicle's driving database, construct a driving mode classifier based on multi-feature extraction and probabilistic neural network, and obtain the classified driving mode;

[0009] S2. Design a multi-step Markov speed predictor, obtain a corresponding Markov chain based on the classified driving patterns, and use the Markov chain to predict the speed of the target vehicle to obtain a predicted speed;

[0010] S3. Input the predicted speed into a hybrid power system model of the target vehicle, output the vehicle's required power, fuel cell output power, lithium battery output power, and lithium battery SoC, optimize the Q learning structure based on the output, update the Q function in the Q learning structure based on the set dual-Q learning strategy, and implement real-time distribution of output power in the hybrid power system based on the updated Q function.

[0011] Furthermore, the step S1 includes:

[0012] S101. Establish and label a standard database of different road profiles, classify typical operating conditions into high-speed mode, urban flow mode, and urban congestion mode, and use this database to train a classifier;

[0013] S102: Obtain a driving database and perform statistical analysis on its time series data to extract standard features that characterize different driving patterns; the standard features include average speed, speed standard deviation, maximum speed, maximum acceleration, minimum acceleration, average value of absolute acceleration, standard deviation of absolute acceleration, percentage of parking time, and percentage of time at a constant speed;

[0014] S103. Construct a driving mode classifier using a probabilistic neural network. The classifier structure includes four parts: an input layer, a pattern layer, a summation layer, and a competitive output layer. The number of nodes in the input layer corresponds to the number of standard features. The pattern layer performs a weighted summation operation on the input signal. The summation layer estimates the sum of various types of probabilities according to the Parzen method. The competitive output layer outputs the classification result of the driving mode.

[0015] Furthermore, in step S103, the summation layer estimates the probabilities of various driving modes according to the Parzen method, including:

[0016] Each network unit in the summation layer is connected to the majority unit of the corresponding category, and each unit estimates the sum of the probabilities of various types according to the Parzen method:

[0017]

[0018] Where, l g represents the number of samples in the gth class, σ represents the smoothing parameter, which is between 0 and 1, and x ij (g) Represents the j-th data of the i-th neuron of class g.

[0019] Furthermore, the step S2 includes:

[0020] S201, fusing standard driving profiles under the same driving mode to form a data set to construct a Markov speed predictor under the mode;

[0021] S202. Define an n-step state transition probability matrix for future speed prediction:

[0022]

[0023] Where k represents the kth time; n is the number of sampling points; a(n+k) and a(k) represent the accelerations of the (n+k)th and kth times, respectively; Represents when the speed state is v l When the acceleration state is a, it is transferred to the acceleration state a after n steps. j the number of times; is the number of steps n, which represents the time when the speed state is v l And the acceleration state is a i The number of times all acceleration states are reached after n steps.

[0024] Furthermore, in step S3, before optimizing the Q-learning structure according to the output, the step further includes:

[0025] Prioritize the Q-learning structure and express it as a five-tuple: (S, A, T, γ, R), where S is the state space, A is the action space, and T is the state transition probability, i.e. γ is the discount factor and R is the reward function.

[0026] Furthermore, in step S3, optimizing the Q learning structure according to the output includes:

[0027] S301. Set state space S: select power demand P m and the battery SoC as the state space S for Q-learning:

[0028] S(t)={P m (t), SoC(t)}

[0029] P m is the total power requirement of the system, and SoC is the current charge of the battery;

[0030] S302, set the action space A: set the fuel cell change rate ΔP fc As the action space A, its unit is kW / s, and the variation of the fuel cell is suppressed by setting the range as follows:

[0031] A=[-5,-2,-1,-0.75,-0.5,-0.3,-0.1,0,0.1,0.3,0.5,0.75,1,2,5];

[0032] S303. Set the reward function R:

[0033]

[0034] The first term in the denominator represents the proportion of hydrogen consumption to the reward; the second term in the denominator is used to guide the battery SoC to follow the set reference value SoC ref Move; the third term in the denominator is used to minimize ΔP fc,k ; The fourth term in the denominator is the expectation that the fuel cell operates near the highest efficiency; w1,…,w4 are weight coefficients.

[0035] Furthermore, in step S3, the Q function in the Q learning structure is updated based on the set dual-Q learning strategy, and the real-time distribution of the output power in the hybrid system is achieved according to the updated Q function, including:

[0036] S304, dividing the obtained sample data into two groups, marked as Q1 and Q2 respectively;

[0037] S305: Use the Q1 group of sample data to determine the maximum action a of the hybrid system. * :

[0038] S306, using Q2 group sample data to calculate the maximum action a * The following values: Since E[Q2(a * )]=q(a * ), where q(a*) represents the true estimated value of the hybrid system, and Q2(a * ) as an unbiased estimate of the true value of the hybrid system;

[0039] S307: Swap the roles of Q1 and Q2, and repeat steps S305-S306 to obtain another unbiased estimate.

[0040] S308 : Allocate the output power in the hybrid power system in real time according to the obtained unbiased estimated value.

[0041] In a second aspect, the present invention proposes a hydrogen-electric hybrid system energy management system based on dual-Q learning and real-time speed prediction, which is applied to implement the above-mentioned energy management method. The system includes:

[0042] The driving classification module is used to obtain the driving database of the target vehicle, build a driving mode classifier based on multi-feature extraction and probabilistic neural network, and obtain the classified driving mode;

[0043] The speed prediction module is used to design a multi-step Markov speed predictor, obtain the corresponding Markov chain based on the classified driving mode, and use the Markov chain to predict the speed of the target vehicle to obtain the predicted speed;

[0044] a power distribution module for inputting the predicted speed into a hybrid power system model of a target vehicle, outputting the vehicle's required power, fuel cell output power, lithium battery output power, and lithium battery SoC; optimizing a Q-learning structure based on the outputs; updating a Q function in the Q-learning structure based on a set dual-Q learning strategy; and achieving real-time distribution of output power in the hybrid power system based on the updated Q function.

[0045] In a third aspect, the present invention proposes an electronic device, comprising a processor module and a memory module storing a computer program, wherein the processor module implements the above-mentioned energy management method when executing the computer program.

[0046] In a fourth aspect, the present invention proposes a computer-readable storage medium having a computer program stored thereon, which implements the above-mentioned energy management method when executed by a processor module.

[0047] The beneficial effects of the present invention are:

[0048] 1. This invention proposes a real-time driving pattern classification and speed prediction implementation method based on data-driven and statistical analysis to predict the near-term vehicle speed, thereby improving the real-time and generalization of the strategy and enhancing the reliability of the energy management strategy.

[0049] 2. This invention innovatively proposes a reward-value function that comprehensively considers system safety, economy, and fuel cell durability, enabling optimal multi-objective performance for hybrid systems. It employs the concept of dual learning to achieve unbiased estimation, employing a dual Q-learning strategy to update the Q-value function, thus avoiding the maximum bias introduced by the Q-learning algorithm when selecting actions.

[0050] 3. This paper uses the battery state of charge as an indicator to design a real-time reference path for power distribution and explores the changing trajectory of SoC under the optimal economic goal. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a flowchart of an implementation method of a hydrogen-electric hybrid system energy management method based on dual reinforcement learning and real-time speed prediction provided by one or more embodiments of the present invention.

[0052] Figure 2 This is a structural diagram of a classifier provided by one or more embodiments of the present invention.

[0053] Figure 3 It is a Markov transition probability matrix of urban circulation conditions at a speed of 10 m / s provided by one or more embodiments of the present invention.

[0054] Figure 4 It is a flow chart of an energy management strategy provided by one or more embodiments of the present invention.

[0055] Figure 5 is a diagram of a fuel cell / battery hybrid system provided by one or more embodiments of the present invention. DETAILED DESCRIPTION

[0056] The present application will be described in further detail below in conjunction with the accompanying drawings. It is necessary to point out here that the following specific implementation methods are only used to further illustrate the present application and cannot be understood as limiting the scope of protection of the present application. Technical personnel in this field can make some non-essential improvements and adjustments to the present application based on the above application content.

[0057] The key to implementing energy management strategies using reinforcement learning lies in the design of the algorithm's state variables and reward function. The dimensionality of the state variables and the number of grids determine the algorithm's accuracy and complexity, while the reward function determines the performance metrics it focuses on. To address this issue, this paper studies key parameters of reinforcement learning to reduce algorithm complexity and improve real-time performance.

[0058] Example 1

[0059] like Figure 1 As shown, this embodiment proposes a hydrogen-electric hybrid system energy management method based on dual reinforcement learning and real-time speed prediction, and the method includes the following steps:

[0060] S1. Obtain the target vehicle's driving database, construct a driving mode classifier based on multi-feature extraction and probabilistic neural network, and obtain the classified driving mode;

[0061] S2. Design a multi-step Markov speed predictor. Obtain the corresponding Markov chain based on the classified driving patterns. Use the Markov chain to predict the speed of the target vehicle and obtain the predicted speed.

[0062] S3. Input the predicted speed into the hybrid power system model of the target vehicle, output the vehicle's required power, fuel cell output power, lithium battery output power, and lithium battery SoC, optimize the Q-learning structure based on the output, update the Q function in the Q-learning structure based on the set dual-Q learning strategy, and implement real-time distribution of output power in the hybrid power system based on the updated Q function.

[0063] Further preferably, step S1 includes:

[0064] S101. By annotating a standard database of different road profiles, we can categorize typical driving conditions into three types: high-speed mode, urban mobility mode, and urban congestion mode. For each type, we use the typical road conditions of that type to train a classifier.

[0065] S102. To reduce the number of input variables for the classifier and improve algorithm efficiency, we statistically analyze the time series data in the driving database and extract key statistical features that can characterize different driving patterns as input to the classifier. The selected features include: average speed, speed standard deviation, maximum speed, maximum acceleration, minimum acceleration, average value of absolute acceleration, standard deviation of absolute acceleration, percentage of parking time, and percentage of time at a constant speed.

[0066] S103, using probabilistic neural network to build driving mode classifier, combined with Figure 2 The classifier structure consists of four parts: input layer, pattern layer, summation layer and competitive output layer. The number of nodes in the input layer corresponds to the number of standard features, the pattern layer performs a weighted summation operation on the input signal, the summation layer estimates the sum of various types of probabilities according to the Parzen method, and the competitive output layer outputs the classification results of the driving mode.

[0067] Further preferably, in step S103, the summation layer estimates the probabilities of various driving modes by summing them according to the Parzen method, including:

[0068] Each network unit in the summation layer is connected to the majority unit of the corresponding category, and each unit estimates the sum of the probabilities of various types according to the Parzen method:

[0069]

[0070] Where, l g represents the number of samples in the gth class, σ represents the smoothing parameter, which is between 0 and 1, and x ij (g) Represents the jth data of the i-th neuron of class g.

[0071] Further preferably, step S2 obtains the corresponding Markov chain through the classified driving mode, thereby performing speed prediction, and the specific steps are as follows:

[0072] S201, fusing standard driving profiles under the same driving mode to form a data set to construct a Markov speed predictor under the mode;

[0073] S202. Define an n-step state transition probability matrix for future speed prediction:

[0074]

[0075] Where k represents the kth time; n is the number of sampling points; a(n+k) and a(k) represent the accelerations of the (n+k)th and kth times, respectively; Represents when the speed state is v lWhen the acceleration state is a, it is transferred to the acceleration state a after n steps. j the number of times; is the number of steps n, which represents the time when the speed state is v l And the acceleration state is a i The number of times all acceleration states are reached after n steps.

[0076] Further, if Figure 3 As shown in Figure 1, the transition probability matrix for different prediction domain lengths at a speed of 10 m / s is given. When the prediction time limit is short, the probability distribution is concentrated around the diagonal line. The longer the prediction range, the more dispersed the probability distribution.

[0077] It should be noted that the Markov prediction method used in this embodiment is a probabilistic prediction method that predicts future changes in each period based on the current event. The present invention integrates standard driving curves under the same driving mode to form a data set, and then constructs a Markov speed predictor for this mode.

[0078] Next, we combine the basic ideas of reinforcement learning (Q learning) to design key energy management strategies.

[0079] Further preferably, in step S3, before optimizing the Q learning structure according to the output, the step further includes: preferentially determining the Q learning structure and expressing it as a five-tuple: (S, A, T, γ, R), where S is the state space, A is the action space, and T is the state transition probability, that is, γ is the discount factor and R is the reward function.

[0080] According to the characteristics of the Markov decision process, the state and reward of the agent at the next moment depend only on the current state and behavior. The strategy π:S→A is defined as the probability distribution of the behavior under a given state and should satisfy the following equation:

[0081]

[0082] To evaluate the policy, the state value function V π (s) is defined as the expected reward of state s under the policy π when the system starts from state s, as follows:

[0083]

[0084] Among them, T is the final moment, t is the current moment, r k+t+1 is the reward at time k+t+1.

[0085] According to the Bellman equation, the optimal state value function is as follows:

[0086]

[0087] The action-value function Q(s,a) is defined as the expected reward of choosing a specific action a in state s and then executing policy π. The optimal Q function can be obtained as follows:

[0088]

[0089] In this embodiment, for the energy management problem of the fuel cell / battery hybrid system, each state needs to reflect the system power demand and component status during the power allocation process. Therefore, the power demand P is selected. m and battery SoC as the state space of the reinforcement learning algorithm. In order to improve the durability of the system, the fuel cell change rate ΔP fc As the action space A, large changes in the fuel cell can be suppressed by setting a range. Setting the reward function is a core step in reinforcement learning. The reward function designed by this invention comprehensively considers the economy, safety, durability, and efficiency of the hybrid system.

[0090] It is understandable that the Q-learning algorithm is an effective reinforcement learning method. The recursive equation form of the Q-function is as follows:

[0091]

[0092] Where α∈(0,1] represents the learning rate. In the Q-learning algorithm, the target policy is a greedy policy that selects actions based on the maximum value of all actions. In order to avoid the maximum deviation caused by the above operation, the present invention adopts the concept of dual learning to achieve unbiased estimation.

[0093] Further preferably, in step S3, optimizing the Q learning structure according to the output includes:

[0094] S301. Set state space S: select power demand P m and the battery SoC as the state space S for Q-learning:

[0095] S(t)={P m (t), SoC(t)}

[0096] P m is the total power demand of the system, which can be calculated from the vehicle speed and acceleration. SoC is the current charge of the battery;

[0097] S302, set action space A: In order to improve the durability of the system, the fuel cell change rate ΔP fc As the action space A, its unit is kW / s, and the variation of the fuel cell is suppressed by setting the range as follows:

[0098] A=[-5,-2,-1,-0.75,-0.5,-0.3,-0.1,0,0.1,0.3,0.5,0.75,1,2,5];

[0099] S303 , setting a reward function R. Setting a reward function is a core step in reinforcement learning. The reward function designed in the present invention comprehensively considers the economy, safety, durability, and efficiency of the hybrid power system.

[0100]

[0101] The first term in the denominator represents the proportion of hydrogen consumption to the reward; the second term in the denominator is used to guide the battery SoC to follow the set reference value SoC ref Movement can effectively keep the lithium battery within a safe range; the third term in the denominator is used to minimize ΔP fc,k , in order to improve the durability of the fuel cell; the fourth term in the denominator is the expectation that the fuel cell works near the highest efficiency; w1,…,w4 are weight coefficients.

[0102] Further preferably, in step S3, the recursive equation form of the Q function is as shown in Formula 1. In order to avoid the maximum deviation caused by the algorithm itself, the present invention adopts the concept of dual learning to achieve unbiased estimation, updates the Q function in the Q learning structure based on the set dual Q learning strategy, and realizes the real-time distribution of the output power in the hybrid system according to the updated Q function, including:

[0103] S304, dividing the obtained sample data into two groups, marked as Q1 and Q2 respectively;

[0104] S305: Use the Q1 group of sample data to determine the maximum action a of the hybrid system. * :

[0105] S306, using Q2 group sample data to calculate the maximum action a * The following values: Since E[Q2(a * )]=q(a * ), where q(a*) represents the true estimated value of the hybrid system, and Q2(a * ) as an unbiased estimate of the true value of the hybrid system;

[0106] S307: Swap the roles of Q1 and Q2, and repeat steps S305-S306 to obtain another unbiased estimate.

[0107] S308 : Allocate the output power in the hybrid power system in real time according to the obtained unbiased estimated value.

[0108] In summary, the use of dual-Q learning algorithm to realize the energy management of hybrid power system can achieve a relatively ideal control effect.

[0109] In this embodiment, based on Markov velocity prediction and dual Q learning strategy, the proposed fuel cell / battery hybrid system energy management strategy framework is as follows: Figure 4 As shown. Offline testing of the fuel cell system and lithium battery is carried out to obtain key parameter systems and establish a dynamic model. Figure 5 As shown in the figure, driving pattern classification and speed prediction are performed based on a large offline database, and then applied online by acquiring real-time driving data. The required power is calculated based on the predicted speed, and a dual-Q learning strategy is then implemented to achieve real-time power allocation in the hybrid system.

[0110] This embodiment also includes building a hybrid power system model for the above-mentioned method, including the following steps: Using a fuel cell / lithium battery as the hybrid power source, a semi-active topology is used for research. In this configuration, the lithium battery is directly connected to the load bus, reducing power losses in intermediate components and improving the system's energy efficiency.

[0111] Furthermore, the vehicle dynamics model is shown in the following formula:

[0112]

[0113] Where m represents the vehicle mass; v represents the vehicle speed; dv / dt represents the driving acceleration; θ represents the road slope; A d Indicates the frontal area of ​​the vehicle; C d represents the aerodynamic drag coefficient; ρ represents the air density.

[0114] Furthermore, the fuel cell model is shown in the following formula:

[0115] P fc =P st -P aux

[0116] Among them, P fc Indicates the net power of the system, P st Indicates the power generated by the fuel cell stack, P aux Indicates the total power consumed by auxiliary systems, including air compressors, water pumps, etc.

[0117] Hydrogen consumption is a key indicator for evaluating the economic efficiency of a system. The hydrogen consumption rate is proportional to the current, and its model is as follows:

[0118]

[0119] Where N represents the number of batteries; M Hrepresents the molar mass of hydrogen; n represents the number of transferred electrons; and F represents the Faraday constant.

[0120] Furthermore, the lithium battery model is shown in the following formula:

[0121]

[0122] Among them, i B Represents the battery current; V oc represents the open circuit voltage; R0 represents the ohmic resistance; R p and C p Represents polarization resistance and polarization capacitance, subscript k represents the current kth time, P B Indicates the output power of the lithium battery, U p represents the voltage at the terminals of the RC network, ΔT represents the sampling time, and τ is equal to R p Multiply by C p , η B Indicates battery efficiency, Q B Indicates the battery capacity.

[0123] According to an embodiment of the present invention, a hybrid power system model, a speed prediction method, and an energy management strategy for a fuel cell hybrid power system are provided. Based on the hybrid power system model, the vehicle's required power, fuel cell output power, lithium battery output power, and lithium battery SoC are derived. These data will serve as the state and action space for reinforcement learning. The Q-value function is updated through a dual-Q learning strategy to avoid the maximum deviation caused by the Q-learning algorithm when selecting actions. In addition, a real-time driving mode classification and speed prediction method based on data-driven and statistical analysis can predict the near-term vehicle speed, thereby improving the real-time and generalization of the strategy. Finally, the output power of the fuel cell is obtained by training the intelligent agent.

[0124] Example 2

[0125] Based on the same inventive concept, this embodiment proposes a hydrogen-electric hybrid system energy management system based on dual-Q learning and real-time speed prediction, which is applied to implement the above-mentioned energy management method. The system includes:

[0126] The driving classification module is used to obtain the driving database of the target vehicle, build a driving mode classifier based on multi-feature extraction and probabilistic neural network, and obtain the classified driving mode;

[0127] The speed prediction module is used to design a multi-step Markov speed predictor. The corresponding Markov chain is obtained through the classified driving mode. The speed of the target vehicle is predicted using the Markov chain to obtain the predicted speed.

[0128] The power distribution module is used to input the predicted speed into the hybrid power system model of the target vehicle, output the vehicle's required power, fuel cell output power, lithium battery output power and lithium battery SoC, optimize the Q learning structure based on the output, update the Q function in the Q learning structure based on the set dual-Q learning strategy, and realize real-time distribution of output power in the hybrid power system according to the updated Q function.

[0129] Example 3

[0130] This embodiment proposes an electronic device, including a processor module and a memory module storing a computer program. When the processor module executes the computer program, it implements the above-mentioned hydrogen-electric hybrid system energy management method based on dual-Q learning and real-time speed prediction.

[0131] Example 4

[0132] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor module, the energy management method for a hydrogen-electric hybrid system based on dual-Q learning and real-time speed prediction as described above is implemented.

[0133] According to the above-mentioned embodiments of the present invention, the present invention provides an energy management strategy for a hydrogen-electric hybrid system based on dual-Q learning and real-time speed prediction, which can perform real-time driving mode classification and speed prediction through data-driven and statistical analysis, and considers real-time tracking of reference behavior trajectories in the design of the reward value function to improve the economy and durability of the system.

[0134] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0135] In addition, the functional modules in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0136] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0137] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A hydrogen-electric hybrid system energy management method based on dual-Q learning and real-time speed prediction, the method comprising the following steps: S1. Obtain the target vehicle's driving database, construct a driving mode classifier based on multi-feature extraction and probabilistic neural network, and obtain the classified driving mode; S2. Design a multi-step Markov speed predictor, obtain a corresponding Markov chain based on the classified driving patterns, and use the Markov chain to predict the speed of the target vehicle to obtain a predicted speed; S3. Inputting the predicted speed into a hybrid power system model of the target vehicle, outputting the vehicle's required power, fuel cell output power, lithium battery output power, and lithium battery SoC; optimizing the Q-learning structure based on the vehicle's required power, fuel cell output power, lithium battery output power, and lithium battery SoC; updating the Q function in the Q-learning structure based on a set dual-Q learning strategy; and implementing real-time distribution of output power in the hybrid power system based on the updated Q function; The step S1 comprises: S101. Establish and label a standard database of different road profiles, classify typical operating conditions into high-speed mode, urban flow mode, and urban congestion mode, and use this database to train a classifier; S102. Obtain a driving database and perform statistical analysis on its time series data to extract standard features that characterize different driving mode behaviors; the standard features include average speed, speed standard deviation, maximum speed, maximum acceleration, minimum acceleration, average value of absolute acceleration, standard deviation of absolute acceleration, percentage of parking time, and percentage of constant speed time; S103. Construct a driving mode classifier using a probabilistic neural network. The classifier structure includes four parts: an input layer, a pattern layer, a summation layer, and a competitive output layer. The number of nodes in the input layer corresponds to the number of standard features. The pattern layer performs a weighted summation operation on the input signal. The summation layer estimates the probabilities of various types using the Parzen method. The competitive output layer outputs the driving mode classification result. In step S103, the summation layer estimates the probabilities of various driving modes by summing them according to the Parzen method, including: Each network unit in the summation layer is connected to the majority unit of the corresponding category, and each unit estimates the sum of the probabilities of various types according to the Parzen method: ; Where, l g represents the number of samples of the g-th class, σ represents the smoothing parameter, which is between 0 and 1. Indicates the first i Neuron-like j data.

2. The hydrogen-electric hybrid system energy management method based on dual-Q learning and real-time speed prediction according to claim 1 is characterized by: The step S2 comprises: S201, fusing standard driving profiles under the same driving mode to form a data set to construct a Markov speed predictor under the mode; S202. Define an n-step state transition probability matrix for future speed prediction: ; in, k represents the kth time; n is the number of sampling points; a ( n + k )and a ( k ) respectively represent the first ( n + k ) and k times the acceleration; Represents when the speed state is , acceleration state is a When, after n Step transfer to acceleration state the number of times; For passing n The number of steps represents the number of steps when the speed state is And the acceleration state is When, after n The number of times the step reaches all acceleration states.

3. The hydrogen-electric hybrid system energy management method based on dual-Q learning and real-time speed prediction according to claim 2 is characterized by: In the step S3, before optimizing the Q learning structure according to the output, the step further includes: Prioritize the Q-learning structure and represent it as a five-tuple: S , A , T , γ , R ),in S is the state space, A is the action space, T is the state transition probability, that is , γ is the discount factor, R is the reward function.

4. The hydrogen-electric hybrid system energy management method based on dual-Q learning and real-time speed prediction according to claim 3 is characterized by: In step S3, optimizing the Q-learning structure according to the output includes: S301, setting state space S : Select power requirement P m and battery SoC as the state space for Q learning S : ; P m is the total power requirement of the system, and SoC is the current charge of the battery; S302: Setting the Action Space A :The fuel cell's rate of change ΔP fc As an action space A , whose unit is kW / s, suppresses the variation of the fuel cell by setting the range as follows: ; S303. Setting reward function R : ; The first term in the denominator represents the proportion of hydrogen consumption to the reward; the second term in the denominator is used to guide the battery SoC According to the set reference value SoC ref Move; the third term in the denominator is used to minimize Δ P fc,k The fourth term in the denominator is the expectation that the fuel cell will operate near its maximum efficiency. w 1,…, w 4 is the weight coefficient.

5. The energy management method for a hydrogen-electric hybrid system based on dual-Q learning and real-time speed prediction according to claim 4 is characterized by: In step S3, the Q function in the Q learning structure is updated based on the set dual-Q learning strategy, and the real-time distribution of the output power in the hybrid system is achieved according to the updated Q function, including: S304: Divide the acquired sample data into two groups, marked as Q 1 and Q 2; S305, use Q 1 set of sample data to determine the maximum action of the hybrid system ; S306, Utilization Q 2 sets of sample data are calculated at the maximum action The following values: , because ,in represents the true estimated value of the hybrid system, Serves as an unbiased estimate of the true value of the hybrid system; S307, Exchange Q 1 and Q For the role of 2, repeat steps S305-S306 to obtain another unbiased estimate; S308 : Allocate the output power in the hybrid power system in real time according to the obtained unbiased estimated value.

6. A hydrogen-electric hybrid system energy management system based on dual-Q learning and real-time speed prediction, characterized in that: Applied to executing the energy management method according to any one of claims 1 to 5, the system comprises: The driving classification module is used to obtain the driving database of the target vehicle, build a driving mode classifier based on multi-feature extraction and probabilistic neural network, and obtain the classified driving mode; The speed prediction module is used to design a multi-step Markov speed predictor, obtain the corresponding Markov chain based on the classified driving mode, and use the Markov chain to predict the speed of the target vehicle to obtain the predicted speed; a power distribution module for inputting the predicted speed into a hybrid power system model of a target vehicle, outputting the vehicle's required power, fuel cell output power, lithium battery output power, and lithium battery SoC; optimizing a Q-learning structure based on the outputs; updating a Q function in the Q-learning structure based on a set dual-Q learning strategy; and achieving real-time distribution of output power in the hybrid power system based on the updated Q function.

7. An electronic device, characterized in that: The system comprises a processor module and a memory module storing a computer program, wherein when the processor module executes the computer program, the energy management method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor module, the energy management method according to any one of claims 1 to 5 is implemented.

Images