A methanol extended-range hybrid vehicle energy management method and system

Abstract

The application discloses a methanol extended-range hybrid vehicle energy management method and system, relates to the technical field of new energy vehicle energy management, and comprises the following steps: taking a vehicle operating state as a state variable, taking an MPC algorithm terminal weight factor as an action variable, constructing a reward function based on a methanol consumption cost, a battery SOH and a battery SOC, and constructing an SAC network; the SAC network is trained, state variables under different working conditions are obtained, and the maximum Q value of a Critic network is selected according to the state variables; in combination with a predicted vehicle speed, a whole vehicle demand power and a battery SOC, a methanol engine output power strategy is obtained through an MPC algorithm, and the state variables are updated after the strategy is executed; and the trained SAC network is used to obtain an optimal methanol engine output power strategy. The method combines deep reinforcement learning and model predictive control, realizes optimal solution and safety guarantee at the same time, and improves the economy and durability of the methanol extended-range hybrid vehicle.

Description

Technical Field

[0001] This invention relates to the field of energy management technology for new energy vehicles, and in particular to an energy management method and system for methanol range-extended hybrid vehicles. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Currently, the energy management strategies of the energy management system (EMS) for hybrid electric vehicles (HEVs) are mainly divided into rule-based methods, optimization-based methods, and learning-based methods.

[0004] Rule-based methods have a simple structure, but they rely on engineering experience and require pre-setting control rules and logic, making them less adaptable to complex and changing working conditions.

[0005] Optimization-based methods typically achieve energy allocation by constructing an optimization problem. On one hand, dynamic programming can be used to find the global optimum, but this method has high computational complexity and is difficult to meet real-time control requirements. On the other hand, model predictive control (MPC) can be used to solve the optimization problem online in a rolling manner, but it is often difficult to obtain the global optimum in multi-objective optimization and nonlinear complex systems, and its control performance is constrained by the trade-off between the prediction time domain length and computational resources.

[0006] Learning-based methods have gradually become a research hotspot with the development of machine learning, especially deep reinforcement learning (DRL) methods, represented by Soft Actor-Critic (SAC), which have shown fast response speed and good control performance in practical applications. However, under different operating conditions, they can usually only obtain approximate optimal solutions, and they have shortcomings in constraint handling, affecting the safety and reliability of the system. Summary of the Invention

[0007] To address the aforementioned issues, this invention proposes an energy management method and system for methanol range-extended hybrid vehicles. It employs a combination of deep reinforcement learning and model predictive control to simultaneously achieve optimal solutions and ensure safety, thereby improving the economy and durability of methanol range-extended hybrid vehicles.

[0008] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides an energy management method for a methanol range-extended hybrid vehicle, comprising: The SAC network is constructed by taking the vehicle's operating state as the state variable, the MPC algorithm terminal weight factor as the action variable, and constructing a reward function based on methanol consumption cost, battery SOH, and battery SOC. The SAC network is trained. The training process includes: obtaining state variables under different operating conditions, selecting the action variables corresponding to the maximum Q value of the Critic network, combining the predicted vehicle speed, vehicle power demand and battery SOC for a set future time period, obtaining the methanol engine output power strategy through the MPC algorithm, updating the state variables after executing the strategy, until the set number of iterations is reached to complete the training. The optimal methanol engine output power strategy is obtained by using the trained SAC network based on the state variables under the current operating conditions.

[0009] As an alternative implementation method, the reward function of the SAC network is: ; in, This represents methanol consumption. For MPC The output power of the methanol engine at any given time. For MPC The output power of the methanol engine at any given time. This is the maximum output power of the methanol engine. It is the change in SOH of the battery. This is the battery SOC reference value. for The state of charge of the battery at any given time. , , These are the weighting coefficients.

[0010] As an alternative implementation method, a trained LSTM network is used to predict vehicle speeds for a predetermined future time period, and the required power of the vehicle is calculated based on the predicted vehicle speeds for that time period. The formula for calculating the required power of the vehicle is as follows: ; in, For vehicle quality, Acceleration caused by gravity The rolling resistance coefficient, For road slope, The air drag coefficient, This refers to the front area of ​​the vehicle. The rotational mass coefficient, The speed is the vehicle speed.

[0011] As an alternative implementation method, the vehicle battery SOC for a future predetermined period is obtained based on the vehicle's RC equivalent circuit model; the vehicle RC equivalent circuit model is as follows: ; ; ; in, and Indicates terminal voltage and load current. This is the battery open-circuit voltage. and The polarization voltages of the two RC branches are... , , , , For model parameters, This is the initial capacity of the battery. This refers to the battery's nominal capacity. for The state of charge of the vehicle's battery at any given time.

[0012] As an alternative implementation method, the objective function of the MPC algorithm is: ; in, This represents methanol consumption. To predict step size The output power of the methanol engine; To predict step size The output power of the methanol engine; This is the minimum output power of the methanol engine. This represents the maximum output power of the methanol engine. To predict step size The values ​​of the state variables under the following conditions; and These are the lower and upper bounds of the state variable, respectively; This serves as a reference value for the state variable. It is the change in battery health status. , , These are the weighting coefficients.

[0013] As an alternative implementation method, the constraints of the MPC algorithm are: ;in, and Battery output power The lower and upper limits.

[0014] Secondly, the present invention provides an energy management system for a methanol range-extended hybrid vehicle, comprising: The network construction module is configured to use the vehicle operating state as the state variable, the MPC algorithm terminal weight factor as the action variable, and to construct a reward function based on methanol consumption cost, battery SOH and battery SOC, thereby constructing the SAC network. The training module is configured to train the SAC network. The training process includes: acquiring state variables under different operating conditions, selecting the action variables corresponding to the maximum Q value of the Critic network, combining the predicted vehicle speed, vehicle power demand and battery SOC for a set future time period, obtaining the methanol engine output power strategy through the MPC algorithm, updating the state variables after executing the strategy, until the set number of iterations is reached to complete the training. The execution module is configured to use the trained SAC network to obtain the optimal methanol engine output power strategy based on the state variables under the current operating conditions.

[0015] Thirdly, the present invention provides an electronic device including a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the method described in the first aspect.

[0016] Fourthly, the present invention provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in the first aspect.

[0017] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0018] Compared with the prior art, the beneficial effects of the present invention are as follows: To address the shortcomings of existing energy management strategies in terms of adaptability, optimality, real-time performance, and constraint satisfaction, this invention proposes an energy management method for methanol range-extended hybrid vehicles. The method uses vehicle operating state as the state variable, the terminal weight factor of the MPC algorithm as the action variable, and constructs a reward function based on methanol consumption cost, battery SOH, and battery SOC to build a SAC network. During the training of the SAC network, the action variable corresponding to the maximum Q value of the Critic network is selected. Combined with the predicted vehicle speed, total vehicle power demand, and battery SOC for a future time period, the methanol engine output power strategy is obtained through the MPC algorithm. By embedding the MPC into the SAC environment for training, the safety of the DRL method's actions is ensured. The terminal weight factor provided by the SAC network improves the global optimality of the MPC, while simultaneously enhancing the hybrid vehicle's economy, durability, and real-time computing speed, achieving superior overall performance under complex real-world conditions.

[0019] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0021] Figure 1 This is a flowchart of the energy management method for methanol range-extended hybrid vehicles provided in Embodiment 1 of the present invention; Figure 2 This is a vehicle structure diagram provided in Embodiment 1 of the present invention; Figure 3 This is a graph showing the methanol engine power output of the methanol range-extended hybrid vehicle energy management method provided in Embodiment 1 of the present invention. Figure 4 A graph showing the power output of a methanol engine using the SAC algorithm alone; Figure 5 A graph showing the power output of a methanol engine using the MPC algorithm alone; Figure 6 The result diagram shows the SOC maintenance situation. Detailed Implementation

[0022] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0023] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0024] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. Furthermore, it should be understood that the terms “comprising” and “including”, and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0026] Example 1 This embodiment provides an energy management method for methanol range-extended hybrid vehicles, such as... Figure 1 As shown, it specifically includes: The SAC network is constructed by taking the vehicle's operating state as the state variable, the MPC algorithm terminal weight factor as the action variable, and constructing a reward function based on methanol consumption cost, battery SOH, and battery SOC. The SAC network is trained. The training process includes: obtaining state variables under different operating conditions, selecting the action variables corresponding to the maximum Q value of the Critic network, combining the predicted vehicle speed, vehicle power demand and battery SOC for a set future time period, obtaining the methanol engine output power strategy through the MPC algorithm, updating the state variables after executing the strategy, until the set number of iterations is reached to complete the training. The optimal methanol engine output power strategy is obtained by using the trained SAC network based on the state variables under the current operating conditions.

[0027] In this embodiment, an SAC network is constructed, with the vehicle operating state as the state variable S, the MPC algorithm terminal weight factor as the action variable A, and a reward function R constructed based on methanol consumption cost, battery SOH, and battery SOC.

[0028] Specifically: In this embodiment, a methanol range-extended vehicle was selected as the experimental vehicle model. Figure 2 This is a diagram of the vehicle's powertrain system structure.

[0029] The power requirement for vehicle operation is: ; in, For vehicle quality, Acceleration caused by gravity The rolling resistance coefficient, For road slope, The air drag coefficient, This refers to the front area of ​​the vehicle. For vehicle speed, is the rotational mass coefficient.

[0030] The equivalent circuit model of the second-order RC circuit is as follows: ; ; ; in, and Indicates terminal voltage and load current. This is the battery open-circuit voltage. and The polarization voltages of the two RC branches are... , , , , For model parameters, This is the initial capacity of the battery. This refers to the battery's nominal capacity. for The state of charge of the vehicle's battery at any given time.

[0031] For the SAC network, the state variables include battery SOC (State of Charge), driving speed (vehicle speed), acceleration, power demand, and the action variable A from the previous moment.

[0032] reward function for: ; in, This represents methanol consumption. For MPC The output power of the methanol engine at any given time. For MPC The output power of the methanol engine at any given time. This is the maximum output power of the methanol engine. It is the change in battery state of health (SOH). This is the battery SOC reference value. for The state of charge of the battery at any given time. , , These are the weighting coefficients.

[0033] The reward function aims to simultaneously optimize fuel economy, battery system durability, and battery SOC stability, which are important considerations for the operation of hybrid vehicles.

[0034] Regarding parameter design, and These represent the actual prices of methanol and batteries, respectively, thus unifying them into the same economic dimension. and All are experimental parameters. When If its value is too high, the SAC agent will excessively suppress the power fluctuations of the methanol engine, thus affecting the optimality of the results; therefore, this value is set to 0.01. The purpose of introducing this term is to ensure that the reward function in the DRL method is consistent with the penalty function in the MPC method. Setting input terms in MPC helps to guarantee the stability and computational convenience of the solution process. Setting the value too high will suppress SOC fluctuations, while setting it too low will maximize battery utilization. An experimental value of 1000 was used. These parameters have been shown to have good effects on fuel economy, battery system durability, and battery SOC stability.

[0035] After constructing the SAC network, the hyperparameters of the SAC network are first initialized. The hyperparameters include: discount rate of 0.99, actor learning rate of 0.0001, critic learning rate of 0.001, experience revisit pool capacity of 100,000, mini-batch size of 1024, and maximum learning period of 500.

[0036] Then, the SAC network is trained using data from different operating conditions as the training set, specifically including: (1) Read the state variable S under the current working condition, and select the action variable A corresponding to the maximum Q value of the Critic network.

[0037] (2) Based on the current vehicle operating status, the trained LSTM network is used to predict the vehicle speed for a future set period. Based on the predicted vehicle speed for a future set period, the power demand formula for the vehicle during operation is used to calculate the vehicle demand power P. The vehicle battery SOC for a future set period is obtained based on the vehicle RC equivalent circuit model. Then, combined with the action variable A obtained in step (1), the methanol engine output power sequence is obtained through the model prediction control algorithm.

[0038] The LSTM network was trained using data from different working conditions as the training set. The hyperparameters of the LSTM network were as follows: input sequence length 5, output sequence length 5, memory unit 64, mini-batch size 128, and maximum training period 50.

[0039] Specifically: For the MPC algorithm, the state variable is the vehicle battery SOC, and the action variable is the methanol engine output power.

[0040] Regarding the constraints: ; The objective function is: ; in, This represents the current value within the prediction step size; To predict step size The output power of the methanol engine; To predict step size The output power of the methanol engine; This is the minimum output power of the methanol engine. This represents the maximum output power of the methanol engine. To predict step size The values ​​of the state variables under the following conditions; and These are the lower and upper bounds of the state variable, respectively; This serves as a reference value for the state variable. It is the change in battery health status. and Battery output power The lower and upper limits; is the weighting coefficient, representing the output power of the methanol engine output by the actor network in the SAC network.

[0041] (3) Select the first position of the methanol engine output power sequence, and after execution, obtain the returned state variable S' and reward R.

[0042] (4) Update the evaluation network parameters through backpropagation, update the target network parameters after a set number of steps, and update the state variable S at the same time.

[0043] (5) Repeat steps (1)-(4) until the set number of iterations is reached or convergence is achieved, then stop training the SAC network.

[0044] Finally, the trained SAC network is used to repeat the above steps for energy management based on the state variables under the current operating conditions, so as to obtain the optimal methanol engine output power strategy and effectively improve EMS performance under different real driving conditions.

[0045] This embodiment compares the performance of three model algorithms, namely the MPC algorithm alone, the SAC algorithm alone, and the method of this embodiment, under real-world conditions.

[0046] The methanol engine power output of the method, the SAC algorithm alone, and the MPC algorithm in this embodiment is as follows: Figures 3-5 As shown, the methanol engine power of this embodiment is kept in the high-efficiency range as much as possible, and the power fluctuates with the change in power demand.

[0047] SOC maintenance status as follows: Figure 6 As shown, the method in this embodiment has better SOC stability and can keep the initial and final SOC times consistent.

[0048] The methanol consumption comparison results are shown in Table 1. In this embodiment, the hybrid vehicle energy management strategy combining deep reinforcement learning and model predictive control showed that methanol consumption per 100 kilometers was reduced by 11.2% compared to the SAC algorithm alone and by 5.8% compared to the MPC algorithm alone, demonstrating better fuel economy than the other two methods.

[0049] Table 1. Comparison of methanol consumption; .

[0050] Example 2 This embodiment provides an energy management system for a methanol range-extended hybrid vehicle, including: The network construction module is configured to use the vehicle operating state as the state variable, the MPC algorithm terminal weight factor as the action variable, and to construct a reward function based on methanol consumption cost, battery SOH and battery SOC, thereby constructing the SAC network. The training module is configured to train the SAC network. The training process includes: acquiring state variables under different operating conditions, selecting the action variables corresponding to the maximum Q value of the Critic network, combining the predicted vehicle speed, vehicle power demand and battery SOC for a set future time period, obtaining the methanol engine output power strategy through the MPC algorithm, updating the state variables after executing the strategy, until the set number of iterations is reached to complete the training. The execution module is configured to use the trained SAC network to obtain the optimal methanol engine output power strategy based on the state variables under the current operating conditions.

[0051] It should be noted that the above modules correspond to the steps described in Embodiment 1, and the examples and application scenarios implemented by the above modules and the corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1. It should also be noted that the above modules, as part of the system, can be executed in a computer system such as a set of computer-executable instructions.

[0052] In further embodiments, the following is also provided: An electronic device includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the method described in Embodiment 1. For brevity, further details are omitted here.

[0053] It should be understood that in this embodiment, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0054] Memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of memory may also include non-volatile random access memory. For example, memory may also store information about the device type.

[0055] A computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in Embodiment 1.

[0056] The method in Example 1 can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor. The software modules can reside in readily available storage media in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not provided here.

[0057] A computer program product includes a computer program that, when executed by a processor, implements the method described in Embodiment 1.

[0058] The present invention also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions included in program modules, which execute in a device on a target real or virtual processor to perform the processes / methods described above. Typically, program modules include routines, programs, libraries, objects, classes, components, data structures, etc., that perform specific tasks or implement specific abstract data types. In various embodiments, the functionality of program modules can be combined or divided among program modules as needed. The machine-executable instructions for the program modules can execute within a local or distributed device. In a distributed device, the program modules can reside in both local and remote storage media.

[0059] The computer program code used to implement the methods of the present invention may be written in one or more programming languages. This computer program code may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the computer or other programmable data processing device, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a computer, partially on a computer, as a stand-alone software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server.

[0060] In the context of this invention, computer program code or related data may be carried by any suitable carrier to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, and the like. Examples of signals may include electrical, optical, radio, sound, or other forms of propagation signals, such as carrier waves, infrared signals, etc.

[0061] Those skilled in the art will recognize that the units and algorithm steps described in connection with the various examples of this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention.

[0062] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. An energy management method for methanol range-extended hybrid vehicles, characterized in that, include: The SAC network is constructed by taking the vehicle's operating state as the state variable, the MPC algorithm terminal weight factor as the action variable, and constructing a reward function based on methanol consumption cost, battery SOH, and battery SOC. The SAC network is trained. The training process includes: obtaining state variables under different operating conditions, selecting the action variables corresponding to the maximum Q value of the Critic network, combining the predicted vehicle speed, vehicle power demand and battery SOC for a set future time period, obtaining the methanol engine output power strategy through the MPC algorithm, updating the state variables after executing the strategy, until the set number of iterations is reached to complete the training. The optimal methanol engine output power strategy is obtained by using the trained SAC network based on the state variables under the current operating conditions.

2. The energy management method for a methanol range-extended hybrid vehicle as described in claim 1, characterized in that, The reward function of the SAC network is: ； in, This represents methanol consumption. For MPC The output power of the methanol engine at any given time. For MPC The output power of the methanol engine at any given time. This is the maximum output power of the methanol engine. It is the change in SOH of the battery. This is the battery SOC reference value. for The state of charge of the battery at any given time. , , These are the weighting coefficients.

3. The energy management method for a methanol range-extended hybrid vehicle as described in claim 1, characterized in that, The trained LSTM network is used to predict vehicle speeds over a predetermined time period. The required power for the entire vehicle is then calculated based on these predicted speeds. The formula for calculating the required power for the entire vehicle is as follows: ； in, For vehicle quality, Acceleration caused by gravity The rolling resistance coefficient, For road slope, The air drag coefficient, This refers to the front area of ​​the vehicle. For vehicle speed, is the rotational mass coefficient.

4. The energy management method for a methanol range-extended hybrid vehicle as described in claim 1, characterized in that, The vehicle battery SOC for a future specified time period is obtained based on the vehicle's RC equivalent circuit model; the vehicle's RC equivalent circuit model is as follows: ； ； ； in, and Indicates terminal voltage and load current. This is the battery open-circuit voltage. and The polarization voltages of the two RC branches are... , , , , For model parameters, This is the initial capacity of the battery. This refers to the battery's nominal capacity. for The state of charge of the vehicle's battery at any given time.

5. The energy management method for a methanol range-extended hybrid vehicle as described in claim 1, characterized in that, The objective function of the MPC algorithm is: ； in, This represents methanol consumption. To predict step size The output power of the methanol engine; To predict step size The output power of the methanol engine; This is the minimum output power of the methanol engine. This represents the maximum output power of the methanol engine. To predict step size The state variable values ​​are as follows; and These are the lower and upper bounds of the state variable, respectively; This serves as a reference value for the state variable. It is the change in battery health status. , , These are the weighting coefficients.

6. The energy management method for a methanol range-extended hybrid vehicle as described in claim 5, characterized in that, The constraints of the MPC algorithm are: ;in, and Battery output power The lower and upper limits.

7. An energy management system for a methanol range-extended hybrid vehicle, characterized in that, include: The network construction module is configured to use the vehicle operating state as the state variable, the MPC algorithm terminal weight factor as the action variable, and to construct a reward function based on methanol consumption cost, battery SOH and battery SOC, thereby constructing the SAC network. The training module is configured to train the SAC network. The training process includes: acquiring state variables under different operating conditions, selecting the action variables corresponding to the maximum Q value of the Critic network, combining the predicted vehicle speed, vehicle power demand and battery SOC for a set future time period, obtaining the methanol engine output power strategy through the MPC algorithm, updating the state variables after executing the strategy, until the set number of iterations is reached to complete the training. The execution module is configured to use the trained SAC network to obtain the optimal methanol engine output power strategy based on the state variables under the current operating conditions.

8. An electronic device, characterized in that, It includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, Used to store computer instructions, which, when executed by a processor, perform the method described in any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, implements the method described in any one of claims 1-6.

Images