A-ECMS energy management method based on predictive road traffic information

Abstract

The invention belongs to the technical field of vehicle energy management, and discloses an A-ECMS energy management method based on predictive road traffic information, and the method comprises the steps: obtaining the historical driving data of a vehicle, and calculating the feature parameters of the historical driving data; inputting the characteristic parameters into an off-line driving data recognition library, and obtaining current optimized power control parameters; acquiring predictive road data in front of the vehicle, and inputting the predictive road data into an off-line driving data recognition library for scene recognition to obtain a predicted scene and a corresponding predictive optimization power control parameter; calculating a long-term energy track based on an online dynamic planning algorithm; calculating a final power control parameter based on a DRL algorithm in combination with the current optimized power control parameter, the predicted optimized power control parameter, the long-term energy track and the current energy track; and calculating an optimal control sequence of the vehicle power system in real time according to the final power control parameters. According to the technical scheme, global optimal energy management can be realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle energy management, and in particular relates to an A-ECMS energy management method based on predictive road traffic information. Background Art

[0002] With the rapid development of the global economy and the continuous growth of population, the demand for energy is increasing. Traditional fuel vehicles rely on fossil fuels such as oil, but these resources are limited and face the crisis of depletion. The large-scale use of fuel vehicles has led to serious air pollution and greenhouse gas emissions, exacerbating environmental problems such as global warming. Therefore, finding clean and efficient ways to use energy has become a top priority.

[0003] Before pure electric vehicle technology is fully mature, hybrid vehicles, as a transitional model, combine the advantages of traditional fuel vehicles and pure electric vehicles, which can not only reduce fuel consumption and emissions, but also maintain a long driving range and good power performance. The core of hybrid vehicles lies in the design and control of their energy management system. Through reasonable energy management strategies, it is possible to achieve optimal configuration and coordinated work of multiple power sources (such as engines, motors and batteries), thereby improving the energy utilization efficiency of the entire vehicle.

[0004] Traditional energy management strategies, including rule-based and instantaneous optimization-based, mainly rely on instantaneous driving cycle information, which is determined by the driver's immediate operation. They require a lot of engineering experience to pre-calibrate the internal rules and control parameters of the strategy. During the control process, fixed rules and control parameters cannot adapt to the complex and changing road environment and driving needs, which will lead to problems such as low energy utilization efficiency, high fuel consumption and increased emissions. Energy management strategies based on global optimization (such as dynamic programming, convex optimization, genetic algorithms, etc.) can provide theoretical optimal solutions, but their calculation volume is huge and difficult to apply in real time. Energy management strategies based on intelligent algorithms (such as model predictive control, reinforcement learning, etc.) optimize control strategies through learning and training. Although they have strong adaptability, they have high requirements for algorithm training and calculation, and are highly dependent on the accuracy of the prediction model. In summary, traditional energy management strategies that rely on instantaneous driving cycle information lack adaptability.

[0005] Although the energy management strategy based on working condition recognition can switch between multiple preset control parameter schemes in real time according to the historical driving information of the past short period of time, it lacks the vision of the traffic flow in the road ahead and still cannot respond in advance to the unknown complex traffic environment in the road ahead; the energy management strategy based on forward-looking driving cycle prediction introduces forward-looking information of the road ahead into energy management, and optimizes energy distribution by predicting driving cycles in the short term, but such algorithms usually require more powerful computing power and are difficult to implement in actual vehicles, and forward-looking information cannot guarantee global energy optimization; the energy management strategy based on global driving cycle prediction further expands the vision of energy management, takes into account the entire travel process, and combines cloud computing and edge computing platforms to provide decision support for vehicles, but the traffic flow information in different sections of the journey changes over time, which will cause the globally planned energy management strategy to be unstable. Summary of the invention

[0006] The purpose of the present invention is to provide an A-ECMS energy management method based on predictive road traffic information to solve the problems existing in the above-mentioned prior art.

[0007] To achieve the above object, the present invention provides an A-ECMS energy management method based on predictive road traffic information, comprising:

[0008] Acquire historical driving data of the vehicle, and calculate characteristic parameters of the historical driving data;

[0009] Inputting the characteristic parameters into an offline driving data recognition library to obtain the current optimized power control parameters; wherein the offline driving data recognition library is a multi-dimensional power parameter optimization library including different driving styles, driving scenes, initial SOC and target SOC;

[0010] Obtaining predictive road data ahead of the vehicle, inputting the predictive road data into the offline driving data recognition library for scene recognition, and obtaining a predicted scene and a corresponding predicted optimized power control parameter; using an online dynamic programming algorithm to calculate a long-term energy trajectory;

[0011] Based on the DRL algorithm, the final power control parameter is calculated by combining the current optimized power control parameter, the predicted optimized power control parameter, the long-term energy trajectory and the current energy trajectory;

[0012] According to the final power control parameters, the optimal control sequence of the vehicle power system is calculated in real time.

[0013] Optionally, the construction process of the offline driving data recognition library includes:

[0014] Acquiring raw data, the raw data including driving-related data, and performing data cleaning and normalization processing on the raw data to obtain pre-processed raw data;

[0015] Perform principal component analysis on the preprocessed raw data to obtain dimension-reduced data;

[0016] Selecting a number of data points from the dimension reduction data as initial cluster centers for cluster analysis to obtain driving scene type data and driving style type data with different characteristic parameters;

[0017] Based on the supervised learning recognition algorithm, a nonlinear relationship between feature parameters and driving scenes and driving styles is constructed, and a driving scene library and a driving style library are established; the feature parameters include driving scene feature parameters and driving style feature parameters;

[0018] The library inputs of the driving scene library and the driving style library are driving scene feature parameters and driving style feature parameters respectively, and the outputs are corresponding driving scenes and driving styles;

[0019] Several initial energy trajectories and target energy trajectories are set, and based on the intelligent optimization algorithm, typical driving conditions with different driving styles, initial energy trajectories and target energy trajectories are iteratively optimized with the goal of minimizing comprehensive fuel consumption, and the corresponding optimized power control parameters are obtained. A multi-dimensional power parameter optimization library with different driving styles, driving scenarios, initial SOC and target SOC is established.

[0020] Optionally, the driving scene characteristic parameters include maximum positive acceleration, positive acceleration standard deviation, positive acceleration average, acceleration time proportion, maximum negative acceleration, negative acceleration standard deviation, negative acceleration average, negative acceleration time proportion, maximum vehicle speed, vehicle speed standard value, average vehicle speed, average acceleration, acceleration standard deviation, average vehicle speed in the cruising section and cruising time proportion.

[0021] Optionally, the driving style characteristic parameters include maximum positive acceleration, positive acceleration standard deviation, positive acceleration average, positive jerk standard deviation, positive jerk average, acceleration time proportion, maximum negative acceleration, negative acceleration standard value, negative acceleration average, negative acceleration time proportion, negative jerk standard deviation, negative jerk average, maximum vehicle speed, average vehicle speed, average acceleration, acceleration standard value, average vehicle speed in the cruising section, cruising time proportion, accelerator pedal standard deviation, accelerator pedal change rate standard deviation, brake pedal standard deviation, brake pedal change rate standard deviation, average jerk, and jerk standard deviation.

[0022] Optionally, the characteristic parameters are input into an offline driving data recognition library to obtain current optimized power control parameters, specifically including:

[0023] The characteristic parameters are input into an offline driving data recognition library for scene recognition and style recognition to obtain typical driving scenes and driving styles corresponding to the characteristic parameters. The offline driving data recognition library outputs the corresponding current energy trajectory, target energy trajectory and current optimized power control parameters based on the recognized typical driving scenes and driving styles.

[0024] Optionally, the predictive road data includes the green wave speed range, the remaining mileage of the entire current driving route, the estimated remaining time for the entire current driving route, the average speed of each section of the entire driving route, the congestion conditions of each section of the entire driving route, the distance at the average speed of each section of the entire driving route, the distance of the congested section and the passing time of the congested section.

[0025] Optionally, an online dynamic programming algorithm is used to calculate the long-term energy trajectory based on the full-course predictive road traffic information. The specific calculation formula is:

[0026]

[0027] The constraints are:

[0028]

[0029] The cost function is:

[0030] V L =V eng / L (T eng ,N eng )+V batt / L (P batt )

[0031] In the formula, E m is the battery open-loop voltage, R int is the internal resistance of the battery, Q c is the total battery capacity, SOC k is the SOC of the kth stage, SOC k+1 is the SOC of the (k+1)th stage, is the average battery power in the kth stage, is the average engine power in the kth stage, is the average engine speed in the kth stage; SOC min and SOC max are the lower and upper limits of SOC, P batt / chg and P batt / dischg are the battery charging power and battery discharging power, T min and T max are the lower and upper limits of engine torque, N min and N max are the lower and upper limits of engine speed, Veng / L and V batt / L are the engine fuel consumption and the battery equivalent fuel consumption, V L For the comprehensive fuel consumption.

[0032] Optionally, based on the DRL algorithm, the final power control parameter is calculated by combining the current optimized power control parameter, the predicted optimized power control parameter, the long-term energy trajectory and the current energy trajectory. The specific calculation formula is:

[0033] s final (t) = F(s t (t),s pre (t),SOC ref (t),SOC t (t))

[0034] Among them, s final is the final power control parameter of A-ECMS, s t is the power control parameter based on short-term working condition identification; s pre The power control parameters corresponding to the road conditions ahead based on predictive road information identification; SOC ref is the long-term energy trajectory obtained based on online dynamic programming; SOC t is the SOC at the current moment, and F is the power control parameter adjustment strategy trained using the deep reinforcement learning algorithm.

[0035] Optionally, according to the final power control parameters, the optimal control sequence of the vehicle power system is calculated in real time, specifically including:

[0036] Constructing a discrete sequence set, wherein the discrete sequence set includes an engine discrete sequence, a motor discrete sequence, and a battery discrete sequence;

[0037] Setting instantaneous optimization constraints of the ECMS based on the discrete sequence set;

[0038] Based on the instantaneous optimization constraint condition and the final power control parameter, an instantaneous optimization objective function is constructed with the goal of minimizing the equivalent fuel consumption of the vehicle power system;

[0039] The instantaneous optimization objective function is solved to obtain the optimal control sequence of the vehicle power system at the current moment.

[0040] The technical effects of the present invention are:

[0041] In order to effectively utilize the big data platform and real-time traffic flow information, improve the ability of energy management strategies to cope with complex working conditions and random traffic environments, and make them close to the global optimal effect, this patent proposes an A-ECMS energy management method based on predicted road traffic information. At the current moment, the A-ECMS energy management strategy based on working condition identification is used to adjust the power control parameters to optimize the power distribution; the working condition is identified by introducing the traffic flow information of the road ahead, and the power control parameters of the road ahead are obtained; by introducing the global driving cycle information of the entire driving process, a simplified online dynamic programming strategy is used to obtain the optimal energy trajectory for the entire process; finally, based on the deep reinforcement learning algorithm, the current power control parameters, the power control parameters of the road ahead and the optimal energy trajectory for the entire process are used as state quantities, and the final power control parameters are used as action quantities to train the deep network, and are applied to calculate the final power control parameters in real time. This energy management strategy combines the advantages of multiple energy management strategies, can cope with complex and changeable traffic environments, has low computational complexity, strong robustness, and has a forward-looking vision and a global vision, and can achieve an energy management strategy close to the global optimal.

[0042] The present invention combines the advantages of multiple energy management strategies and uses an energy management strategy based on working condition identification to improve the adaptability of the energy management strategy to working conditions, which is more robust than other prediction algorithms. In terms of driving vision, instantaneous optimization based on the current driving state, short-term prediction optimization based on forward-looking road information, and global energy trajectory planning based on full-journey traffic flow information are considered respectively. At the same time, the deep reinforcement learning (DRL) algorithm is combined to determine the final power control parameters. This combination of long-term and short-term strategies helps to achieve energy management close to the global optimal level. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0045] Figure 1 Develop a flow chart for the offline portion of an embodiment of the present invention;

[0046] Figure 2 It is a flow chart of principal component analysis in an embodiment of the present invention;

[0047] Figure 3 is a flow chart of the k-means clustering algorithm in an embodiment of the present invention;

[0048] Figure 4 is a flowchart of online energy trajectory planning based on predictive road information in an embodiment of the present invention;

[0049] Figure 5 is a flowchart of reinforcement learning training and application in an embodiment of the present invention;

[0050] Figure 6 This is a diagram of the overall strategy architecture in an embodiment of the present invention;

[0051] Figure 7 A flowchart for driving scene style recognition and power parameter control selection in an embodiment of the present invention;

[0052] Figure 8 The figure is a flowchart of discrete sequence calculation in an embodiment of the present invention. DETAILED DESCRIPTION

[0053] Various exemplary embodiments of the present invention will now be described in detail. This detailed description should not be considered as limiting the present invention, but should be understood as a more detailed description of certain aspects, features, and embodiments of the present invention.

[0054] It should be understood that the terms described in the present invention are only for describing special embodiments and are not intended to limit the present invention. In addition, for the numerical range in the present invention, it should be understood that each intermediate value between the upper and lower limits of the scope is also specifically disclosed. Each smaller range between the intermediate value in any stated value or stated range and any other stated value or intermediate value in the described range is also included in the present invention. The upper and lower limits of these smaller ranges can be independently included or excluded in the scope.

[0055] Unless otherwise indicated, all technical and scientific terms used herein have the same meanings as those of ordinary skill in the art generally understood by the present invention. Although the present invention describes only preferred methods, any method similar or equivalent to that described herein may also be used in the implementation or testing of the present invention. All documents mentioned in this specification are incorporated by reference to disclose and describe the methods related to the documents. In the event of a conflict with any incorporated document, the content of this specification shall prevail.

[0056] It will be apparent to those skilled in the art that various modifications and variations may be made to the specific embodiments of the present invention description without departing from the scope or spirit of the present invention. Other embodiments derived from the present invention description will be apparent to those skilled in the art. The present application description and examples are exemplary only.

[0057] The words “include,” “including,” “have,” “contain,” etc. used in this article are open-ended terms, meaning including but not limited to.

[0058] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0059] Embodiment 1

[0060] like Figure 1 - Figure 8 As shown, in this embodiment, an A-ECMS energy management method (Adaptive-Equivalent ConsumptionMinimization Strategy) based on predictive road traffic information is provided, including: obtaining historical driving data of the vehicle, calculating characteristic parameters of the historical driving data; inputting the characteristic parameters into an offline driving data recognition library to obtain current optimized power control parameters; wherein the offline driving data recognition library is a multidimensional power parameter optimization library including different driving styles, driving scenarios, initial SOC and target SOC; obtaining predictive road data in front of the vehicle, inputting the predictive road data into the offline driving data recognition library for scene recognition, and obtaining predicted scenarios and corresponding predicted optimized power control parameters; using an online dynamic programming algorithm to calculate a long-term energy trajectory; based on a DRL algorithm, calculating the final power control parameters in combination with the current optimized power control parameters, the predicted optimized power control parameters, the long-term energy trajectory and the current energy trajectory; and calculating the optimal control sequence of the vehicle power system in real time according to the final power control parameters.

[0061] In order to effectively utilize the big data platform and real-time traffic flow information, improve the ability of the energy management strategy to cope with complex working conditions and random traffic environments, and make it have a near-global optimal effect, this patent proposes an A-ECMS energy management method based on predicted road traffic information. At the current moment, the A-ECMS energy management strategy based on working condition identification is used to adjust the power control parameters to optimize the power distribution; the working condition is identified by introducing the traffic flow information of the road ahead, and the power control parameters of the road ahead are obtained; by introducing the global driving cycle information of the entire driving process, a simplified online dynamic programming strategy is used to obtain the optimal energy trajectory for the entire process; finally, based on the deep reinforcement learning algorithm, the current power control parameters, the power control parameters of the road ahead and the optimal energy trajectory for the entire process are used as state quantities, and the final power control parameters are used as action quantities to train the deep network, and are applied to calculate the final power control parameters in real time. This energy management strategy combines the advantages of multiple energy management strategies, can cope with complex and changeable traffic environments, has low computational complexity, strong robustness, and has a forward-looking vision and a global vision, and can achieve an energy management strategy close to the global optimal.

[0062] Figure 1 The offline development process includes the development of a driving style and driving scenario library and classification recognition system based on offline big data learning, and the development of a driving style and scene recognition power control parameter optimization library based on offline global optimization.

[0063] 1. Development of driving style and driving scenario library and classification recognition system based on offline big data learning

[0064] The offline big data learning model aims to establish a typical driving scene library and a driving style library to provide data support and data processing basis for online work. The learning model is divided into data analysis, unsupervised scene clustering and driving style clustering, clustering effect evaluation, classification recognition training and other parts. It is developed in the MATLAB environment using M language, and mathematical tools such as PCA principal component analysis, k-means unsupervised clustering algorithm, and distance discrimination method are used to establish online driving scene identifiers and driving style identifiers. The clustering effect is evaluated by clustering evaluation indicators such as silhouette coefficient and Calinski-Harabasz index; the accuracy of the driving identifier is evaluated by mathematical evaluation indicators such as root mean square error.

[0065] Offline big data learning analyzes the driving-related data provided in the database, obtains various driving styles and driving scene types through algorithm clustering, and establishes a typical driving scene library and a driving style library. The original driving data obtained through testing is obtained from the database. These data cover most of the characteristic parameters that can represent driving styles and driving scenes during driving, as shown in the following table:

[0066] Table 1 Driving scene characteristic parameters

[0067] Serial number Driving scene characteristic parameters 1 maximum positive acceleration 2 Positive acceleration standard deviation 3 Positive acceleration average 4 Acceleration time ratio 5 Maximum negative acceleration 6 Negative acceleration standard deviation 7 Negative acceleration average 8 Negative acceleration time ratio 9 Maximum speed 10 Speed ​​standard value 11 Average speed 12 Average acceleration 13 Acceleration standard deviation 14 Average speed during cruising 15 Percentage of cruising time

[0068] Table 2 Driving style characteristic parameters

[0069]

[0070]

[0071] First, the data is processed to fill in the outliers that may appear in the data, clean the noise, and normalize the data to eliminate the interference of data magnitude differences on clustering. PCA principal component analysis is used to reduce the dimension of feature parameters and set the cumulative contribution threshold. N principal components with cumulative contributions exceeding the threshold are selected as clustering parameters to reduce the number of clustering parameters, making the clustering parameters easy to calculate and highly representative. The principal component analysis process is as follows: Figure 2 As shown;

[0072] In the figure, the coefficient matrix C is:

[0073]

[0074] Subsequently, the CH index and DB index are used to evaluate the clustering effect of the characteristic parameter matrix under different numbers of cluster centers. The calculation formula of the CH index is:

[0075]

[0076] Among them, B k is the covariance matrix between classes, W k is the covariance matrix of the intra-class data, and the detailed formula is

[0077]

[0078]

[0079] where c q represents the center point of class q, c e Represents the center point of the data set, n q represents the number of data in class q, C q represents the data set of class q. The calculation formula of DB index is

[0080]

[0081] where s i Indicates the dispersion of sample points in the class, M ij is the distance dispersion between the two cluster centers.

[0082] After calculating the CH index and DB index, the best performing cluster centers are selected comprehensively, and the k-means unsupervised clustering algorithm is used for clustering to obtain driving scene types (such as congested scenes, high-speed scenes, etc.) and driving style types (such as aggressive style, mild style, etc.) with different characteristics in the database, and the cluster center coordinates of each type of driving scene and driving style are obtained. The process and related calculation formulas of the k-means clustering algorithm are as follows: Figure 3 As shown;

[0083] Finally, by evaluating the recognition accuracy and computational real-time performance of a variety of supervised learning recognition algorithms (such as neural network method, distance discrimination method, etc.), a suitable recognition algorithm is selected to construct a nonlinear relationship between feature parameters and driving scenes and driving styles, and to establish a driving scene library and a driving style library. The clustered driving scene library and driving style library will be used to train supervised learning recognition algorithms (such as neural network method, distance discrimination method, etc.) to establish online driving scene identifiers and driving style identifiers. Among them, the library inputs of the driving scene library and the driving style library are driving scene feature parameters and driving style feature parameters, respectively, and the library outputs are the corresponding driving scenes and driving styles, as shown in Table 3;

[0084] Table 3 Library types and library output

[0085]

[0086]

[0087] 2. Driving style and scene recognition power control parameter optimization library based on offline global optimization

[0088] Use offline global optimization algorithms to optimize power control parameters for different driving styles and driving scenarios, so that the energy management strategy can be adjusted in real time according to the current driving characteristics. Developed in Matlab environment using M language, intelligent optimization algorithms are used to globally optimize equivalent factors for typical driving scenarios, typical driving styles, initial SOC, and target SOC, and establish a power control parameter optimization library.

[0089] First, construct typical driving conditions for each driving scenario and driving style type as the target of the global optimization algorithm. In the driving scenario fragment library, connect several scene fragments closest to the cluster center of each type of scene to construct a typical scene of this type of driving scenario. In the driving style fragment library, select the characteristic parameters of the cluster center of each type of driving style as the driver characteristics of the driving condition. Set multiple initial SOCs and target SOCs between 30% and 80% as the start and end SOCs of the intelligent optimization algorithm.

[0090] Subsequently, an intelligent optimization algorithm is used to iteratively optimize typical driving conditions with different driving styles, initial SOCs and target SOCs with the goal of minimizing comprehensive fuel consumption, and the corresponding optimized power control parameters are obtained, and a multi-dimensional power parameter optimization library for different driving styles, driving scenarios, initial SOCs and target SOCs is established.

[0091] 3. Online energy trajectory planning based on predictive road information

[0092] During driving, predictive road information provides energy management with traffic flow information of the road ahead. The intelligent energy management system receives global traffic flow information, predicts the possible changes in driving data when the vehicle enters the road ahead, and identifies and plans the driving scene of the road ahead and the energy information of the entire node, as shown in the figure above. Among them, the road ahead is the road within a certain mileage window ahead. During driving, the road ahead window continues to move forward. The method for identifying the driving scene of the road ahead uses some offline content.

[0093] The intelligent energy management system receives traffic flow information of the road ahead that is about to be entered, predicts the possible changes in driving data when the vehicle enters the road ahead, adjusts the power control parameters in advance, and optimizes energy management. Based on the traffic flow information of the road ahead, the SOC trajectory of the vehicle reaching the road ahead is planned in advance using an online dynamic planning global optimization algorithm. The update frequency of energy trajectory planning is consistent with the update frequency of obtaining predictive road information. When the traffic flow of the road ahead changes, the SOC trajectory is replanned to adapt to changes in traffic scenarios. The required predictive road information is shown in Table 4.

[0094] Table 4 Predictive road information

[0095]

[0096] Based on the map navigation information, the traffic flow information (such as average vehicle speed) of the navigation section ahead in the future can be obtained, which can be used to plan the long-term optimal SOC trajectory in the online dynamic planning. The online dynamic planning takes the average vehicle speed in each stage as the requirement to plan the long-term SOC trajectory. The discrete settings of the online dynamic planning are shown in Table 5;

[0097] Table 5 Discrete settings for online dynamic programming

[0098]

[0099]

[0100] The control variables are:

[0101]

[0102] in, is the average battery power in the kth stage, is the average engine power in the kth stage, is the average engine speed in the kth stage.

[0103] The state transfer equation is:

[0104]

[0105] Among them, E m is the battery open-loop voltage, R int is the internal resistance of the battery, Q c is the total battery capacity, SOC k is the SOC of the kth stage, SOC k+1 is the SOC of the (k+1)th stage.

[0106] The constraints are:

[0107]

[0108] Among them, SOC min and SOC max are the lower and upper limits of SOC, P batt / chg and P batt / dischg are the battery charging power and battery discharging power, T min and T max are the lower and upper limits of engine torque, N min and N max are the lower and upper limits of engine speed respectively. The cost function is

[0109] V L =V eng / L (T eng ,N eng )+V batt / L (P batt )

[0110] Among them, V eng / L and V batt / L are the engine fuel consumption and the battery equivalent fuel consumption, V L For the comprehensive fuel consumption.

[0111] 4. Power control parameter regulation strategy based on deep reinforcement learning

[0112] The deep reinforcement learning (DRL) algorithm is used to train the power control parameter adjustment strategy to obtain the final power control parameters of A-ECMS, as shown in the following formula:

[0113] s final (t) = F(st (t),s pre (t),SOC ref (t),SOC t (t))

[0114] Among them, s final is the final power control parameter of A-ECMS, s t is the power control parameter based on short-term working condition identification, which makes the energy consumption optimal at the current moment; pre The power control parameters corresponding to the working conditions of the road ahead based on predictive road information are used to make the energy consumption have the short-term optimal performance in the foreseeable future; SOC ref Based on the long-term energy trajectory obtained by online dynamic programming, the energy management strategy is provided with global energy consumption optimization guidance; SOC t is the SOC at the current moment, and F is the power control parameter adjustment strategy trained using the deep reinforcement learning algorithm.

[0115] The state space of deep reinforcement learning is as follows:

[0116] x(t)=[s t (t),s pre (t),SOC ref (t),SOC t (t)] T

[0117] For each state combination x in the state space t , the action space output corresponds to action a t Defined as the possible A-ECMS final equivalence factor s final ;

[0118] A=[a 1 ,a 2 ,…,a m ,…,a n-1 ,a n ] T

[0119] For each action of the agent, the environment returns the corresponding reward r t+1 :

[0120] r t+1 =V eng / L (T eng ,N eng )+V batt / L (P batt )

[0121] Among them, V eng / L and V batt / L are the engine fuel consumption and the battery equivalent fuel consumption, VL is the comprehensive fuel consumption, T eng and N eng are engine torque and speed, P batt is the battery power.

[0122] The reinforcement learning environment consists of four parts, namely the hybrid propulsion system model, the A-ECMS energy management strategy model, the energy trajectory prediction model based on predicted road information, and the operating condition identification system model.

[0123] Figure 5 The process of reinforcement learning training and application is described. Using historical driving speed curves, in each learning episode, the environment randomly extracts a speed curve and corresponding predictive road information (extracted from the map API) from the historical data that the agent interacts with. In the agent model, the upper and lower limits of the battery SOC are set to control the energy consumption level of the entire route. When the battery SOC exceeds the upper limit or falls below the lower limit, the environment will execute an early termination event.

[0124] 5. Online A-ECMS control strategy based on predictive road information

[0125] The A-ECMS control strategy based on predictive road information combines offline big data learning, global optimization, power control parameter adjustment strategy based on deep reinforcement learning, and predictive energy trajectory planning with the A-ECMS strategy, so that the optimization effect of the hybrid vehicle energy management strategy breaks through the limitations of instantaneous optimality and approaches the global optimality.

[0126] During driving, the final power control parameter solution combines the optimization results of working condition identification and predictive road information, and uses the power control parameter adjustment strategy based on deep reinforcement learning to output the final power control parameters, thereby improving the adaptability of the energy management system to driving scenarios and making the energy management strategy have a certain degree of road foresight. The overall strategy architecture is shown in the figure below: Figure 6 as shown.

[0127] First, the characteristic parameters are calculated by collecting historical short-term driving data, and the characteristic parameters of the historical driving data are calculated and input into the driving scene identifier and driving style identifier to identify the current driving scene and driving style. Find the optimized power control parameters s corresponding to the current driving scene, driving style, current SOC and target SOC t The collected driving scene data will also be uploaded to the cloud for scene extraction calculations to regularly update (OTA) the typical driving scene and driving style library.

[0128] Subsequently, the traffic flow information of the road section within a certain mileage window ahead is obtained based on the predictive road information, and is input into the driving scene identifier to identify the driving scene of the road section ahead within a certain mileage window, and find the corresponding optimized power control parameter s pre At the same time, online dynamic programming is used to obtain the future long-term energy trajectory SOC ref .

[0129] Finally, the power control parameter adjustment strategy based on the deep reinforcement learning algorithm is used to output the final power control parameter s final To A-ECMS energy control strategy, optimize the power distribution of the hybrid system. The final power control parameter calculation formula is as follows

[0130] s final (t) = F(s t (t),s pre (t),SOC ref (t),SOC t (t))

[0131] The A-ECMS control target calculation model calculates in real time the optimal energy distribution of the hybrid system power architecture among multiple energy sources, maximizing energy utilization efficiency and reducing fuel consumption while meeting driving requirements and power system constraints.

[0132] Based on the DRL (Deep Reinforcement Learning) algorithm, the current optimized power control parameters, short-term predicted optimized power control parameters, long-term energy trajectory and current energy trajectory are used as reinforcement learning state quantities, and the final power control parameters are used as actions. The deep network is trained offline and applied in real time to calculate the final power control parameters. According to the final power control parameters, based on the ECMS control strategy, the optimal control sequence of the vehicle power system is calculated in real time.

[0133] First, the required torque of the vehicle is calculated by analyzing driving intention, accelerator pedal position, vehicle speed, battery status, and motor characteristics. The optimal working point is then found through the ECMS algorithm, which has the characteristics of fast calculation speed and real-time control, and takes the total equivalent fuel consumption as the optimization target. The overall structure is as follows:

[0134] (1) Construct a discretized sequence:

[0135] First, through the current driving state of the vehicle, such as vehicle speed, engine speed, etc., the operating point sequence of all engines can be discretized. The corresponding formula is as follows:

[0136] (T EngVec ,ω EngVec )=f(v,ω Eng )

[0137] Where, T EngVec ,ω EngVec is the torque sequence obtained by the engine discrete and the corresponding speed sequence, which together constitute the engine discrete sequence; v is the current vehicle speed; ω Eng is the current vehicle engine speed;

[0138] Secondly, the discrete sequence points of the motor can be determined by the obtained engine discrete sequence, the driver's required torque (wheel-end required torque) and the current vehicle speed. The corresponding formula is as follows:

[0139] (T MotVec ,ω MotVec )=f(T EngVec ,ω EngVec ,v,T req )

[0140] Where, T MotVec ,ω MotVec is the torque sequence and the corresponding speed sequence obtained by the motor discrete, which together constitute the motor discrete sequence; T req It is the torque required by the driver (wheel end torque required).

[0141] Finally, the possible discrete sequence points of the battery can be calculated through the discrete engine discrete sequence, motor discrete sequence, vehicle speed and driver demand torque (wheel end demand torque). The corresponding formula is as follows:

[0142] P BattVec =f(T EngVec ,ω EngVec ,T MotVec ,ω MotVec ,v,T req )

[0143] Where P BattVec is the discrete sequence point of battery power, i.e., the discrete sequence of the battery;

[0144] The overall schematic diagram is as follows Figure 8 As shown:

[0145] (2) Instantaneous optimization constraints of ECMS:

[0146] The constraints are mainly divided into three parts, namely engine module constraints, motor constraints and battery module constraints. The total torque of the engine meets the external characteristic constraints; the motor torque needs to meet the external characteristic constraints; the battery needs to meet the charge and discharge limit power constraints; the ECMS constraint conditions are shown in the following formula:

[0147]

[0148] In the formula, ω Motis the real-time speed of the motor, rpm; ω Eng is the engine command speed, rpm; T Max is the maximum torque at the corresponding speed, Nm; P BattMaxDch is the maximum discharge power of the battery, W; P BattMaxch The maximum charging power of the battery;

[0149] (3) Constructing the instantaneous optimization objective function of ECMS

[0150] In order to enable the battery pack to be fully discharged throughout the entire cycle, thereby reducing the pure engine fuel consumption, this can be achieved by adjusting the equivalent factor, and its Hamiltonian function is shown in the following formula:

[0151]

[0152]

[0153] In the formula, is the engine fuel consumption rate, kg / s; is the battery equivalent engine fuel consumption rate, kg / s; s final (t) is the instantaneous fuel equivalent factor, Q lhv is the lower calorific value of fuel oil, kj / kg; P batt (t) is the battery power, kw; p(SOC(t)) is the SOC penalty function.

[0154] The SOC penalty function is used to correct the battery SOC and control the SOC in a range with low internal resistance and high efficiency, thus eliminating the biased use of power in the control strategy. The expression of p(SOC(t)) is as follows:

[0155]

[0156] In the formula, SOC max and SOC min are the upper and lower limits of the battery SOC, and e is the penalty function exponent.

[0157] (4) Solving the optimal working point:

[0158] When the Hamiltonian function H reaches its minimum value, the system obtains the optimal solution:

[0159]

[0160] In the formula, is the equivalent fuel consumption, kg; u * (t) is the optimal control sequence at time t;

[0161] The above is only a preferred specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An A-ECMS energy management method based on predictive road traffic information, It is characterized in that include: Acquire historical driving data of the vehicle, and calculate characteristic parameters of the historical driving data; Inputting the characteristic parameters into an offline driving data recognition library to obtain the current optimized power control parameters; wherein the offline driving data recognition library is a multi-dimensional power parameter optimization library including different driving styles, driving scenes, initial SOC and target SOC; Obtaining predictive road data ahead of the vehicle, inputting the predictive road data into the offline driving data recognition library for scene recognition, and obtaining a predicted scene and a corresponding predicted optimized power control parameter; using an online dynamic programming algorithm to calculate a long-term energy trajectory; Based on the DRL algorithm, the final power control parameter is calculated by combining the current optimized power control parameter, the predicted optimized power control parameter, the long-term energy trajectory and the current energy trajectory; According to the final power control parameters, the optimal control sequence of the vehicle power system is calculated in real time.

2. The A-ECMS energy management method based on predictive road traffic information according to claim 1, It is characterized in that The construction process of the offline driving data recognition library includes: Acquiring raw data, the raw data including driving-related data, and performing data cleaning and normalization processing on the raw data to obtain pre-processed raw data; Perform principal component analysis on the preprocessed raw data to obtain dimension-reduced data; Selecting a number of data points from the dimension reduction data as initial cluster centers for cluster analysis to obtain driving scene type data and driving style type data with different characteristic parameters; Based on the supervised learning recognition algorithm, a nonlinear relationship between feature parameters and driving scenes and driving styles is constructed, and a driving scene library and a driving style library are established; the feature parameters include driving scene feature parameters and driving style feature parameters; The library inputs of the driving scene library and the driving style library are driving scene feature parameters and driving style feature parameters respectively, and the outputs are corresponding driving scenes and driving styles; Several initial energy trajectories and target energy trajectories are set, and based on the intelligent optimization algorithm, typical driving conditions with different driving styles, initial energy trajectories and target energy trajectories are iteratively optimized with the goal of minimizing comprehensive fuel consumption, and the corresponding optimized power control parameters are obtained. A multi-dimensional power parameter optimization library with different driving styles, driving scenarios, initial SOC and target SOC is established.

3. The A-ECMS energy management method based on predictive road traffic information according to claim 2, It is characterized in that The driving scene characteristic parameters include maximum positive acceleration, positive acceleration standard deviation, positive acceleration average, acceleration time proportion, maximum negative acceleration, negative acceleration standard deviation, negative acceleration average, negative acceleration time proportion, maximum vehicle speed, vehicle speed standard value, average vehicle speed, average acceleration, acceleration standard deviation, average vehicle speed in the cruising section and cruising time proportion.

4. The A-ECMS energy management method based on predictive road traffic information according to claim 2, It is characterized in that The driving style characteristic parameters include maximum positive acceleration, positive acceleration standard deviation, positive acceleration average, positive jerk standard deviation, positive jerk average, acceleration time proportion, maximum negative acceleration, negative acceleration standard value, negative acceleration average, negative acceleration time proportion, negative jerk standard deviation, negative jerk average, maximum vehicle speed, average vehicle speed, average acceleration, acceleration standard value, average vehicle speed in the cruising section, cruising time proportion, accelerator pedal standard deviation, accelerator pedal change rate standard deviation, brake pedal standard deviation, brake pedal change rate standard deviation, average jerk, and jerk standard deviation.

5. The A-ECMS energy management method based on predictive road traffic information according to claim 1, It is characterized in that Input the characteristic parameters into the offline driving data recognition library to obtain the current optimized power control parameters, specifically including: The characteristic parameters are input into an offline driving data recognition library for scene recognition and style recognition to obtain typical driving scenes and driving styles corresponding to the characteristic parameters. The offline driving data recognition library outputs the corresponding current energy trajectory, target energy trajectory and current optimized power control parameters based on the recognized typical driving scenes and driving styles.

6. The A-ECMS energy management method based on predictive road traffic information according to claim 1, It is characterized in that The predictive road data includes the green wave speed range, the remaining mileage of the entire current driving route, the estimated remaining time for the entire current driving route, the average speed of each section of the entire driving route, the congestion situation of each section of the entire driving route, the distance at the average speed of each section of the entire driving route, the distance of the congested section and the passing time of the congested section.

7. The A-ECMS energy management method based on predictive road traffic information according to claim 1, It is characterized in that An online dynamic programming algorithm is used to calculate the long-term energy trajectory based on the full-course predictive road traffic information. The specific calculation formula is: The constraints are: The cost function is: V L =V eng / L (T eng ,N eng )+V batt / L (P batt ) In the formula, E m is the battery open-loop voltage, R int is the internal resistance of the battery, Q c is the total battery capacity, SOC k is the SOC of the kth stage, SOC k+1 is the SOC of the (k+1)th stage, is the average battery power in the kth stage, is the average engine power in the kth stage, is the average engine speed in the kth stage; SOC min and SOC max are the lower and upper limits of SOC, respectively, and P batt / chg and P batt / dischg are the battery charging power and battery discharging power, T min and T max are the lower and upper limits of engine torque, N min and N max are the lower and upper limits of engine speed, V eng / L and V batt / L are the engine fuel consumption and the battery equivalent fuel consumption, V L For the comprehensive fuel consumption.

8. The A-ECMS energy management method based on predictive road traffic information according to claim 1, It is characterized in that Based on the DRL algorithm, the final power control parameter is calculated by combining the current optimized power control parameter, the predicted optimized power control parameter, the long-term energy trajectory and the current energy trajectory. The specific calculation formula is: s final (t)=F(s t (t),s pre (t),SOC ref (t),SOC t (t)) Among them, s final is the final power control parameter of A-ECMS, s t is the power control parameter based on short-term working condition identification; s pre The power control parameters corresponding to the road conditions ahead based on predictive road information identification; SOC ref is the long-term energy trajectory obtained based on online dynamic programming; SOC t is the SOC at the current moment, and F is the power control parameter adjustment strategy trained using the deep reinforcement learning algorithm.

9. The A-ECMS energy management method based on predictive road traffic information according to claim 1, It is characterized in that According to the final power control parameters, the optimal control sequence of the vehicle power system is calculated in real time, including: Constructing a discrete sequence set, wherein the discrete sequence set includes an engine discrete sequence, a motor discrete sequence, and a battery discrete sequence; Setting instantaneous optimization constraints of the ECMS based on the discrete sequence set; Based on the instantaneous optimization constraint condition and the final power control parameter, an instantaneous optimization objective function is constructed with the goal of minimizing the equivalent fuel consumption of the vehicle power system; The instantaneous optimization objective function is solved to obtain the optimal control sequence of the vehicle power system at the current moment.

Images