Parallel hybrid power energy management control method and system based on reinforcement learning

Abstract

The invention discloses a parallel hybrid power energy management control method and system based on reinforcement learning. The method comprises the steps of S1, data acquisition; s2, motion track prediction and kinetic energy recovery; s3, global power distribution; s4, executing power source control; and S5, a user interaction step. By constructing a set of intelligent closed-loop energy management framework integrating environment perception, trajectory prediction, power optimization and execution coordination, the environment understanding ability of a convolutional neural network, the sequence decision-making ability of reinforcement learning and the traditional vehicle energy management demand are deeply fused; and the comprehensive energy efficiency and the self-adaption level of the hybrid power system in a complex real traffic scene are remarkably improved. According to the scheme, on the basis of high-precision visual perception and personalized track prediction, the deceleration demand and the power change trend are recognized in a prospective mode, the kinetic energy recovery system can be intelligently triggered and accurately adjusted on the premise that safety is guaranteed, and therefore the pertinence and the recovery amount of braking energy recovery are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to a parallel hybrid energy management control method and system based on reinforcement learning, belonging to the technical field of vehicle energy management. BACKGROUND

[0002] As an important transitional technology between traditional internal combustion engine vehicles and pure electric vehicles, hybrid vehicles aim to balance range and energy consumption performance by simultaneously equipping engines and electric motors in vehicles and coordinating their work. The core task of the energy management system, which is the core control system of hybrid vehicles, is to intelligently allocate the output power of the engine and the electric motor according to the real-time driving conditions, and efficiently recover kinetic energy during braking or coasting, thereby comprehensively improving the fuel economy and emission level of the vehicle.

[0003] However, existing hybrid energy management strategies still face several challenges in practical applications. Traditional control methods are mostly based on preset rules or static optimization models, and their switching and torque distribution decisions for power sources often rely on current instantaneous state parameters, lacking forward-looking judgment of the future short-time running trend of the vehicle. This makes it difficult for the system to make optimal energy allocation and recovery decisions in advance in complex and variable actual traffic environments, especially when frequent acceleration, deceleration, and obstacles or curves are encountered. As a result, the engine and the electric motor may not be able to run continuously in their respective high-efficiency intervals, and the timing and intensity of kinetic energy recovery often do not match the actual deceleration demand, thereby restricting the further improvement of the vehicle's energy efficiency. In addition, the individualized operation habits of the driver also have a significant impact on the energy management effect, and traditional systems usually have difficulty in adaptive learning and matching. SUMMARY

[0004] The present application provides a parallel hybrid energy management control method and system based on reinforcement learning, which significantly improves the comprehensive energy efficiency and adaptive level of the hybrid system in complex real traffic scenarios by deeply integrating the environmental understanding ability of convolutional neural networks, the sequence decision-making ability of reinforcement learning, and the traditional vehicle energy management requirements.

[0005] Technical solution: The parallel hybrid energy management control method based on reinforcement learning comprises the following steps:

[0006] S1, data acquisition: real-time acquisition of vehicle running data, the vehicle running data including visual image information, vehicle yaw angle, gear signal, brake signal, obstacle distance, vehicle speed, accelerator pedal pressure, battery temperature and road surface condition information;

[0007] S2, motion trajectory prediction and kinetic energy recovery: based on the data collected in S1, the road conditions and obstacles are identified through a convolutional neural network, and the driver's driving habits are combined to predict the vehicle's motion trajectory in the next 2 seconds using a reinforcement learning algorithm; according to the motion trajectory, it is judged whether the vehicle needs to slow down, if the judgment result is to slow down, kinetic energy recovery is performed, and the kinetic energy recovery intensity is dynamically adjusted according to the real-time brake information;

[0008] S3, global power distribution: based on the data collected in S1 and the motion trajectory prediction result of S2, the optimal efficiency point under the current working condition is calculated, and according to the optimal efficiency point, the preset sub-unit algorithm and optimization algorithm are iterated to generate the globally optimal power distribution scheme, that is, the target output power of the engine and the motor is obtained; the motion trajectory prediction result is used as an input parameter for power distribution to predict future power demand changes;

[0009] S4, power source control execution: according to the globally optimal power distribution scheme obtained in S3, the corresponding power source combination mode is selected and executed to control the engine and the motor to output power and switch;

[0010] S5, user interaction step: the user system provides vehicle information and control interface for the driver.

[0011] The application constructs a closed-loop control framework integrating motion trajectory prediction, global power distribution and kinetic energy recovery decision-making, uses a reinforcement learning algorithm to make forward-looking predictions of future driving needs, and optimizes the power distribution of the engine and the motor and the brake energy recovery strategy based on this, solves the technical problems of rigid power source switching, energy efficiency optimization lag and disconnection between kinetic energy recovery and real deceleration demand caused by lack of foresight in traditional methods, and maximizes energy utilization efficiency and optimizes driving experience under all working conditions.

[0012] Optionally, in order to accurately and robustly extract structured information essential to energy management decision-making from complex visual environments, a dual-task (semantic segmentation and object detection) parallel processing architecture based on a pre-trained CNN backbone network is used to extract and fuse deep features from continuous image frames, accurately and in real time identify drivable areas, lane lines and the positions and categories of various obstacles, and provide high-quality environmental perception input for subsequent motion trajectory prediction. The road conditions and obstacles are identified through a convolutional neural network, which is:

[0013] Continuous frame images are acquired, and original images are standardized, including color space normalization, image size unification, data enhancement, and time synchronization. Then, a pre-trained CNN model is used as a backbone network to extract spatial semantic features layer by layer and output high-dimensional feature maps. The high-dimensional feature maps are used for road and lane line segmentation and obstacle detection.

[0014] According to the high-dimensional feature maps, a semantic segmentation head is used to output the class labels of each pixel, including road surface, lane line, shoulder, and sidewalk. The output result is a segmentation mask map, which represents the area of the drivable path.

[0015] A target detection head is used to output the bounding box and class, and the detected objects include vehicles, pedestrians, cones, and guardrails. The output result is a set of positions (x t , y t ), classes, and confidence levels of vehicles in the local lane coordinate system.

[0016] Then, redundant detection boxes are removed by non-maximum suppression, the segmentation mask and detection coordinates are restored to the original image scale, and morphological optimization and geometric fitting post-processing are performed on the results.

[0017] Finally, the structured drivable area, lane line contour, and obstacle information are output.

[0018] To make the motion trajectory prediction adapt to the personalized operation style of different drivers and improve the accuracy of prediction and the acceptance of decision-making, an inverse reinforcement learning is introduced to infer the implicit driving preference reward function from the historical trajectory of the driver, and a proximal policy optimization (PPO) algorithm is driven based on the personalized reward function for rolling optimization calculation, realizing personalized and high-precision prediction of the future short-time motion trajectory, and making the energy management strategy more humanized and adaptive. Combined with the driving habits of the driver, a reinforcement learning algorithm is used to predict the motion trajectory of the vehicle within the next 2 seconds, which is specifically:

[0019] The historical driving trajectory of the driver is used as an expert driving trajectory to solve the implicit reward function using inverse reinforcement learning, which is specifically:

[0020] The real historical trajectory of a single driver is used as an expert trajectory set D, and the trajectory data includes global coordinates, longitudinal speed, heading angle in global coordinates, instantaneous curvature of the driving path, and longitudinal acceleration. Using maximum entropy inverse reinforcement learning, the optimized reward function of the driver is output. ;

[0021] Confirm the current driver, load the corresponding reward function , and then input the state sequence S t-k：t, the reward function is calculated based on the proximal policy optimization (PPO) algorithm, and a predicted motion trajectory in the future 2 seconds is generated through rolling execution.

[0022] Preferably, in order to ensure that the start of kinetic energy recovery is safe and efficient, avoid invalid recovery or affect driving safety, a composite start threshold of vehicle speed and battery state of charge SOC is set, and based on the predicted motion trajectory, a deceleration demand judgment is made in combination with the double criteria of future time to collision TTC and minimum safety distance d safe , so that the kinetic energy recovery system is intelligently triggered only when the vehicle is in need of deceleration and meets the basic recovery conditions, thereby maximizing energy recovery efficiency under the premise of safety. The kinetic energy recovery judgment condition in S2 is specifically:

[0023] whether the current vehicle speed is greater than 3 km / h and the vehicle SOC is less than 90%, if yes, whether the vehicle needs to decelerate according to the motion trajectory, if yes, the kinetic energy recovery is started;

[0024] The process of judging whether the vehicle needs to decelerate according to the motion trajectory is:

[0025] According to the motion trajectory, the following checks are performed on each dynamic or static obstacle:

[0026] Calculate the minimum Euclidean distance d min between the trajectory of the ego vehicle and the trajectory of the obstacle within the future time window;

[0027] If d min < d safe , there is a risk;

[0028] ;

[0029] ;

[0030] Time to collision TTC check:

[0031] TTC=d / Δv(when Δv>0), Δv=v ego -v obj ;

[0032] Where, v ego : ego vehicle speed; v obj : speed of the front obstacle; d: distance between the vehicle and the obstacle; d safe : safety distance; d0: static buffer distance; t headway : safe following time; Δv: relative speed;

[0033] When TTC < 3 seconds and Δv > 0, or d min < d safeWhen this happens, a deceleration request is triggered.

[0034] The preferred option, in order to accurately find the power allocation point with the optimal global energy consumption in hybrid mode and overcome the problem of local optima, is to comprehensively consider the current vehicle status and predicted information, query the efficiency map, and systematically traverse the torque distribution coefficients between the engine and the electric motor. Optimization calculations are performed with the overall system efficiency as the optimization objective, achieving refined and globally optimal matching of the target output power of the engine and the electric motor, ensuring that the two power sources always work collaboratively in the high-efficiency range. Specifically, S3 is as follows:

[0035] Enter the current vehicle speed V ego Wheel-side torque requirement T w Based on trajectory prediction information, query the efficiency map of the motor or engine to obtain the driving efficiency in hybrid mode; calculate the speed of the motor and engine respectively according to the optimal operating point of comprehensive energy consumption, and calculate the comprehensive efficiency of the system by combining the efficiency maps of the motor and engine. At the same time, iterate through different torque distribution coefficients λ to find the optimal combination that makes the system most efficient; then record the torque distribution scheme corresponding to the maximum efficiency as the optimal strategy, and finally output the target output power of the motor and engine based on this optimal strategy.

[0036] In a preferred embodiment, to transform the globally optimal power allocation scheme into specific, reliable power source action commands adaptable to different operating conditions, a multi-level state judgment and switching logic based on vehicle speed, SOC, and power demand is designed. Differentiated power source selection and coupling rules are specified for typical operating conditions such as low-speed start-up, medium-to-high-speed cruising, and high load. This achieves precise and smooth matching between the power source mode and the vehicle's real-time needs, ensuring a balance between power performance, economy, and driving smoothness. Specifically, S4 includes:

[0037] S401. Check the vehicle's movement status. If the vehicle is starting or moving at low speed, proceed to S402; if the vehicle is not starting, proceed to S403.

[0038] S402. Determine whether the current state of charge (SOC) of the vehicle's power battery is greater than 20%. If so, the power required for the vehicle to drive is relatively small, and the electric motor provides power alone. Otherwise, the vehicle's power battery is not charged enough. In order to ensure vehicle performance and charge the battery, the electric motor is turned off and the engine is switched to drive. At the same time, the electric motor charges the power battery in reverse.

[0039] S403. Record the vehicle's current speed V, select the optimal efficiency-power point based on the MAP chart to obtain the optimal operating point, and determine the vehicle speed threshold V within the pre-calibrated engine high-efficiency range. th Determine whether the current vehicle speed V is greater than V. thwhether to enter the high efficiency interval of the engine, if yes, enter S404, if not, enter S405;

[0040] S404, at this time the vehicle is running in the high efficiency interval of the engine, judge whether the current SOC of the vehicle is greater than 90%, if yes, the battery power is sufficient, and the engine is driven alone, and the battery is not charged; if not, the battery needs to be charged, and the engine is driven alone, and the motor is reversed to charge the power battery;

[0041] determine whether the vehicle enters a heavy load working condition, if yes, the engine remains in the high efficiency interval, and the remaining power is provided by the motor, if not, the current driving mode is maintained;

[0042] S405, determine whether the power provided by the motor meets the power demand of the vehicle, and whether the current SOC of the vehicle is greater than 20%, if yes, keep the motor as the power source, if not, it indicates that the power demand of the vehicle increases, and the engine is started as auxiliary power to drive the vehicle together with the motor.

[0043] Optionally, in order to solve the timing or power conflict that may occur between the driving energy flow and the recovered energy flow, realize unified and coordinated management of the whole vehicle energy flow, by introducing the kinetic energy recovery decision as a parallel input in the power source control execution process, and establishing the priority execution principle, the deep cooperation between the braking energy recovery and the driving / charging process is realized, and the power balance and safe operation of the whole vehicle system are maintained while ensuring that the recovered energy can be effectively stored. It also includes:

[0044] S406, kinetic energy recovery coordination control: during the execution of the power source control process of S402 to S405, if the decision of starting kinetic energy recovery is received from S2 at the same time, the kinetic energy recovery is preferentially executed under the premise that the safe charging power limit of the power battery is not violated and the power source does not exit its high efficiency working interval; wherein the electric energy generated by the kinetic energy recovery is used to charge the power battery, and the charging process is cooperatively managed with the working state of the power source.

[0045] The system for implementing the parallel hybrid energy management control method based on reinforcement learning includes a sensor system, a motion trajectory prediction module, a kinetic energy recovery decision module, a global power distribution module, a power source control execution module, and a parallel power system.

[0046] The sensor system is used to collect vehicle operation data, and its output end is respectively connected with the input ends of the motion trajectory prediction module and the global power distribution module in communication, for providing real-time data required for executing S1, S2 and S3 steps.

[0047] The input end of the motion trajectory prediction module is in communication connection with the sensor system, and the motion trajectory prediction module processes information through a convolutional neural network and a reinforcement learning algorithm based on sensor data to perform the function of predicting the motion trajectory of the vehicle within the next 2 seconds in S2;

[0048] The input end of the kinetic energy recovery decision module is in communication connection with the output end of the motion trajectory prediction module, and the kinetic energy recovery decision module determines whether deceleration is needed and decides the recovery strength according to the predicted motion trajectory to perform the decision function of kinetic energy recovery in S2;

[0049] The input end of the global power distribution module is in communication connection with the sensor system and the output end of the motion trajectory prediction module, and the global power distribution module performs iterative optimization through a distribution unit algorithm and an optimization algorithm based on the real-time state and predicted information of the vehicle to perform the function of generating a globally optimal power distribution scheme in S3;

[0050] The input end of the power source control execution module is in communication connection with the output end of the global power distribution module and the kinetic energy recovery decision module, and the power source control execution module generates a coordinated control instruction according to the global power distribution scheme and the kinetic energy recovery decision to perform the function of selecting a power source combination mode and coordinating power output and kinetic energy recovery in S4.

[0051] The control end of the parallel power system is electrically connected to the output end of the power source control execution module to receive and execute the driving and recovery instructions from the module; the parallel power system includes an engine, an electric motor, a power battery, and a transmission mechanism, wherein the torques of the engine and the electric motor are coupled at the gear ring of the transmission mechanism.

[0052] In order to reliably implement the foregoing intelligent control method on a physical level, a complete closed-loop control system entity is constructed by hardware integration and software deployment of the sensor system, the motion trajectory prediction module, the kinetic energy recovery decision module, the global power distribution module, the power source control execution module, and the parallel power system according to the functions and connection relationships, and automatic and integrated operation is realized from environmental information perception, intelligent algorithm decision to power unit execution, thereby providing solid hardware support and system guarantee for the application of the method.

[0053] Beneficial effects: the application constructs a set of intelligent closed-loop energy management framework integrating environment perception, trajectory prediction, power optimization and execution coordination, deeply fuses the environment understanding ability of convolutional neural network, the sequence decision ability of reinforcement learning and the traditional vehicle energy management demand, and significantly improves the comprehensive energy efficiency and self-adaptive level of the hybrid power system in complex real traffic scenarios. The scheme is based on high-precision visual perception and personalized trajectory prediction, and prospectively identifies deceleration demand and power change trend, so that the kinetic energy recovery system can be intelligently triggered and accurately adjusted under the premise of ensuring safety, thereby improving the pertinence and recovery amount of brake energy recovery; at the same time, the global power optimization algorithm integrated with the predictive information can overcome the limitations of traditional methods such as lag and local optimum, realize fine and globally optimal matching of engine and motor output power, and effectively improve the cooperative working efficiency of dual power sources. In addition, by designing layered and coordinated power source switching and energy flow management logic, the system can respond to user driving demand while ensuring smooth power output, natural mode switching and safe unification of driving and recovery process, and can maximize energy utilization rate under all working conditions, and fully meet the comprehensive requirements of economy, environmental protection and driving comfort of the vehicle. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only a part of the embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of the provided drawings.

[0055] Figure 1 The method flowchart of the present application is as follows:

[0056] Figure 2 The power source control execution flowchart of the present application is as follows. DETAILED DESCRIPTION

[0057] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0058] In the description of the present application, it should be understood that the terms "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application.

[0059] In the present application, unless otherwise explicitly specified and limited, "on" or "under" of a first feature to a second feature can include that the first and second features are in direct contact, or that the first and second features are not in direct contact but are in contact through another feature between them. Moreover, "on", "above" and "above" of the first feature to the second feature includes that the first feature is directly above and obliquely above the second feature, or only indicates that the first feature is higher in horizontal height than the second feature. The first feature "below", "below" and "below" of the second feature includes that the first feature is directly below and obliquely below the second feature, or only indicates that the first feature is less than the second feature in horizontal height.

[0060] As shown in Figure 1 The parallel hybrid energy management control method based on reinforcement learning comprises the following steps:

[0061] S1, data acquisition: real-time acquisition of vehicle operation data, the vehicle operation data comprising visual image information, vehicle yaw angle, gear signal, brake signal, obstacle distance, vehicle speed, accelerator pedal pressure, battery temperature and road surface condition information;

[0062] S2, motion trajectory prediction and kinetic energy recovery: based on the data collected in S1, the road condition and obstacle are identified through a convolutional neural network, and the driving habit of the driver is combined to predict the motion trajectory of the vehicle within the next 2 seconds by using a reinforcement learning algorithm; according to the motion trajectory, it is judged whether the vehicle needs to slow down, if the judgment result is that it needs to slow down, kinetic energy recovery is carried out, and the kinetic energy recovery strength is dynamically adjusted according to the real-time brake information;

[0063] The road condition and obstacle identification through the convolutional neural network is specifically:

[0064] Continuous frame images are acquired, and the original images are standardized, including color space normalization, image size unification, data enhancement, and time synchronization; then a pre-trained CNN model is used as a backbone network to extract spatial semantic features layer by layer to output high-dimensional feature maps, which are used for road and lane line segmentation and obstacle detection at the same time;

[0065] Based on the high-dimensional feature map, the semantic segmentation head outputs the category label for each pixel. The categories include: road surface, lane line, shoulder, and sidewalk. The output is a segmentation mask map, which represents the area where the drivable path is located.

[0066] Simultaneously, the object detection head outputs bounding boxes and categories. Detected objects include: vehicles, pedestrians, traffic cones, and guardrails. The output result is a set of vehicle positions in the local lane coordinate system (x, y, y). t y t ), category and confidence level;

[0067] Next, redundant detection boxes are removed by non-maximum suppression, the segmentation mask and detection coordinates are restored to the original image scale, and the results are post-processed by morphological optimization and geometric fitting.

[0068] The final output includes structured information on drivable areas, lane lines, and obstacles.

[0069] Based on the driver's driving habits, a reinforcement learning algorithm is used to predict the vehicle's trajectory within the next 2 seconds.

[0070] Inverse reinforcement learning is used to solve for the implicit reward function of the driver's historical driving trajectory as the expert driving trajectory, specifically:

[0071] The real historical trajectory of a single driver is used as the expert trajectory set D. The trajectory data includes global coordinates, longitudinal velocity, heading angle in global coordinates, instantaneous curvature of the driving path, and longitudinal acceleration. Maximum entropy inverse reinforcement learning is used to output the optimized reward function for that driver. ;

[0072] Identify the current driver and load their corresponding reward function. Then input the vehicle's state sequence S at the current moment. t-k：t Based on this reward function, the PPO algorithm is optimized using a near-end strategy for calculation, and a predicted motion trajectory for the next 2 seconds is generated through rolling execution. Specifically:

[0073] The first phase involves offline training to build a personalized predictive agent:

[0074] Specifically, the driver's historical driving trajectory data is used as an expert trajectory set D, and each trajectory is divided into a state sequence τ=(s0, s1, ..., s2) with a fixed duration T. T ), wherein the time length T is not less than the duration of the predicted motion trajectory, s t =(x t y t v t , ψ t k t ayt , a xt ) contains global coordinates, longitudinal velocity, heading angle in global coordinates, instantaneous curvature of the travel path, and longitudinal acceleration. Global coordinates are converted to Frenet coordinates (s, d) with respect to the lane centerline, where s is the longitudinal displacement and d is the lateral offset. The velocity v t and the heading angle deviation Δψ t are normalized:

[0075]

[0076] where, is the normalized velocity; is the arithmetic mean of the velocity in the trajectory set data; is the standard deviation of the velocity in the trajectory set data; is the angle between the vehicle's heading direction and the x-axis of the global coordinate system; is the expected heading angle at the lane centerline corresponding to the current position; is the arithmetic mean of the heading angle in the trajectory set data; is the standard deviation of the heading angle in the trajectory set data.

[0077] The personalized reward function is solved by maximum entropy IRL,

[0078] The reward function is parameterized:

[0079] The reward function is a linear combination of features:

[0080] where, is the personalized weight vector to be learned; is the state feature vector extracted from the state.

[0081] The maximum entropy IRL is used to find the weight w ∗ that maximizes the probability of the expert trajectory distribution, whose log-likelihood gradient is:

[0082] where the iterative optimization process of the maximum entropy IRL is completed through the iteration of the outer loop and the inner loop:

[0083] Inner loop: policy optimization

[0084] Fix the reward function weight w, substitute it into to determine the current environmental feedback standard. Subsequently, in the simulation environment, a policy π θ (a|s) is trained using a proximal policy optimization algorithm, so that it learns to maximize the cumulative return under the current reward function R w (s) in the interaction with the environment. This loop is performed independently until the policy πθ converge or reach a preset number of training steps.

[0085] outer loop: reward function modification

[0086] running policy π θ to collect trajectory samples generated by it and calculate the feature expectation of these trajectories . Compare it with the expert trajectory feature expectation , calculate the loss function gradient of maximum entropy IRL . According to this gradient, update the reward function weight using gradient ascent method.

[0087] Repeat the inner loop and the outer loop until the change of weight w is less than the set threshold, indicating that the reward function can fully explain the expert behavior. At this time, output the final personalized reward function .

[0088] The fixed reward function is , and the final policy network π θ* (a∣s) is trained in the simulation environment using the proximal policy optimization algorithm. Update the policy network parameters θ by optimizing the loss function. The PPO loss function is as follows:

[0089]

[0090] where: is the advantage function calculated by generalized advantage estimation; is the value function loss; is the policy entropy; c1 is the value loss coefficient; c2 is the entropy regularization coefficient; is the new policy probability; is the old policy probability; is the clipping range coefficient; is the experience expectation; (r, ) is a clipping function that limits the probability ratio r to ;

[0091] After training, we get , which together with constitutes a personalized driving trajectory prediction agent.

[0092] The second stage is online application, which uses the interaction between the agent and the environment to obtain the predicted trajectory

[0093] Specifically, the driver identification module is activated, receiving driver identification information from onboard sensors or the user interface. Based on this driver identification information, the agent invokes the personalized predictive agent reward function and policy network corresponding to that driver. Status information is updated in real-time via the vehicle bus and sensors, executing the following loop at a fixed frequency:

[0094] Obtain the vehicle's original state observations at the current time t, including: global coordinates, velocity, acceleration, heading angle, and instantaneous curvature of the driving path.

[0095] The standard state vector S required by the agent is transformed from the original observations. t .

[0096] New state S t Push the historical state into the buffer and remove the oldest state, always keeping the buffer with the k most recent states, to form the current state sequence S. t−k+1：t .

[0097] Rolling optimization loop (in the prediction time domain i=0 to 19) Inner iteration):

[0098] For each planning step i:

[0099] Clear the output sequence of the predicted trajectory for this round. .

[0100] Set the current state S t−k+1:t Input Policy Network π θ* (a|s) yields the action distribution. The desired action is then selected from this distribution. The next state is calculated by parameterizing the vehicle dynamics model with the action input.

[0101]

[0102] in, : Vehicle longitudinal acceleration command, positive value indicates acceleration, negative value indicates deceleration. : The vehicle's front wheel steering angle command; a positive value indicates a left turn, and a negative value indicates a right turn. The longitudinal velocity at the current step size; For the next long longitudinal velocity; The heading angle for the current step size; For the next longer heading angle; The x-coordinate of the current step size; Let x be the x-coordinate of the next step length; The ordinate of the current step size; The vertical axis represents the length of the next step; L represents the wheelbase of the vehicle. To control the cycle length, take 100 ms.

[0103] After deriving the next predicted state, calculate its personalized reward value , select the candidate trajectory with the highest personalized reward value as the current optimal prediction output.

[0104] Update the planning state to the predicted state, i.e. , add the predicted state to the current prediction trajectory sequence.

[0105] After completing 20-step rolling optimization, output the complete predicted state sequence:

[0106]

[0107] wherein, is the first prediction step trajectory state; is the second prediction step trajectory state; is the 20th prediction step trajectory state.

[0108] After completing one cycle of trajectory prediction and output, the system obtains the new actual vehicle state , updates the historical state sequence S t-k+2：t+1 , and executes the rolling optimization cycle with the current state s t+1 as the starting point.

[0109] The kinetic energy recovery judgment condition in S2 is specifically:

[0110] Determine whether the current vehicle speed is greater than 3 km / h and the vehicle SOC is less than 90%. If yes, determine whether the vehicle needs to decelerate according to the motion trajectory. If yes, start kinetic energy recovery;

[0111] When the electric vehicle speed is too low, the vehicle will produce a strong jerk feeling. Considering that the average vehicle speed is about 3 km in the reversing and parking working conditions, 3 km is selected as the kinetic energy recovery intervention speed threshold. For example, the Ningde Times Technology White Paper points out that the power battery SOC window is between 20%-80%, and the battery cycle life is significantly improved. The battery SOC is lower than 20% for a long time, which will accelerate the battery degradation, and the cycle number will be significantly reduced. Therefore, the lower limit of the battery SOC window is maintained above 20%, and considering that the electric vehicle range will decrease due to temperature in winter, the upper limit of the SOC window is increased to 90% to avoid the problem of range decrease due to low temperature.

[0112] The process of determining whether the vehicle needs to decelerate according to the motion trajectory is:

[0113] According to the motion trajectory, the following checks are performed on each dynamic or static obstacle:

[0114] Calculate the minimum Euclidean distance d between the ego trajectory and the obstacle trajectory within the future time window min ;

[0115] If d min < d safe , there is a risk;

[0116] ;

[0117] ;

[0118] According to the Road Traffic Safety Law, when a motor vehicle travels on a highway at a speed exceeding 100 kilometers per hour, it shall maintain a distance of more than 100 meters from the vehicle in front of the same lane, and when the speed is less than 100 kilometers per hour, the distance from the vehicle in front of the same lane can be appropriately shortened, but the minimum distance shall not be less than 50 meters. Considering the driver's reaction time and braking performance, while taking into account driving safety and driving smoothness, avoid opening energy recovery as soon as the minimum following distance is reached, which will affect the driving experience. Therefore, take the following distance between aggressive and conservative, for example, when the vehicle travels at 80 km / h, the minimum following distance is 50 meters, when t headway =1.8, d0=4 , when t headway =1.5, d0=2 , the following distance for triggering kinetic energy recovery is 35m~44m, which realizes neither affects the driving smoothness, nor takes into account the principle of safe driving.

[0119] Collision time TTC check:

[0120] TTC=d / Δv (when Δv>0), Δv=v ego -v obj ;

[0121] Where, v ego : ego speed; v obj : speed of the front obstacle; d: distance between the vehicle and the obstacle; d safe : safety distance; d0: static buffer distance; t headway : safe following time; Δv: relative speed;

[0122] When TTC < 3 seconds and Δv > 0, or d min < d safe , the deceleration request is triggered.

[0123] S3, global power distribution: based on the data collected in S1 and the motion trajectory prediction result in S2, the optimal efficiency point under the current working condition is calculated, and according to the optimal efficiency point, the preset sub-unit algorithm and optimization algorithm are iterated and optimized to generate a globally optimal power distribution scheme, that is, the target output power of the engine and the motor is obtained; the motion trajectory prediction result is used as an input parameter of power distribution to predict future power demand changes;

[0124] Input the current vehicle speed V ego , the wheel side demand torque T w and the trajectory prediction information, query the efficiency Map of the motor or engine to obtain the driving efficiency in the hybrid mode; according to the optimal working point of the comprehensive energy consumption, the speed of the motor and the engine is calculated respectively, and the system comprehensive efficiency is calculated combined with the efficiency Map of the motor and the engine, while different torque distribution coefficients λ are traversed to find the optimal combination that makes the system efficiency highest; then the torque distribution scheme corresponding to the maximum efficiency is recorded as the optimal strategy, and finally the target output power of the motor and the engine is output based on the optimal strategy, specifically:

[0125] According to the current vehicle speed and the transmission coefficient, the speed of the engine and the motor is calculated:

[0126]

[0127] The torque distribution coefficient λ is discretized into a series of discrete points with a certain step size, and then the enumeration method is used to calculate the distribution torque of the engine under different torque distribution coefficients:

[0128]

[0129] Then the power P of the engine is calculated eng :

[0130]

[0131] The instantaneous actual fuel consumption rate of the engine is:

[0132]

[0133] The instantaneous equivalent fuel consumption rate of the battery is:

[0134]

[0135]

[0136] The system comprehensive energy consumption is:

[0137]

[0138] The system comprehensive efficiency is:​

[0139]

[0140] Compare the overall system efficiency under different allocation coefficients ,Will The torque distribution scheme at maximum is recorded as the optimal strategy.

[0141] in, Equivalent fuel coefficient for discharge; The effective electrical energy of a battery discharge; Fuel has a low calorific value; Battery discharge power; : Equivalent combustion coefficient of battery discharge; The transmission coefficient of the engine; : The transmission coefficient of the motor; The instantaneous efficiency of the engine can be obtained from the engine universal characteristic curve; Motor efficiency.

[0142] like Figure 2 As shown, S4, power source control execution: Based on the globally optimal power allocation scheme obtained in S3, select and execute the corresponding power source combination mode, and control the engine and electric motor to output and switch power; at the same time, based on the kinetic energy recovery decision result in S2, coordinate and control the working state of the kinetic energy recovery system and the power source under the premise of satisfying the power allocation scheme.

[0143] S401. Check the vehicle's movement status. If the vehicle is starting or moving at low speed, proceed to S402; if the vehicle is not starting, proceed to S403.

[0144] S402. Determine whether the current state of charge (SOC) of the vehicle's power battery is greater than 20%. If so, the power required for vehicle operation is relatively small, and the electric motor provides power alone. Otherwise, the vehicle's power battery is insufficient. In order to ensure vehicle performance and charge the battery, the electric motor is turned off and switched to engine drive. At the same time, the electric motor charges the power battery in reverse.

[0145] S403. Record the vehicle's current speed V, select the optimal efficiency-power point based on the MAP chart to obtain the optimal operating point, and determine the vehicle speed threshold V within the pre-calibrated engine high-efficiency range. th Determine whether the current vehicle speed V is greater than V. th This refers to whether the engine has entered its high-efficiency range. If so, it enters S404; otherwise, it enters S405.

[0146] S404, at this time the vehicle is running in the engine high efficiency interval, determine whether the current SOC of the vehicle is greater than 90%, if so, the battery power is sufficient, driven by the engine alone, the battery is not charged; if not, the battery needs to be charged, driven by the engine alone, while the motor reversely charges the power battery;

[0147] Determine whether the vehicle enters the heavy load working condition, if so, the engine remains in the high efficiency interval, the remaining power is provided by the motor, if not, maintain the current driving mode;

[0148] S405, determine whether the power provided by the motor meets the power demand of the vehicle, and the current SOC of the vehicle is greater than 20%, if so, keep the motor as the power source, if not, it means that the power demand of the vehicle increases, start the engine as auxiliary power, and drive the vehicle together with the motor.

[0149] S406, kinetic energy recovery coordination control: in the process of executing S402 to S405, if the decision to start kinetic energy recovery is received from S2 at the same time, the kinetic energy recovery is preferentially executed under the premise that the safe charging power limit of the power battery is not violated and the power source does not exit its high efficiency working interval, that is:

[0150] Firstly, the system obtains the current battery temperature, SOC and maximum safe charging power P max from the battery management system BMS in real time. plan At the same time, the expected charging power P regen of the engine and the motor under the current power distribution scheme is calculated (such as the part of the engine surplus power used for charging).

[0151] Then, the system evaluates whether P plan + P regen ≤ P max is met if the kinetic energy recovery power P regen is superimposed.

[0152] If the above safety conditions and efficiency conditions are met, the system preferentially executes the kinetic energy recovery instruction. If any condition is not met, the system starts coordination: for example, limit the amplitude of P regen according to the preset strategy (such as ensuring braking safety), or fine-tune the power distribution scheme to realize the coordinated management of kinetic energy recovery and driving / charging under the premise of meeting all constraints.

[0153] Among them, the electric energy generated by the kinetic energy recovery is used to charge the power battery, and the charging process is cooperatively managed with the working state of the power source.

[0154] S5, user interaction step: providing vehicle information and control interface for the driver through the user system.

[0155] The system for implementing the parallel hybrid energy management control method based on reinforcement learning comprises a sensor system, a motion trajectory prediction module, a kinetic energy recovery decision module, a global power distribution module, a power source control execution module, and a parallel power system.

[0156] The sensor system is used to collect vehicle operation data, and its output end is in communication connection with the input ends of the motion trajectory prediction module and the global power distribution module, for providing real-time data required for executing the steps S1, S2 and S3.

[0157] The input end of the motion trajectory prediction module is in communication connection with the sensor system, which processes information through a convolutional neural network and a reinforcement learning algorithm based on sensor data, to execute the function of predicting the motion trajectory of the vehicle within the next 2 seconds in the step S2.

[0158] The input end of the kinetic energy recovery decision module is in communication connection with the output end of the motion trajectory prediction module, which judges whether deceleration is needed and decides the recovery intensity according to the predicted motion trajectory, to execute the decision function about kinetic energy recovery in the step S2.

[0159] The input end of the global power distribution module is in communication connection with the sensor system and the output end of the motion trajectory prediction module, which iteratively optimizes through a sub-unit algorithm and an optimization algorithm based on the real-time state and predicted information of the vehicle, to execute the function of generating a globally optimal power distribution scheme in the step S3.

[0160] The input end of the power source control execution module is in communication connection with the output ends of the global power distribution module and the kinetic energy recovery decision module, which generates coordinated control instructions according to the global power distribution scheme and the kinetic energy recovery decision, to execute the function of selecting a power source combination mode and coordinating the power output and kinetic energy recovery in the step S4.

[0161] The control end of the parallel power system is electrically connected with the output end of the power source control execution module, to receive and execute the driving and recovery instructions from the module; the parallel power system comprises an engine, an electric motor, a power battery and a transmission mechanism, wherein the torques of the engine and the electric motor are coupled at the gear ring of the transmission mechanism.

[0162] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part.

[0163] The foregoing description of the disclosed embodiments enables a person skilled in the art to make or use the application. Numerous modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other embodiments without the use of the inventive faculty. Therefore, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A parallel hybrid power energy management and control method based on reinforcement learning, characterized in that, Includes the following steps: S1. Data Acquisition: Real-time acquisition of vehicle operation data, including visual image information, vehicle yaw angle, gear signal, brake signal, obstacle distance, vehicle speed, accelerator pedal pressure, battery temperature and road condition information; S2. Motion trajectory prediction and kinetic energy recovery: Based on the data collected in S1, road conditions and obstacles are identified through a convolutional neural network, and combined with the driver's driving habits, a reinforcement learning algorithm is used to predict the vehicle's motion trajectory within the next 2 seconds; based on the motion trajectory, it is determined whether the vehicle needs to decelerate. If the determination result is that deceleration is required, kinetic energy recovery is performed, and the intensity of kinetic energy recovery is dynamically adjusted according to real-time braking information. S3, Global Power Allocation: Based on the data collected in S1 and the motion trajectory prediction results in S2, the optimal efficiency point under the current operating condition is calculated. According to the optimal efficiency point, the optimal power allocation scheme is generated through iterative optimization using a preset unit algorithm and optimization algorithm, that is, the target output power of the engine and the electric motor are obtained respectively. The motion trajectory prediction results are used as input parameters for power allocation to predict future changes in power demand. S4, Power Source Control Execution: Based on the globally optimal power allocation scheme obtained in S3, select and execute the corresponding power source combination mode to control the engine and electric motor to output and switch power; S5. User interaction steps: Provide the driver with vehicle information and control interface through the user system.

2. The parallel hybrid power energy management and control method based on reinforcement learning according to claim 1, characterized in that: The specific steps for recognizing road conditions and obstacles using convolutional neural networks are as follows: The system acquires consecutive frame images and performs standardization processing on the original images, including color space normalization, image size unification, data augmentation, and time synchronization. Then, a pre-trained CNN model is used as the backbone network to extract spatial semantic features layer by layer and output high-dimensional feature maps. The high-dimensional feature maps are used for road and lane line segmentation and obstacle detection. Based on the high-dimensional feature map, the semantic segmentation head outputs the category label for each pixel. The categories include: road surface, lane line, shoulder, and sidewalk. The output is a segmentation mask map, which represents the area where the drivable path is located. Simultaneously, the object detection head outputs bounding boxes and categories. Detected objects include: vehicles, pedestrians, traffic cones, and guardrails. The output result is a set of vehicle positions in the local lane coordinate system (x, y, y). t y t ), category and confidence level; Next, redundant detection boxes are removed by non-maximum suppression, the segmentation mask and detection coordinates are restored to the original image scale, and the results are post-processed by morphological optimization and geometric fitting. The final output includes structured information on drivable areas, lane lines, and obstacles.

3. The parallel hybrid power energy management and control method based on reinforcement learning according to claim 1, characterized in that: Based on the driver's driving habits, a reinforcement learning algorithm is used to predict the vehicle's trajectory within the next 2 seconds. Inverse reinforcement learning is used to solve for the implicit reward function of the driver's historical driving trajectory as the expert driving trajectory, specifically: The real historical trajectory of a single driver is used as the expert trajectory set D. The trajectory data includes global coordinates, longitudinal velocity, heading angle in global coordinates, instantaneous curvature of the driving path, and longitudinal acceleration. Maximum entropy inverse reinforcement learning is used to output the optimized reward function for that driver. ; Identify the current driver and load their corresponding reward function. Then input the vehicle's state sequence S at the current moment. t-k：t Based on this reward function, the near-end policy optimization (PPO) algorithm is used for calculation, and a predicted motion trajectory for the next 2 seconds is generated through rolling execution.

4. The parallel hybrid power energy management and control method based on reinforcement learning according to claim 1, characterized in that: The specific kinetic energy recovery judgment condition in S2 is as follows: Determine if the current vehicle speed is greater than 3km / h and the vehicle's SOC is less than 90%. If so, determine whether the vehicle needs to decelerate based on its trajectory. If so, activate regenerative braking. The process of determining whether a vehicle needs to slow down based on its trajectory is as follows: For each dynamic or static obstacle based on its trajectory, perform the following checks: Calculate the minimum Euclidean distance d between the vehicle trajectory and the obstacle trajectory within a future time window. min ; If d min < d safe There is a risk involved; ； ； Time-of-Collision (TTC) check: TTC=d / Δv(when Δv>0), Δv=v ego -v obj ; Among them, v ego Vehicle speed; v obj d: Speed ​​of the obstacle ahead; d: Distance between the vehicle and the obstacle; safe : Safety distance; d0: Stationary buffer distance; t headway : Safe following distance; Δv: Relative speed; When TTC < 3 seconds and Δv > 0, or d min < d safe When this happens, a deceleration request is triggered.

5. The parallel hybrid power energy management and control method based on reinforcement learning according to claim 1, characterized in that: Specifically, S3 is: Enter the current vehicle speed V ego Wheel-side torque requirement T w Based on trajectory prediction information, query the efficiency map of the motor or engine to obtain the driving efficiency in hybrid mode; The speeds of the motor and engine are calculated based on the optimal operating point of comprehensive energy consumption, and the overall system efficiency is calculated by combining the efficiency maps of the motor and engine. At the same time, different torque distribution coefficients λ are traversed to find the optimal combination that maximizes system efficiency. The torque distribution scheme corresponding to the maximum efficiency is then recorded as the optimal strategy, and finally the target output power of the motor and engine is output based on this optimal strategy.

6. The parallel hybrid power energy management and control method based on reinforcement learning according to claim 1, characterized in that: S4 specifically includes: S401. Check the vehicle's movement status. If the vehicle is starting or moving at low speed, proceed to S402; if the vehicle is not starting, proceed to S403. S402. Determine whether the current state of charge (SOC) of the vehicle's power battery is greater than 20%. If so, the power required for vehicle operation is relatively small, and the electric motor provides power alone. Otherwise, the vehicle's power battery is insufficient. In order to ensure vehicle performance and charge the battery, the electric motor is turned off and switched to engine drive. At the same time, the electric motor charges the power battery in reverse. S403. Record the vehicle's current speed V, select the optimal efficiency-power point based on the MAP chart to obtain the optimal operating point, and determine the vehicle speed threshold V within the pre-calibrated engine high-efficiency range. th Determine whether the current vehicle speed V is greater than V. th This refers to whether the engine has entered its high-efficiency range. If so, it enters S404; otherwise, it enters S405. S404. At this time, the vehicle is operating in the engine's high-efficiency range. Determine whether the vehicle's current SOC is greater than 90%. If so, the battery has sufficient charge and the engine drives the vehicle alone without charging the battery. Otherwise, the battery needs to be charged and the engine drives the vehicle alone, while the electric motor charges the battery in reverse. Determine if the vehicle has entered a high-load condition. If so, the engine will keep operating in the high-efficiency range and the remaining power will be provided by the electric motor. Otherwise, maintain the current drive mode. S405. Determine whether the power provided by the electric motor meets the vehicle's power requirements and whether the vehicle's current SOC is greater than 20%. If it does, continue to use the electric motor as the power source. If it does not meet the requirements, it indicates that the vehicle's power demand has increased. Start the engine as auxiliary power to drive the vehicle together with the electric motor.

7. The parallel hybrid power energy management and control method based on reinforcement learning according to claim 6, characterized in that: Also includes: S406, Kinetic Energy Recovery Coordination Control: During the power source control process of S402 to S405, if a decision to start kinetic energy recovery is received from S2 at the same time, kinetic energy recovery will be prioritized without violating the power battery's safe charging power limit or causing the power source to exit its high-efficiency operating range. The electrical energy generated by kinetic energy recovery is used to charge the power battery, and this charging process is coordinated with the operating status of the power source.

8. A system for implementing the reinforcement learning-based parallel hybrid power energy management and control method according to any one of claims 1-7, characterized in that, It includes a sensor system, a motion trajectory prediction module, a kinetic energy recovery decision module, a global power distribution module, a power source control and execution module, and a parallel power system; The sensor system is used to collect vehicle operation data. Its output is connected to the input of the motion trajectory prediction module and the global power allocation module, respectively, to provide real-time data required to execute steps S1, S2 and S3. The input terminal of the motion trajectory prediction module is communicatively connected to the sensor system. Based on sensor data, it processes information through convolutional neural networks and reinforcement learning algorithms to perform the function of predicting the vehicle's motion trajectory within the next 2 seconds in step S2. The input end of the kinetic energy recovery decision module is communicatively connected to the output end of the motion trajectory prediction module. It determines whether deceleration is needed and decides the recovery intensity based on the predicted motion trajectory in order to execute the decision function on kinetic energy recovery in step S2. The input terminal of the global power allocation module is communicatively connected to the output terminal of the sensor system and the motion trajectory prediction module, respectively. Based on the real-time vehicle status and prediction information, it iteratively optimizes through unit-based algorithms and optimization algorithms to perform the function of generating the global optimal power allocation scheme in step S3. The input terminal of the power source control execution module is communicatively connected to the output terminals of the global power allocation module and the kinetic energy recovery decision module, respectively. Based on the global power allocation scheme and kinetic energy recovery decision, it generates coordinated control commands to execute the function of selecting the power source combination mode and coordinating the power output and kinetic energy recovery in step S4. The control terminal of the parallel power system is electrically connected to the output terminal of the power source control execution module, and receives and executes drive and recovery commands from the module; the parallel power system includes an engine, an electric motor, a power battery and a transmission mechanism, wherein the torques of the engine and the electric motor are coupled at the gear ring of the transmission mechanism.

Images