Electric vehicle transmission system torsional vibration suppression method combining reinforcement learning and proportional differential control

Abstract

The invention provides an electric vehicle torsional vibration control method based on reinforcement learning and proportional differential control, and aims to solve the problems of insufficient torsional vibration suppression effect and poor smoothness of an electric vehicle transmission system. The method comprises the following steps: firstly, building a torsional vibration simulation dynamical model of the electric vehicle transmission system, describing interaction among components and system dynamic response under different working conditions (acceleration, deceleration and starting), and correcting the simulation model through test data; secondly, designing a proportional differential controller by taking a torsion angle and a longitudinal acceleration tracking error of the transmission system as an adjustment basis; and finally, intelligently adjusting a proportional gain and a differential gain of a proportional-differential controller by using a depth deterministic strategy gradient (DDPG) method. Reinforcement learning and proportional differential control are fused into electric vehicle torsional vibration control, the method has the advantages of being high in control precision and adaptability, and the torsional vibration suppression effect of a transmission system and the dynamic performance of a vehicle under different working conditions can be effectively improved.

Description

Technical Field

[0001] The present invention belongs to the field of electric vehicle torsional vibration control, and specifically relates to a method for suppressing torsional vibration of an electric vehicle transmission system by combining reinforcement learning with proportional differential control. Background Art

[0002] With the increasing popularity of electric vehicles, users are increasingly demanding smoothness and comfort, and drivetrain torsional vibration control has become a key technology. Electric vehicle drivetrain torsional vibration control technology relies on advanced control algorithms to precisely regulate motor output torque, suppress torsional vibrations in the drivetrain, and achieve smooth and stable driving. It is currently a hot topic and a challenge in the electric vehicle field.

[0003] Some existing patents, such as patent CN117141251A, design a method for active torsional vibration control during motor drive. This method combines the reconstructed torque of non-co-phase torque superposition with the feedback torque based on a two-degree-of-freedom torsional vibration model to form the motor output torque. This solves the problem of difficulty in rapidly reducing the torsional vibration of the transmission system due to sudden changes in motor torque during motor drive. However, this patent lacks adaptive control of algorithm parameters. Patent CN118199454A proposes a torsional vibration suppression strategy for an electric drive system based on two-layer model predictive control. By controlling the motor's target torque and reducing current harmonics and torque fluctuations, this strategy suppresses the torsional vibration of the integrated electric drive system, enhances the stability of the electric drive system, and improves vehicle comfort. However, this patent lacks discussion of the algorithm's self-learning capabilities.

[0004] Traditional control methods, such as proportional-derivative (PD) control, while capable of rapid response and oscillation suppression, remain limited in their ability to handle complex dynamic conditions and nonlinear characteristics, making it difficult to achieve optimal torsional vibration suppression. In electric vehicle drivetrains, in particular, sudden changes in motor output torque can easily induce torsional vibration, leading to longitudinal vehicle jerk and impacting ride smoothness and comfort. Furthermore, while deep reinforcement learning (DRL) has demonstrated significant potential in intelligent control, enabling dynamic adjustment of control parameters through self-learning mechanisms to adapt to complex operating conditions, its application in torsional vibration control requires further exploration and verification. Therefore, developing an adaptive and robust electric vehicle torsional vibration control algorithm to enhance the dynamic response and torsional vibration suppression of the drivetrain remains a pressing technical challenge. Summary of the Invention

[0005] The present invention provides a method for suppressing torsional vibration of an electric vehicle transmission system by combining reinforcement learning with proportional-differential control. By building a torsional vibration dynamics model and designing a proportional-differential controller based on the torsion angle and longitudinal acceleration tracking error, the Deep Deterministic Policy Gradient (DDPG) algorithm is used to intelligently optimize the controller parameters to achieve rapid response and torsional vibration suppression under different operating conditions.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] A method for suppressing torsional vibration in an electric vehicle transmission system that combines reinforcement learning with proportional-differential control is described. The transmission system includes a power battery, a battery energy management system, a motor, a motor controller, a speed reducer, and wheels. The power battery supplies power to the motor through the battery energy management system, and the motor controller adjusts the motor speed, which is then transmitted to the wheels via the speed reducer. The control method includes:

[0008] S1: Build a torsional vibration dynamics model of the electric vehicle drive system, modeling the motor, gearbox, drive shaft, wheels, and entire vehicle components. Describe the interactions between components and the system's dynamic response under different operating conditions (acceleration, deceleration, and starting). Optimize and improve the model by comparing simulation and test data to ensure model accuracy.

[0009] S2: Based on the dynamic response of the transmission system under different working conditions, the proportional differential controller is designed with the transmission system torsion angle and longitudinal acceleration tracking error as the adjustment basis to optimize the proportional gain k p and the differential gain k d To adjust the motor output torque to meet the torsional vibration suppression requirements of the transmission system.

[0010] S3: Based on the torsional vibration suppression requirements of the transmission system, the Deep Deterministic Policy Gradient (DDPG) algorithm is used to train the proportional differential control parameter optimization model and intelligently adjust the proportional differential controller proportional gain k. p and the differential gain k d , including state space, action space and reward function design.

[0011] S4: Train the proportional differential control parameters of the DDPG reinforcement learning algorithm under different operating conditions and output them to the electric vehicle to quickly respond to and suppress the torsional vibration of the transmission system.

[0012] Furthermore, the torsional vibration dynamics model of the electric vehicle transmission system is constructed in step S1 as follows:

[0013] S11: Based on Newton's second law and the lumped mass method, the inertia elements are set as the motor, gearbox, final reducer, wheels, and body, and the torsional vibration dynamics equation of the transmission system is written as:

[0014]

[0015] Where J1~J5 are the rotational inertia of each inertia component, K1~K4 are the torsional stiffness between each inertia component, C1~C4 are the damping between each inertia component, θ1~θ5 are the torsional angular displacement of each inertia component, T m is the motor output torque, T L is the vehicle resistance torque.

[0016] S12: Setting different working conditions, i.e., different motor torque requirements, inputting them into the torsional vibration dynamics model, and solving the transmission system dynamic response, including the torsional angular displacement, angular velocity, and angular acceleration of each inertia element;

[0017] S13: Verify the rationality of the constructed torsional vibration dynamics model through response data collected from the actual vehicle.

[0018] Furthermore, the tracking error in step S2 is expressed as:

[0019] e(t)={θ ref -θ act , a ref -a act} (2)

[0020] Where θ ref is the expected value of the torsion angle of the transmission system; θ act is the actual value of the torsion angle of the transmission system; a ref is the expected value of longitudinal acceleration; θ road is the actual value of longitudinal acceleration.

[0021] According to the principle of proportional differential controller, the motor torque control output T can be obtained m (t) is:

[0022]

[0023] Furthermore, in step S3, the state space is the angular displacement of each inertial element; the action space is the proportional differential controller parameters; the reward function includes the longitudinal acceleration stability difference and the transmission system torsion angle difference. The state information, expected longitudinal acceleration and transmission system torsion angle values and accumulated reward values obtained during the training process are stored in the cache pool, and sample data are randomly collected for neural network training, so as to adjust the proportional gain k p and the differential gain k d Perform adaptive tuning.

[0024] Furthermore, the step S3 is specifically as follows:

[0025] S31: Rewrite equation (1) into matrix form:

[0026]

[0027] Where θ is the angular displacement vector, is the angular velocity vector, is the angular acceleration vector, J is the moment of inertia matrix, T is the external torque vector, C is the damping matrix, and K is the stiffness matrix, which are expressed as follows:

[0028]

[0029] S32: Angular displacement vector θ = [θ1θ2θ3θ4θ5] T As the state variable S of the DDPG reinforcement learning algorithm; set the action variable to the proportional differential controller parameter A={k p k d Set the reward function R to the difference in longitudinal acceleration stability and the difference in transmission system torsion angle:

[0030] R=-[α′·(θ ref -θ act ) 2 +β′·(a ref -a act ) 2 ] (5)

[0031] Where α′ and β′ are the weight factors of the longitudinal acceleration term and the transmission system torsion angle term, respectively.

[0032] S33: Run the reinforcement learning algorithm for training, saving the state S, action a, reward function R, and other data generated during the interaction process into the experience pool. The agent Q network is then trained through small batch random sampling. The agent selects actions based on the current state (adjusting differential-integral control parameters) and receives feedback based on the reward function R to update the evaluation Q network. Through multiple iterative learning, the agent obtains the optimal differential-integral control parameters under different operating conditions, thereby optimizing control performance.

[0033] Compared with the prior art, the advantages of the present invention are:

[0034] This paper proposes a method for controlling torsional vibration in electric vehicles based on reinforcement learning and proportional-differential control. This method addresses the issues of insufficient torsional vibration suppression and poor ride comfort in electric vehicle transmission systems. By building a transmission system torsional vibration simulation dynamics model and incorporating a proportional-differential controller, it achieves high-precision control of the transmission system's torsional angle and longitudinal acceleration, providing a novel solution for torsional vibration control in electric vehicles.

[0035] 2. This invention describes a method for controlling torsional vibration in electric vehicles based on reinforcement learning and proportional-differential control. This method combines the Deep Deterministic Policy Gradient (DDPG) algorithm with proportional-differential control to achieve intelligent adjustment of proportional and differential gains. By incorporating the reinforcement learning algorithm, the controller's adaptability and robustness under various operating conditions are significantly improved, effectively suppressing torsional vibration in the drivetrain.

[0036] 3. This invention describes a method for controlling torsional vibration in electric vehicles based on reinforcement learning and proportional-differential control. It designs a control strategy based on the transmission system's torsional angle and longitudinal acceleration tracking errors. By adjusting the proportional and differential gains in real time, it achieves rapid response and precise suppression of transmission system torsional vibrations, significantly improving the vehicle's dynamic performance and ride comfort, providing important guidance for torsional vibration control in electric vehicles. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A schematic diagram of the electric vehicle system structure used in the embodiment;

[0038] Figure 2 A flow chart of a method for controlling torsional vibration of an electric vehicle based on reinforcement learning and proportional-differential control used in the embodiment;

[0039] Figure 3 The five-degree-of-freedom concentrated mass model of the transmission system used in the embodiment;

[0040] Figure 4 A schematic diagram of the proportional differential control algorithm used in the embodiment;

[0041] Figure 5 Flowchart of optimizing proportional differential control parameters using the DDPG reinforcement learning algorithm used in this embodiment.

[0042] Labels in the figure: 1. Power battery, 2. Battery energy management system, 3. Motor, 4. Motor controller, 5. Reducer, 6. Wheel. DETAILED DESCRIPTION

[0043] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the examples described are only some embodiments of the present invention, not all embodiments. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are also within the scope of protection of the present invention.

[0044] Example

[0045] A method for controlling torsional vibration of an electric vehicle based on reinforcement learning and proportional-differential control is described. The transmission system includes a power battery 1, a battery energy management system 2, a motor 3, a motor controller 4, a reducer 5, and wheels 6. The power battery 1 supplies power to the motor 3 via the battery energy management system 2. The motor controller 4 adjusts the torque of the motor 3, which is then transmitted to the wheels 6 via the reducer 5.

[0046] The flow chart of the torsional vibration control method based on reinforcement learning and proportional differential for electric vehicles is as follows: Figure 2 As shown in the figure. First, a dynamic model for torsional vibration simulation of the electric vehicle transmission system is built to describe the interaction between the components and the dynamic response of the system under different working conditions. The simulation model is then corrected using test data. Then, a proportional-differential controller is designed based on the transmission system torsional angle and longitudinal acceleration tracking errors. Finally, the proportional and differential gains of the proportional-differential controller are intelligently adjusted using the DDPG reinforcement learning method. The torsional vibration control method specifically includes:

[0047] S1: Build a torsional vibration dynamics model of the electric vehicle transmission system, and model the motor, gearbox, drive shaft, wheels and vehicle components, such as Figure 3 As shown in the figure, the interaction between components and the dynamic response of the system under different working conditions (acceleration, deceleration, and starting) are described. The model is optimized and improved by comparing simulation and test data to ensure model accuracy. The specific process is as follows:

[0048] S11: Based on Newton's second law and the lumped mass method, the inertia elements are set as the motor, gearbox, final reducer, wheels, and body, and the torsional vibration dynamics equation of the transmission system is written as:

[0049]

[0050] Where J1~J5 are the rotational inertia of each inertia component, K1~K4 are the torsional stiffness between each inertia component, C1~C4 are the damping between each inertia component, θ1~θ5 are the torsional angular displacement of each inertia component, T m is the motor output torque, T L is the vehicle resistance torque.

[0051] S12: Setting different working conditions, i.e., different motor torque requirements, inputting them into the torsional vibration dynamics model, and solving the transmission system dynamic response, including the torsional angular displacement, angular velocity, and angular acceleration of each inertia element;

[0052] S13: Verify the rationality of the constructed torsional vibration dynamics model through response data collected from the actual vehicle.

[0053] S2: Based on the dynamic response of the transmission system under different working conditions, the proportional differential controller is designed with the transmission system torsion angle and longitudinal acceleration tracking error as the adjustment basis to optimize the proportional gain k p and the differential gain k d To adjust the motor output torque to meet the transmission system torsional vibration suppression requirements, such as Figure 4 As shown. Among them, the tracking error is expressed as:

[0054] e(t)={θ ref -θ act , a ref -a act} (2)

[0055] Where θ ref is the expected value of the torsion angle of the transmission system; θ act is the actual value of the torsion angle of the transmission system; a ref is the expected value of longitudinal acceleration; θ road is the actual value of longitudinal acceleration.

[0056] According to the principle of proportional differential controller, the motor torque control output T can be obtained m (t) is:

[0057]

[0058] S3: Based on the torsional vibration suppression requirements of the transmission system, the Deep Deterministic Policy Gradient (DDPG) algorithm is used to train the proportional differential control parameter optimization model and intelligently adjust the proportional differential controller proportional gain k. p and the differential gain k d , including state space, action space and reward function design, such as Figure 5 As shown in the figure. The state space is the angular displacement of each inertial element; the action space is the proportional differential controller parameters; the reward function includes the longitudinal acceleration stability difference and the transmission system torsion angle difference. The state information, expected longitudinal acceleration and transmission system torsion angle values and accumulated reward values obtained during the training process are stored in the cache pool, and sample data are randomly collected for neural network training to adjust the proportional gain k. p and the differential gain k d Perform adaptive tuning.

[0059] S31: Rewrite equation (3) into matrix form:

[0060]

[0061] Where θ is the angular displacement vector, is the angular velocity vector, is the angular acceleration vector, J is the moment of inertia matrix, T is the external torque vector, C is the damping matrix, and K is the stiffness matrix, which are expressed as follows:

[0062]

[0063]

[0064] S32: Angular displacement vector θ = [θ1θ2θ3θ4θ5] T As the state variable S of the DDPG reinforcement learning algorithm; set the action variable to the proportional differential controller parameter A={k p k d Set the reward function R to the difference in longitudinal acceleration stability and the difference in transmission system torsion angle:

[0065] R=-[α′·(θ ref -θ act ) 2 +β′·(a ref -a act ) 2 ] (5)

[0066] Where α′ and β′ are the weight factors of the longitudinal acceleration term and the transmission system torsion angle term, respectively.

[0067] S33: Run the reinforcement learning algorithm for training, saving the state S, action a, reward function R, and other data generated during the interaction process into the experience pool. The agent Q network is then trained through small batch random sampling. The agent selects actions based on the current state (adjusting differential-integral control parameters) and receives feedback based on the reward function R to update the evaluation Q network. Through multiple iterative learning, the agent obtains the optimal differential-integral control parameters under different operating conditions, thereby optimizing control performance.

[0068] S4: Train the proportional differential control parameters of the DDPG reinforcement learning algorithm under different operating conditions and output them to the electric vehicle to quickly respond to and suppress the torsional vibration of the transmission system.

[0069] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. All equivalent structural transformations made by using the contents of the present invention description and drawings under the inventive concept of the present invention, or direct / indirect application in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. A method for controlling torsional vibration of an electric vehicle by combining reinforcement learning with proportional differential control, characterized in that: The steps include: S1: Build a torsional vibration dynamics model of the electric vehicle transmission system, modeling the motor, gearbox, drive shaft, wheels, and entire vehicle components. Describe the interactions between components and the system's dynamic response under different operating conditions. Optimize and improve the model by comparing simulation and test data to ensure model accuracy. S2: Based on the dynamic response of the transmission system under different working conditions, the proportional differential controller is designed with the transmission system torsion angle and longitudinal acceleration tracking error as the adjustment basis to optimize the proportional gain k p and the differential gain k d To adjust the motor output torque to meet the transmission system torsional vibration suppression requirements; S3: Based on the torsional vibration suppression requirements of the transmission system, the proportional differential control parameter optimization model is trained using a deep deterministic policy gradient algorithm to intelligently adjust the proportional differential controller proportional gain k p and the differential gain k d , including state space, action space and reward function design; S4: Train the proportional differential control parameters of the DDPG reinforcement learning algorithm under different operating conditions and output them to the electric vehicle to quickly respond to and suppress the torsional vibration of the transmission system.

Images