A reinforcement learning self-tuning method for torsional vibration suppression control parameters in hybrid power systems during mode switching.

By combining the lumped parameter method and reinforcement learning agents, the adaptive and safety issues of transmission system torsional vibration suppression during mode switching in hybrid electric vehicles are solved. This enables smoothness and adaptive adjustment of control parameters throughout the entire life cycle of hybrid electric vehicles, thereby improving the torsional vibration suppression effect and safety of the transmission system.

CN123116431APending Publication Date: 2026-11-17ZHENGZHOU UNIVERSITY OF LIGHT INDUSTRY
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202611014768.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-07-09
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing hybrid electric vehicle mode switching control methods have shortcomings in dynamic modeling, basic controller structure, control parameter tuning, and reward function design, resulting in poor torsional vibration suppression of the transmission system and a lack of adaptability and safety constraints, making it difficult to meet the smoothness requirements throughout the entire life cycle and under various operating conditions.

Method used

A lumped parameter method is used to establish a dynamic model for the mode switching of a hybrid power system. A basic torsional vibration suppression controller is designed, and key parameters are adaptively tuned through a reinforcement learning agent, including torque coordination, power source torque compensation, and motor active damping modules. Combined with multi-dimensional state perception and online real-time correction, adaptive adjustment of control parameters and safety constraints are achieved.

Benefits of technology

It achieves accurate description and suppression of torsional vibration response of transmission system, improves smoothness and safety during mode switching, covers the entire life cycle of hybrid electric vehicles and control adaptability under multiple operating conditions, and ensures the safety, reliability and multi-objective optimization of control parameters.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123116431A_ABST
    Figure CN123116431A_ABST
Patent Text Reader

Abstract

This invention discloses a reinforcement learning self-tuning method for torsional vibration suppression control parameters in hybrid electric vehicle (HEV) mode switching, belonging to the field of torsional vibration control and reinforcement learning parameter self-tuning for HEVs. The method first establishes a dynamic model of the HEV mode switching, describing the torque coupling relationship between the engine, motor, clutch, drive shaft, and the vehicle's longitudinal dynamics. Then, a basic torsional vibration suppression controller is designed, incorporating torque coordination, power source torque compensation, and active motor damping. The active motor damping coefficient, power source torque compensation coefficient, torque transition time, and torque change rate limit are used as adjustable control parameters. Next, a reinforcement learning agent is constructed, taking the mode switching state, power source operating state, and transmission system torsional vibration response as state inputs, and the correction amount of the adjustable control parameters as the action output. A reward function that balances torsional vibration suppression, impact reduction, and control smoothness is designed. After training under multiple mode switching conditions, the agent adaptively tunes the control parameters online, and a safety constraint module provides amplitude limiting protection for the output. This invention can automatically adjust control parameters according to operating conditions, effectively reducing transmission system torsional vibration and longitudinal impact, and significantly improving the smoothness and adaptability of HEV mode switching.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of torsional vibration control and reinforcement learning parameter self-tuning for hybrid electric vehicles, specifically a reinforcement learning self-tuning method for torsional vibration suppression control parameters for mode switching in hybrid electric systems. Background Technology

[0002] With the increasing adoption of hybrid electric vehicles, users are demanding smoother vehicle mode switching and greater ride comfort. During mode switching in a hybrid system, the complex coupling between the power sources and the transmission system can easily induce torsional vibrations in the transmission. Precisely coordinating the torque output of the power sources and suppressing torsional vibrations in the transmission system during mode switching to achieve smooth and stable vehicle operation is currently a hot topic and a significant challenge in the field of hybrid electric vehicle control.

[0003] Some existing patents, such as the invention patent with patent number CN114771498A, propose an engine torque control system for hybrid electric vehicle mode switching. This system achieves torque adjustment during engine operating condition transitions through demand torque estimation, motor load torque observation, and engine torque feedback control, improving engine starting performance and reducing torque fluctuations. However, this method mainly focuses on engine torque control and lacks comprehensive feedback on the torsional vibration response of the transmission system and adaptive tuning of control parameters. The invention patent with patent number CN113602254A proposes a method, device, and hybrid electric vehicle for coordinated control of engine starting and transmission shifting. This method reduces the impact of the engine starting and shifting coupling process on vehicle smoothness by judging engine start requests, correcting gear shifting boundaries, and coordinating the timing of engine starting and transmission shifting. However, this method focuses on the coordinated control of engine starting and transmission shifting and does not establish a parameter tuning mechanism for the torsional vibration response of the transmission system during mode switching.

[0004] Traditional hybrid mode switching control methods have at least the following technical drawbacks: I. Dynamic Modeling: Traditional methods often employ simplified models or establish fixed-parameter models for each mode. This makes it difficult to accurately describe the elastic deformation of the drive shaft and the system's damping characteristics, resulting in a lack of precise physical basis for controller design and a heavy reliance on empirical trial-and-error for control parameter tuning. Furthermore, model parameters cannot adaptively adjust under different mode switching conditions, leading to model jumps at mode switching boundaries and affecting the smooth transition of control parameters. II. Basic Controller Structure: Traditional basic torsional vibration suppression controllers typically only possess a single torque coordination function, making it difficult to effectively compensate for the torque build-up lag caused by physical delays in engine intake, fuel injection, and combustion. They also lack the ability to actively generate damping torque based on the torsional vibration response of the transmission system, failing to achieve rapid and effective attenuation of torsional angular velocity oscillation amplitude. In addition, traditional controllers often use fixed control parameters, which are fixed once calibrated, lacking the flexibility to adapt to changing operating conditions. III. Control Parameter Tuning: Traditional methods often use fixed control parameters, achieving certain results under calibrated conditions, but easily exhibiting insufficient adaptability when facing varying operating conditions. If the motor's active damping coefficient and torque transition time are too small, the torsional vibration of the transmission system will be difficult to effectively attenuate; if they are too large, it may cause sudden changes in torque response and longitudinal impact. Especially after the damping characteristics of the transmission system change due to clutch wear, the fixed parameter control method cannot achieve adaptive compensation, making it difficult to meet the smoothness requirements of hybrid vehicles throughout their entire life cycle and under all operating conditions. IV. Control parameter safety constraints. Existing adaptive control methods usually lack effective safety constraint mechanisms for the tuned control parameters. When the parameter tuning strategy outputs abnormally or the agent's input state signal is distorted, the out-of-control control parameters may directly act on the actuator, leading to motor overload, excessive clutch torque transmission, or excessive load on the transmission bearings, posing serious safety hazards. V. Reward function design. The reward function design in existing learning-based control methods is relatively simple, usually guided by a single objective (such as only suppressing torsional vibration or only tracking torque). It is difficult to take into account multiple conflicting control objectives such as suppressing torsional vibration, reducing impact, ensuring torque tracking accuracy, and limiting sudden changes in control quantity. As a result, optimizing one objective often leads to the deterioration of the performance of other control objectives.

[0005] Therefore, there is an urgent need to propose an adaptive tuning method for torsional vibration suppression control parameters for hybrid power system mode switching. This method should automatically adjust key control parameters based on the mode switching state and the torsional vibration response of the transmission system to improve the torsional vibration suppression effect and power transmission smoothness. Summary of the Invention

[0006] This invention proposes a reinforcement learning self-tuning method for torsional vibration suppression control parameters in hybrid electric vehicles during mode switching. This method primarily aims to suppress torsional vibration in the transmission system during mode switching processes such as pure electric drive, engine engagement, hybrid drive, and engine disengagement. First, a dynamic model describing the dynamic response of mode switching is established. Then, a basic torsional vibration suppression controller is designed. Finally, a reinforcement learning agent is used to adaptively tune the key parameters of the basic controller, thereby improving the control adaptability and smoothness under different mode switching conditions.

[0007] The objective of this invention can be achieved through the following technical solutions: A reinforcement learning self-tuning method for torsional vibration suppression control parameters in a hybrid power system, wherein the hybrid power system is a parallel P2 hybrid power system, including an engine, clutch, drive motor, motor controller, power battery, battery management system, gearbox, driveshaft, final drive, and wheels; the power battery supplies power to the drive motor through the battery management system, the motor controller regulates the output torque of the drive motor, and the engine output torque and drive motor output torque are coupled before the gearbox, and then transmitted to the wheels via the gearbox, driveshaft, and final drive; the control method includes: S1: Establish a dynamic model for the mode switching of the hybrid power system, model the longitudinal dynamics of the engine, motor, clutch, drive shaft and the whole vehicle, and analyze the coupling relationship of torque changes during pure electric drive, engine intervention, hybrid drive and engine withdrawal. S2: Based on the torsional vibration response of the transmission system during mode switching, a basic torsional vibration suppression controller is designed. The basic torsional vibration suppression controller includes a torque coordination module, a power source torque compensation module, and a motor active damping module. The motor active damping coefficient, power source torque compensation coefficient, torque transition time, and torque change rate limit are used as adjustable control parameters. The torque transition time determines the duration for the torque to transition from the current value to the target value.

[0008] S3: Construct a reinforcement learning agent, taking the mode switching state, power source operating state and transmission system torsional vibration response as state inputs, taking the correction amount of the adjustable control parameters as action outputs, and designing a reward function with the goal of reducing transmission system torsional vibration, reducing longitudinal impact and maintaining smooth control input. S4: Train the reinforcement learning agent under multiple mode switching conditions to obtain adaptive tuning strategies for control parameters under different mode switching states; S5: The trained reinforcement learning agent is used for the mode switching control of the hybrid power system. Based on the current mode switching conditions and the torsional vibration response of the transmission system, the adjustable control parameters of the basic torsional vibration suppression controller are corrected online, and the output parameters are limited and corrected through the safety constraint module.

[0009] Furthermore, the hybrid power system mode switching dynamic model in step S1 is established using the lumped parameter method. Components such as the engine, motor, clutch driving end, clutch driven end, drive shaft, and wheels are equivalent to multiple lumped inertial elements, and equivalent stiffness and damping parameters are introduced. The dynamic equations of this model describe the coupling relationship between the angular displacement, angular velocity, and angular acceleration of each inertial element in matrix form. The equations include equivalent inertia matrix, damping matrix, and stiffness matrix, which take corresponding parameter values ​​according to different mode switching states. The model inputs include engine torque, motor torque, clutch transmitted torque, and load resistance torque, which act on the system through corresponding torque input matrices. After solving the dynamic equations to obtain the angular displacement and angular velocity of each inertial element, the angular displacement components corresponding to the input and output ends of the drive shaft are selected to characterize the torsional deformation of the transmission system, thereby defining the torsional angle and torsional angular velocity of the transmission system.

[0010] Further, in step S2, the torque coordination module is used to generate the basic target torque of the engine and motor during the mode switching process. It calculates the basic target torque of the engine and motor during the mode switching process based on the engine and motor torque before mode switching, the target torque of the engine and motor after mode switching, and the torque transition time. The power source torque compensation module calculates the power source torque compensation amount based on the power source torque compensation coefficient and the engine target torque to compensate for the drive torque deviation caused by the engine torque build-up lag. The motor active damping module generates the motor active damping compensation torque based on the motor active damping coefficient and the torsional vibration response of the transmission system to suppress the torsional vibration of the transmission system. The output torque of the mode switching basic torsional vibration suppression controller consists of the engine control torque and the motor control torque, wherein the engine control torque is composed of the engine basic target torque, and the motor control torque is obtained by superimposing the motor basic target torque, the power source torque compensation amount, and the motor active damping compensation torque.

[0011] Further, in step S3, the reinforcement learning agent uses the mode switching state, power source operating state, and transmission system torsional vibration response as state variables, and the correction amount of the adjustable control parameters in the basic torsional vibration suppression controller as action variables. The effect of control parameter tuning is evaluated through a reward function. The state variables include the mode switching state, engine speed, motor speed, vehicle demand torque, vehicle speed, battery SOC, transmission system torsional angle, torsional angular velocity, and vehicle longitudinal acceleration. Based on the state variables at the current moment, the reinforcement learning agent outputs action variables, i.e., the control parameter correction vector, through its policy function. This vector includes the correction amount of the motor active damping coefficient, the power source torque compensation coefficient, the torque transition time, and the torque change rate limit. The reward function includes a transmission system torsional vibration penalty term, a longitudinal impact penalty term, a torque tracking error penalty term, and a control input change penalty term, which are used to suppress torsional vibration, reduce impact, ensure torque tracking accuracy, and limit control quantity abrupt changes, respectively. Each penalty term is multiplied by its corresponding weight coefficient, summed, and a constant term is added to form the final reward value.

[0012] Further, in step S5, after the reinforcement learning agent outputs the control parameter correction amount, the preset basic control parameter vector is added to the control parameter correction amount vector output by the agent to obtain the control parameter vector after reinforcement learning tuning. This vector includes the motor active damping coefficient, the power source torque compensation coefficient, the torque transition time, and the torque change rate limit. To avoid the reinforcement learning output parameters exceeding the actuator's allowable range or causing control anomalies, a safety constraint module is set to limit the updated control parameters, ensuring that the final control parameters always remain between the preset lower and upper limits of the allowable control parameter range. In addition, when the reinforcement learning agent outputs an anomaly, the parameters do not meet the safety boundaries, or the control system detects that the torque change rate exceeds the limit, the control system switches to the preset basic control parameters to ensure system reliability.

[0013] Compared with the prior art, the advantages of this invention are: 1. Based on centralized parameter unified modeling, accurate description of torsional vibration response throughout the mode switching process is achieved. This invention employs a lumped parameter method to establish a dynamic model for hybrid power system mode switching. Components such as the engine, motor, clutch, drive shaft, and wheels are represented as multiple lumped inertial elements, and equivalent stiffness and damping parameters are introduced. This accurately describes the influence of torque changes on torsional vibration response during different mode switching processes, including pure electric drive, engine engagement, hybrid drive, and engine disengagement. The equivalent inertia matrix, damping matrix, and stiffness matrix in the model can be assigned corresponding parameter values ​​according to different mode switching states. The same model framework covers different transmission configuration states such as clutch disengagement, slippage, and engagement, avoiding model jumps at mode switching boundaries. This model also serves as a high-fidelity interactive environment for offline training of reinforcement learning agents, allowing agent training to be primarily completed offline based on the dynamic model, supplemented only by a small number of real-vehicle tests in the final verification stage. This significantly shortens the development cycle and mitigates safety risks in the early training phase.

[0014] 2. A three-layer collaborative basic control architecture that combines inherent vibration damping capabilities with adjustable parameter flexibility. The basic torsional vibration suppression controller designed in this invention modularizes three functions: torque coordination, power source torque compensation, and active motor damping. The torque coordination module transforms abrupt torque commands into smooth transition trajectories, avoiding direct excitation of torsional vibration in the transmission system. The power source torque compensation module uses a faster-responding motor to compensate for the torque build-up lag caused by physical delays in the engine, ensuring continuous and accurate wheel-end drive torque. The active motor damping module generates a reverse compensation torque based on the torsional angular velocity of the transmission system, effectively attenuating the amplitude of torsional angular velocity oscillations and reducing the periodic alternating stress on the drive shaft. These three modules each contain key adjustable parameters such as torque transition time, power source torque compensation coefficient, and active motor damping coefficient, collectively forming a reinforcement learning tuning object. This enables the controller to possess basic vibration suppression capabilities while providing the flexibility for parameters to adapt to changing operating conditions.

[0015] 3. Multi-dimensional state perception combined with multi-condition training enables adaptive tuning of optimal parameters across the entire operating range. This invention selects nine categories of state variables, including mode switching state, engine speed, motor speed, vehicle demand torque, vehicle speed, battery SOC, transmission system torsional angle, torsional angular velocity, and vehicle longitudinal acceleration. It provides comprehensive operational condition perception for the intelligent agent from four dimensions: mode switching process, power source operating point, system energy state, and torsional vibration response. The training phase covers switching from pure electric drive to hybrid drive, engine intervention, switching from hybrid drive to pure electric drive, engine deactivation, and the power source torque handover process under different driver torque demands, covering almost all mode switching scenarios in actual use of hybrid vehicles. Four core parameters—motor active damping coefficient, power source torque compensation coefficient, torque transition time, and torque change rate limit—work collaboratively to fundamentally solve the dilemma of "torsional vibration attenuation being difficult when parameters are too small, and shocks being caused when parameters are too large" in fixed control parameters.

[0016] 4. Online real-time correction combined with dual security protection ensures safe and reliable control throughout the entire lifecycle. When the trained reinforcement learning agent is used for online vehicle control, the control parameters are no longer fixed static values ​​during the calibration phase, but are dynamically adjusted according to changes in driving conditions and torsional vibration response fluctuations. When clutch wear causes changes in the damping characteristics of the transmission system, the agent can automatically adjust the parameters according to the actual torsional vibration response, achieving adaptive compensation for the performance degradation of the transmission system. The safety constraint module limits the updated control parameters to ensure that the four core parameters are always within a safe range; at the same time, it sets three fault switching mechanisms: abnormal agent output, parameters exceeding safety boundaries, and torque change rate exceeding limits. When any of these conditions is triggered, the system automatically switches to the preset basic control parameters. Even in extreme cases such as occasional agent failure or abnormal input status signals, the vehicle can still reliably complete the mode switching.

[0017] 5. The multi-dimensional reward function accurately anchors multiple objectives, balancing torsional vibration suppression and ride comfort. The reward function designed in this invention includes a torsional vibration penalty term, a longitudinal impact penalty term, a torque tracking error penalty term, and a control input change penalty term, which precisely correspond to the four core control objectives: suppressing torsional vibration, reducing impact, ensuring torque tracking accuracy, and limiting abrupt changes in control input. The torsional vibration penalty term reduces alternating stress on the driveshaft at its source; the longitudinal impact penalty term reflects a control philosophy oriented towards driving comfort; the torque tracking error penalty term ensures that torque tracking accuracy is not sacrificed; and the control input change penalty term avoids unexpected system oscillations caused by frequent parameter jumps. The weighting coefficients can be flexibly adjusted according to different vehicle models and operating conditions, effectively balancing multiple conflicting control objectives and demonstrating strong engineering applicability. Attached Figure Description

[0018] Figure 1A schematic diagram of the parallel P2 hybrid power system used in the embodiment; Figure 2 The flowchart shows the reinforcement learning self-tuning method for torsional vibration suppression control parameters for hybrid power system mode switching used in the embodiment. Figure 3 The lumped-parameter torsional vibration dynamics model of the parallel P2 hybrid power transmission system used in the embodiment; Figure 4 This is a schematic diagram of the mode switching basic torsional vibration suppression controller structure used in the embodiment; Figure 5 The flowchart shows the process of tuning torsional vibration suppression control parameters for the reinforcement learning agent used in the embodiment.

[0019] The diagram is labeled as follows: 1. Engine, 2. Clutch, 3. P2 motor, 4. Motor controller, 5. Power battery, 6. Battery energy management system, 7. Gearbox, 8. Drive shaft, 9. Main reducer, 10. Wheel. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described examples are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0021] Example A reinforcement learning self-tuning method for torsional vibration suppression control parameters in a hybrid power system, wherein the transmission system is a parallel P2 hybrid power system, such as... Figure 1 As shown, the system includes an engine 1, a clutch 2, a P2 motor 3, a motor controller 4, a power battery 5, a battery energy management system 6, a gearbox 7, a drive shaft 8, a final drive 9, and wheels 10. The power battery 5 provides electrical energy to the P2 motor 3 through the battery energy management system 6, and the motor controller 4 regulates the output torque of the P2 motor 3. The engine 1 is coupled to the P2 motor 3 at the front end of the gearbox 7 via the clutch 2, and the coupled power is transmitted to the wheels 10 through the gearbox 7, drive shaft 8, and final drive 9.

[0022] A flowchart illustrating the reinforcement learning self-tuning method for torsional vibration suppression control parameters during mode switching in a parallel P2 hybrid power system is shown below. Figure 2 As shown. The specific implementation process of this method is as follows: S1: Establish a dynamic model for the mode switching of the hybrid power system.

[0023] The longitudinal dynamics of the engine, motor, clutch, drive shaft, and vehicle are modeled, with a focus on analyzing the coupling relationships of torque changes during pure electric drive, engine engagement, hybrid drive, and engine disengagement. In the modeling, components such as the engine, motor, clutch driving end, clutch driven end, drive shaft, and wheels are equivalent to multiple lumped inertial elements, and equivalent stiffness and damping parameters are introduced to describe the impact of power source torque changes on the torsional vibration response of the transmission system during mode switching.

[0024] The model is established using the lumped parameter method, such as Figure 3 As shown, its dynamic equation is: (1) In the formula, For each inertial element, angular displacement vector It is the angular velocity vector. It is the angular acceleration vector. , , respectively mode The equivalent inertia matrix, damping matrix, and stiffness matrix are as follows: For engine torque, This is the motor torque. To transmit torque to the clutch, For load resistance torque, , , , These are the torque input matrices.

[0025] After obtaining the angular displacement and angular velocity of each inertial element by solving equation (1), the angular displacement components corresponding to the input and output ends of the transmission shaft are selected. , These are used to characterize the torsional deformation of a transmission system. The torsional angle and torsional angular velocity of a transmission system are defined as follows: (2) In the formula, The transmission ratio is... For the torsional angle of the transmission system, This refers to the torsional angular velocity of the transmission system.

[0026] S2: Design mode switching basic torsional vibration suppression controller.

[0027] Based on the torsional vibration response of the transmission system during mode switching, a basic torsional vibration suppression controller is designed, the structure of which is as follows: Figure 4As shown, the system mainly includes a torque coordination module, a power source torque compensation module, and a motor active damping module. The torque coordination module generates the basic target torques for the engine and motor during mode switching. The power source torque compensation module compensates for drive torque deviations caused by engine torque build-up lag. The motor active damping module generates vibration suppression compensation torque based on the torsional vibration response of the transmission system. In this embodiment, the motor active damping coefficient, power source torque compensation coefficient, torque transition time, and torque change rate limit are used as adjustable control parameters for the controller.

[0028] The calculation formulas for each module are as follows: Torque Coordination Module: (3) In the formula, , These are the basic target torques of the engine and motor during the mode switching process, respectively. , These are the engine and motor torques before mode switching, respectively. , These are the target torques of the engine and motor after mode switching, respectively. This is the torque transition time.

[0029] Power source torque compensation module: (4) In the formula, This is the torque compensation amount of the power source. This is the torque compensation coefficient for the power source. This represents the engine's target torque.

[0030] Motor active damping module: (5) In the formula, For the motor's active damping compensation torque, This represents the active damping coefficient of the motor.

[0031] Finally, the output torque of the basic torsional vibration suppression controller is: (6) In the formula, To control the engine torque, This is the control torque for the motor.

[0032] S3: Construct a reinforcement learning agent and design a reward function.

[0033] This step constructs a reinforcement learning agent for adaptive tuning of the adjustable control parameters described in S2. The agent's state inputs include mode switching state, engine speed, motor speed, vehicle torque demand, vehicle speed, battery SOC, transmission system torsion angle, torsion angular velocity, and vehicle longitudinal acceleration; the action output is a correction vector for the adjustable control parameters, specifically expressed as: (7) In the formula, For the control parameter correction vector, superscript Indicates matrix transpose. For a moment State variables, To reinforce the policy function of the learning agent, This is the correction amount for the motor's active damping coefficient. This is the correction amount for the torque compensation coefficient of the power source. This is the torque transition time correction amount. This is the correction amount for the torque change rate limit.

[0034] The training objective of the reinforcement learning agent is to reduce torsional vibration of the transmission system, decrease longitudinal impact, and maintain smooth control input. The reward function designed for this purpose includes penalties for torsional vibration of the transmission system, longitudinal impact, torque tracking error, and control input variation, expressed as follows: (8) In the formula, As a reward value, This represents the longitudinal acceleration fluctuation of the vehicle. To drive the desired torque at the wheel end, To drive the actual torque at the wheel end, To control the amount of input variation, , , , , The weighting coefficients are as follows: the first four penalty terms are used to suppress torsional vibration, reduce shock, ensure torque tracking accuracy, and limit abrupt changes in control quantity, respectively. The fifth term is a constant term used to adjust the reward benchmark under different states to improve the agent's exploration efficiency in sparse reward scenarios. Since the constant term does not change with actions, it does not affect the convergence direction of the optimal policy during policy optimization.

[0035] S4: Reinforcement learning agent training.

[0036] The reinforcement learning agent was trained under various mode-switching conditions, including switching from pure electric drive to hybrid drive, engine intervention, switching from hybrid drive to pure electric drive, engine deactivation, and the torque handover process under different driver torque demands. During training, the reinforcement learning agent interacted with the dynamic model established in step S1, collecting state, action, reward, and next state data, and iteratively optimized the strategy function based on reward feedback. After training, the agent can output control parameter corrections adapted to the current operating conditions based on the current mode-switching state and the torsional vibration response of the transmission system, thus eliminating dependence on fixed calibration parameters.

[0037] S5: Online Applications and Security Constraints.

[0038] The trained reinforcement learning agent is deployed in the mode switching control of the hybrid power system. Based on the current mode switching conditions and the torsional vibration response of the transmission system, the adjustable control parameters of the basic torsional vibration suppression controller are corrected online, and the output parameters are limited and corrected through a safety constraint module. The process is as follows: Figure 5 As shown.

[0039] Specifically, after the intelligent agent outputs the control parameter correction amount, the basic control parameters are updated online in the following manner: (9) In the formula, To reinforce the learning of the tuned control parameter vector, For the preset basic control parameter vector, This refers to the vector of control parameter corrections output by the reinforcement learning agent. Control parameters include the motor active damping coefficient, power source torque compensation coefficient, torque transition time, and torque change rate limit.

[0040] To prevent reinforcement learning output parameters from exceeding the actuator's allowable range or causing control anomalies, a safety constraint module is set to limit the updated control parameters: (10) In the formula, This is the control parameter vector after safety constraint correction. This is the lower limit of the allowable range for the control parameter. This limits the upper limit of the allowable range for control parameters. This limiting process ensures that the reinforcement learning output parameters are always within the acceptable and safe range for the engine, motor, clutch, and transmission system.

[0041] Furthermore, when the reinforcement learning agent outputs abnormally, parameters exceed safety boundaries, or the control system detects an excessive rate of torque change, the control system will automatically switch to preset basic control parameters to ensure system reliability. (11) The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural transformations made using the contents of the present invention's specification and drawings under the inventive concept of the present invention, or direct / indirect applications in other related technical fields, are included within the patent protection scope of the present invention.

Claims

1. A reinforcement learning self-tuning method for torsional vibration suppression control parameters in hybrid power systems, characterized in that, Includes the following steps: S1: Establish a dynamic model for the mode switching of the hybrid power system, model the longitudinal dynamics of the engine, motor, clutch, drive shaft and the whole vehicle, and analyze the coupling relationship of torque changes during pure electric drive, engine intervention, hybrid drive and engine withdrawal. S2: Based on the torsional vibration response of the transmission system during mode switching, a basic torsional vibration suppression controller is designed. The basic torsional vibration suppression controller includes a torque coordination module, a power source torque compensation module, and a motor active damping module. The motor active damping coefficient, power source torque compensation coefficient, torque transition time, and torque change rate limit are used as adjustable control parameters. S3: Construct a reinforcement learning agent, taking the mode switching state, power source operating state and transmission system torsional vibration response as state inputs, taking the correction amount of the adjustable control parameters as action outputs, and designing a reward function with the goal of reducing transmission system torsional vibration, reducing longitudinal impact and maintaining smooth control input. S4: Train the reinforcement learning agent under multiple mode switching conditions to obtain adaptive tuning strategies for control parameters under different mode switching states; S5: The trained reinforcement learning agent is used for the mode switching control of the hybrid power system. Based on the current mode switching conditions and the torsional vibration response of the transmission system, the adjustable control parameters of the basic torsional vibration suppression controller are corrected online, and the output parameters are limited and corrected through the safety constraint module.

2. The method for reinforcement learning self-tuning of torsional vibration suppression control parameters for mode switching in a hybrid power system according to claim 1, characterized in that, The hybrid power system mode switching dynamic model in step S1 is established using the lumped parameter method. The engine, motor, clutch driving end, clutch driven end, drive shaft and wheels are equivalent to multiple lumped inertial elements, and equivalent stiffness and damping parameters are introduced. After obtaining the angular displacement and angular velocity of each inertial element by solving the dynamic equation, the angular displacement components of the input and output ends of the drive shaft are selected to characterize the torsional deformation of the transmission system, and then the torsional angle and torsional angular velocity of the transmission system are defined.

3. The method for reinforcement learning self-tuning of torsional vibration suppression control parameters for mode switching in a hybrid power system according to claim 1, characterized in that, The torque coordination module in step S2 is used to generate the basic target torque of the engine and motor during the mode switching process based on the target torque of the engine and motor before and after the mode switching and the torque transition time.

4. The method for reinforcement learning self-tuning of torsional vibration suppression control parameters for mode switching in a hybrid power system according to claim 1, characterized in that, The power source torque compensation module in step S2 calculates the compensation amount based on the power source torque compensation coefficient and the engine target torque, which is used to compensate for the drive torque deviation caused by the engine torque build-up lag.

5. The method for reinforcement learning self-tuning of torsional vibration suppression control parameters for mode switching in a hybrid power system according to claim 1, characterized in that, In step S2, the motor active damping module generates a vibration suppression compensation torque based on the motor active damping coefficient and the torsional vibration response of the transmission system, which is used to suppress the torsional vibration of the transmission system.

6. The method for reinforcement learning self-tuning of torsional vibration suppression control parameters for mode switching in a hybrid power system according to claim 1, characterized in that, In step S2, the output torque of the basic torsional vibration suppression controller consists of the engine's basic target torque and the motor's control torque is obtained by superimposing the motor's basic target torque, the power source torque compensation, and the motor's active damping compensation torque.

7. The method for reinforcement learning self-tuning of torsional vibration suppression control parameters for mode switching in a hybrid power system according to claim 1, characterized in that, The state variables in step S3 include mode switching state, engine speed, motor speed, vehicle required torque, vehicle speed, battery SOC, transmission system torsion angle, torsion angular velocity, and vehicle longitudinal acceleration. The reinforcement learning agent outputs action variables, namely control parameter correction vectors, based on the current state variables through its policy function. These vectors include corrections for motor active damping coefficient, power source torque compensation coefficient, torque transition time, and torque change rate limit.

8. The method for reinforcement learning self-tuning of torsional vibration suppression control parameters for mode switching in a hybrid power system according to claim 1, characterized in that, The reward function in step S3 includes a transmission system torsional vibration penalty term, a longitudinal impact penalty term, a torque tracking error penalty term, and a control input change penalty term. Each term is multiplied by its corresponding weight coefficient, summed, and a constant term is added to form the final reward value.

9. The method for reinforcement learning self-tuning of torsional vibration suppression control parameters for mode switching in a hybrid power system according to claim 1, characterized in that, In step S5, the preset basic control parameter vector is added to the control parameter correction vector output by the reinforcement learning agent to obtain the tuned control parameter vector. This vector includes the motor active damping coefficient, the power source torque compensation coefficient, the torque transition time and the torque change rate limit value. The safety constraint module performs amplitude limiting processing on the updated control parameters so that the final control parameters are always between the preset lower limit and the upper limit of the control parameter allowable range. When the reinforcement learning agent outputs abnormally, the parameters exceed the safety boundary, or the control system detects that the torque change rate exceeds the limit, the control system automatically switches to the preset basic control parameters.

Citation Information

Patent Citations

  • Engine starting and transmission gear shifting coordinated control method and device and hybrid electric vehicle

    CN113602254A

  • Engine torque control system for mode switching of hybrid electric vehicle

    CN114771498A