Occupant Motion Sickness Prevention Seat Control Method Based on Maximum Entropy Reinforcement Learning

Abstract

This invention discloses a method for controlling occupant motion sickness prevention seats based on maximum entropy reinforcement learning, comprising: building a preliminary control model for occupant motion sickness prevention seats based on maximum entropy reinforcement learning; and determining the state variables s of the preliminary model. t Determine the motion quantity 'a' of the preliminary model. t ; Determine the update strategies for the initial model's policy network, Q-network, target Q-network, and entropy coefficients; Set the reward function for the initial model; Optimize the initial model based on the update strategy and reward function, and set an end flag d; Real-time acquisition of state variables as input to the optimized model, and based on the target stiffness value k output by the model. t and target damping value c t The excitation current of the magnetorheological damper is controlled. Occupant motion sickness is prevented by real-time control of the seat suspension system.

Description

Technical Field

[0001] This invention belongs to the field of intelligent vehicle cockpits, specifically relating to a method for controlling occupant motion sickness prevention seats based on maximum entropy reinforcement learning. Background Technology

[0002] With the rapid development of vehicle intelligence, automated driving technology, and artificial intelligence, the in-vehicle experience is constantly changing. As passengers spend more time on activities unrelated to driving, such as reading, working, or entertainment, the mismatch between vestibular and visual information increases, leading to greater conflict in the brain and thus exacerbating motion sickness.

[0003] Traditional vehicle seating systems are primarily designed for static comfort and crash safety, with their dynamic characteristics being fixed or passively adjusted, unable to adapt to real-time vehicle motion, road conditions, and occupant status. Therefore, they cannot actively adjust the damping of the seat suspension system to help occupants counteract changes in external acceleration, nor can they change the stiffness of the seat suspension system to isolate or absorb vibrations at specific frequencies that cause motion sickness.

[0004] Due to the increasing demand for comfort, semi-active / active seat suspension systems have emerged. For example, Chinese patent application CN202411050007.4, "Anti-motion sickness control method, device, anti-motion sickness damping system and electronic equipment," proposes a method based on a preset proportional-integral-derivative (PID) algorithm to control the working state of an electromagnetic damper, minimizing the acceleration changes felt by passengers and achieving the purpose of preventing motion sickness. However, this method has poor robustness, poor adaptability to changing conditions, and limited ability to handle nonlinearity. Chinese patent application CN202411832056.3, "Anti-motion sickness semi-active seat with damped filter control method," proposes a weighted sliding mode control system to suppress motion sickness frequencies. It precisely adjusts the damping coefficient of the magnetorheological damper through a control law to improve riding comfort and reduce motion sickness. However, this algorithm is highly dependent on an accurate model and is difficult to implement. Chinese patent application CN201811097813.1, "An embedded control system and method for alleviating motion sickness in a seat", proposes a particle swarm optimization algorithm that aims to minimize the vibration acceleration of the human body and optimizes the control by adjusting the damping and stiffness of the seat suspension; however, this algorithm is essentially an offline optimization and cannot be used for real-time control. Summary of the Invention

[0005] This invention discloses a method for controlling occupant motion sickness prevention seats based on maximum entropy reinforcement learning, including:

[0006] Step (1): Build a preliminary model for occupant motion sickness prevention seat control based on maximum entropy reinforcement learning;

[0007] Step (2): Determine the input state variables s for the preliminary model of the occupant motion sickness protection seat control. t This includes vehicle driving information and seat movement information;

[0008] Vehicle driving information includes vehicle three-axis acceleration, vehicle three-axis angular velocity, vehicle three-axis speed, accelerator pedal opening, brake master cylinder pressure, and steering wheel angle. Vehicle three-axis acceleration and vehicle three-axis angular velocity are collected through inertial measurement unit, vehicle three-axis speed is collected through inertial navigation system, and accelerator pedal opening, brake master cylinder pressure, and steering wheel angle are collected through vehicle controller local area network bus.

[0009] The seat motion information includes the seat's three-axis acceleration, seat's three-axis velocity, and seat suspension's three-axis travel. The seat's three-axis acceleration is collected by an acceleration sensor, the seat's three-axis velocity is obtained by integrating the seat's three-axis acceleration using an integrator, and the seat suspension's three-axis travel is collected by a cable-operated sensor.

[0010] State quantity s t for:

[0011] s t =[a x , a y , a z , ω x , ω y , ω z , v x , v y , v z , P throttle , P break , θ steer , a sx , a sy, a sz , v sx , v sy , v sz , d x , d y , d z ];

[0012] In the formula, a x , a y , a z For the vehicle's three-axis acceleration; ω x , ω y , ω z v is the angular velocity of the vehicle's three axles. x , v y , v z P represents the speed of the vehicle's three axes. throttle P is the accelerator pedal opening.break The pressure of the master cylinder; θ steer The steering wheel angle; a sx ,a sy, a sz The three-axis acceleration of the seat; v sx , v sy , v sz For the three-axis velocity of the seat; d x , d y , d z This refers to the three-axis travel of the seat suspension. x , a y , a z , ω x , ω y , ω z Associated with the six degrees of freedom in the six-degree-of-freedom subjective vertical conflict model, it reflects the incidence of occupant motion sickness and the dynamic information of the vehicle; v x , v y , v z This reflects the vehicle's operating condition; P throttle ,P break , θ steer This reflects the driver's intention; a sx , a sy, a sz , v sx , v sy , v sz , d x , d y , d z It reflects the dynamic information of the seating system itself.

[0013] Step (3): Determine the relationship between the seat suspension stiffness value k and the seat suspension damping value c and the motion sickness dose value (MSDV), and set the target stiffness value k. t Standard damping value c t The motion quantity 'a' output from the preliminary model of the occupant motion sickness protection seat control t ;

[0014] Step (31): Construct a single-degree-of-freedom seat suspension dynamic model:

[0015] (1),

[0016] In the formula, m is the mass of the seat cushion and the occupant; t is time, expressed as time t; x(t) is the absolute displacement of the seat cushion and the occupant at time t; Let t be the derivative of the absolute displacement of the seat cushion and the occupant with respect to time, that is, the absolute velocity of the seat cushion and the occupant at time t. Let y(t) be the second derivative of the absolute displacement of the seat cushion and the occupant with respect to time at time t, that is, the absolute acceleration of the seat cushion and the occupant at time t; y(t) is the absolute displacement of the floor at time t. Let t be the derivative of the absolute displacement of the floor with respect to time, i.e., the absolute velocity of the floor at time t.

[0017] Step (32): Perform a Fourier transform on the time-domain signals of the absolute displacement x(t) of the seat cushion and occupant and the absolute displacement y(t) of the floor to convert them into frequency-domain signals, i.e.:

[0018] (2),

[0019] In the formula, ω is the frequency, and the input of frequency ω is fixed under the same working conditions; X is the amplitude of the absolute displacement x(t) of the seat cushion and the occupant at frequency ω; Y is the amplitude of the absolute displacement y(t) of the floor at frequency ω; and i is a complex number.

[0020] Step (33): Substitute the frequency domain signal obtained in step (32) into the single-degree-of-freedom seat suspension dynamics model built in step (31) to obtain:

[0021] (3),

[0022] Step (34): Calculate the transfer function amplitude of the single-degree-of-freedom seat suspension dynamics model using equation (3). :

[0023] (4),

[0024] In the formula, a s (ω) represents the instantaneous acceleration in the frequency domain of the seat cushion and the occupant; a f (ω) represents the instantaneous acceleration of the floor in the frequency domain.

[0025] Step (35): Calculate the instantaneous acceleration a in the frequency domain of the seat cushion and the occupant using equation (4). s (ω):

[0026] (5),

[0027] Step (36): Calculate the frequency-weighted instantaneous acceleration a of the seat cushion and the occupant in the frequency domain. w (ω):

[0028] (6),

[0029] In the formula, W f (ω) is the frequency weighting function.

[0030] Step (37): The instantaneous acceleration a after frequency weighting in the frequency domain w (ω) is converted into the instantaneous acceleration a in the time domain by inverse Fourier transform after frequency weighting. w (t):

[0031] (7),

[0032] In the formula, This is the inverse Fourier transform.

[0033] Step (38): Calculate the motion sickness dose value (MSDV):

[0034] (8),

[0035] In the formula, T is the total time under vibration.

[0036] Step (39): Obtain the relationship between the seat suspension stiffness value k and the seat suspension damping value c and the motion sickness dose value MSDV using equation (8):

[0037] (9),

[0038] In the formula, This is a Fourier transform.

[0039] Step (310): Determine the motion quantity a output by the preliminary model of the occupant motion sickness protection seat control using equation (9). t :

[0040] (10)

[0041] In the formula, k t c is the target stiffness value calculated by the algorithm at time t; t The target damping value is calculated by the algorithm at time t.

[0042] Step (4): Determine the policy network, Q network, target Q network, and entropy coefficient update strategy for the preliminary model of occupant motion sickness protection seat control;

[0043] Step (5): Set the reward function r(s) of the preliminary model for occupant motion sickness protection seat control. t ,a t ):

[0044] (11)

[0045] in, (12)

[0046] In the formula, ω1, ω2, ω3, ω4, ω5 are the weight coefficients of each item; C sicknessC is the motion sickness assessment cost function, used to penalize actions that easily cause motion sickness in occupants. acceleration C is the acceleration mutation cost function, used to penalize actions that easily cause passengers to feel impact and jerking. smooth C is the smoothness cost function, used to evaluate the smoothness of the control itself; accuracy C is the accuracy cost function, used to evaluate the execution accuracy of control; energy α is the energy cost function, used to evaluate the economics of control methods. x , α y , α z , β x , β y , β z , α k , α c , β k , β c , κ k , κ c ΔMSDV represents the internal weighting coefficients for each item. x (t), ΔMSDV y (t), ΔMSDV z (t) represents the changes in longitudinal, transverse, and vertical motion dose values ​​at time t; Δa x (t), Δa y (t), Δa z (t) represents the change in the three-axis acceleration of the seat at time t; Δk t Δc is the stiffness adjustment at time t. t k is the damping adjustment amount at time t; a c represents the actual stiffness value of the seat suspension system at time t; a k represents the actual damping value of the seat suspension system at time t. p c is the stiffness value under passive conditions. p This represents the damping value under passive conditions.

[0047] Step (6): Based on the update strategy of step (4) and the reward function r(s) set in step (5) t ,a t ), and collect the state variable s t The data was input into the preliminary control model for the occupant motion sickness protection seat for optimization, and an end flag d was set.

[0048] When the number of model updates reaches the set number of learning steps, the end flag d1 is output, the round ends, and the optimized occupant motion sickness prevention seat control model based on maximum entropy reinforcement learning is obtained.

[0049] If the seat suspension travel exceeds its maximum travel or the motion sickness dose exceeds the set value, output the end flag d2, the round ends, and optimization is performed again.

[0050] Step (7): Real-time acquisition of the vehicle's state variables during motion is used as input to the optimized occupant motion sickness prevention seat control model based on maximum entropy reinforcement learning. The target stiffness value k output by the model is then used as the basis for further analysis. t and target damping value c t The excitation current of the magnetorheological damper is controlled.

[0051] The beneficial effects of this invention are:

[0052] (1) In the occupant motion sickness protection seat control method based on maximum entropy reinforcement learning proposed in this invention, the real-time update of the policy network can cope with changing working conditions and has strong robustness and smoothness.

[0053] (2) The occupant motion sickness protection seat control method based on maximum entropy reinforcement learning in this invention application gets rid of the dependence on precise complex dynamic models. It is a model-free reinforcement learning method. Through training, it can achieve direct mapping from multi-dimensional state variables to optimal motion control.

[0054] (3) In the maximum entropy reinforcement learning-based occupant motion sickness protection seat control method of this invention, a multi-objective composite reward function is set up that considers motion sickness protection, impact smoothness, control smoothness, control accuracy and energy consumption economy. Through training, multi-objective coordinated control can be performed to make the most reasonable motion output. Attached Figure Description

[0055] Figure 1 This is a schematic diagram of the occupant motion sickness prevention seat control method based on maximum entropy reinforcement learning of the present invention;

[0056] Figure 2 This is a diagram of the single-degree-of-freedom seat suspension dynamics model of the present invention;

[0057] Figure 3 This is a schematic diagram of the training platform according to an embodiment of the present invention;

[0058] Figure 4 This is a schematic diagram of the implementation platform of an embodiment of the present invention;

[0059] Figure 5 This is a schematic diagram comparing the stiffness of the seat suspension system in the embodiments and comparative examples of the present invention;

[0060] Figure 6 This is a schematic diagram comparing the damping of the seat suspension system in the embodiments and comparative examples of the present invention;

[0061] Figure 7This is a schematic diagram comparing the vertical acceleration of the seats in the embodiments and comparative examples of the present invention;

[0062] Figure 8 This is a schematic diagram comparing the vertical motion sickness dose values ​​of the embodiments and comparative examples of the present invention. Detailed Implementation

[0063] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0064] The maximum entropy reinforcement learning-based occupant motion sickness prevention seat control method aims to adjust the damping and stiffness of the seat system, reduce the motion sickness dose value (MSDV) felt by the occupant, and improve occupant comfort.

[0065] The occupant motion sickness prevention seat control method based on maximum entropy reinforcement learning implemented in this example is as follows: Figure 1 As shown, it includes:

[0066] Step (1): Build a preliminary model for occupant motion sickness prevention seat control based on maximum entropy reinforcement learning, including a policy network π ϕ Two Q networks and Two target Q-networks and Entropy coefficient α and experience pool ;

[0067] Step (2): Determine the input state variables s for the preliminary model of the occupant motion sickness protection seat control. t This includes vehicle driving information and seat movement information;

[0068] Vehicle driving information includes vehicle three-axis acceleration, vehicle three-axis angular velocity, vehicle three-axis speed, accelerator pedal opening, brake master cylinder pressure, and steering wheel angle. Vehicle three-axis acceleration and vehicle three-axis angular velocity are collected through inertial measurement unit, vehicle three-axis speed is collected through inertial navigation system, and accelerator pedal opening, brake master cylinder pressure, and steering wheel angle are collected through vehicle controller local area network bus.

[0069] The seat motion information includes the seat's three-axis acceleration, seat's three-axis velocity, and seat suspension's three-axis travel. The seat's three-axis acceleration is collected by an acceleration sensor, the seat's three-axis velocity is obtained by integrating the seat's three-axis acceleration using an integrator, and the seat suspension's three-axis travel is collected by a cable-operated sensor.

[0070] State quantity s t for:

[0071] s t =[a x , a y , a z , ωx , ω y , ω z , v x , v y , v z , P throttle , P break , θ steer , a sx , a sy, a sz , v sx , v sy , v sz , d x , d y , d z ];

[0072] In the formula, a x , a y , a z For the vehicle's three-axis acceleration; ω x , ω y , ω z v is the angular velocity of the vehicle's three axles. x , v y , v z P represents the speed of the vehicle's three axes. throttle P is the accelerator pedal opening. break The pressure of the master cylinder; θ steer The steering wheel angle; a sx ,a sy, a sz The three-axis acceleration of the seat; v sx , v sy , v sz For the three-axis velocity of the seat; d x , d y , d z This refers to the three-axis travel of the seat suspension. x , a y , a z , ω x , ω y , ω z Associated with the six degrees of freedom in the six-degree-of-freedom subjective vertical conflict model, it reflects the incidence of occupant motion sickness and the dynamic information of the vehicle; v x , v y , v z This reflects the vehicle's operating condition; P throttle ,P break , θ steer This reflects the driver's intention; a sx , a sy, a sz, v sx , v sy , v sz , d x , d y , d z It reflects the dynamic information of the seating system itself.

[0073] Step (3): Determine the relationship between the seat suspension stiffness value k and the seat suspension damping value c and the motion sickness dose value (MSDV), and set the target stiffness value k. t Standard damping value c t The motion quantity 'a' output from the preliminary model of the occupant motion sickness protection seat control t ;

[0074] Step (31): Build a single-degree-of-freedom seat suspension dynamic model as follows Figure 2 As shown:

[0075] (1),

[0076] In the formula, m is the mass of the seat cushion and the occupant; t is time, expressed as time t; x(t) is the absolute displacement of the seat cushion and the occupant at time t; Let t be the derivative of the absolute displacement of the seat cushion and the occupant with respect to time, that is, the absolute velocity of the seat cushion and the occupant at time t. Let y(t) be the second derivative of the absolute displacement of the seat cushion and the occupant with respect to time at time t, that is, the absolute acceleration of the seat cushion and the occupant at time t; y(t) is the absolute displacement of the floor at time t. Let t be the derivative of the absolute displacement of the floor with respect to time, i.e., the absolute velocity of the floor at time t.

[0077] Step (32): Perform a Fourier transform on the time-domain signals of the absolute displacement x(t) of the seat cushion and occupant and the absolute displacement y(t) of the floor to convert them into frequency-domain signals, i.e.:

[0078] (2),

[0079] In the formula, ω is the frequency, and the input of frequency ω is fixed under the same working conditions; X is the amplitude of the absolute displacement x(t) of the seat cushion and the occupant at frequency ω; Y is the amplitude of the absolute displacement y(t) of the floor at frequency ω; and i is a complex number.

[0080] Step (33): Substitute the frequency domain signal obtained in step (32) into the single-degree-of-freedom seat suspension dynamics model built in step (31) to obtain:

[0081] (3),

[0082] Step (34): Calculate the transfer function amplitude of the single-degree-of-freedom seat suspension dynamics model using equation (3). :

[0083] (4),

[0084] In the formula, a s (ω) represents the instantaneous acceleration in the frequency domain of the seat cushion and the occupant; a f (ω) represents the instantaneous acceleration of the floor in the frequency domain.

[0085] Step (35): Calculate the instantaneous acceleration a in the frequency domain of the seat cushion and the occupant using equation (4). s (ω):

[0086] (5),

[0087] Step (36): Calculate the frequency-weighted instantaneous acceleration a of the seat cushion and the occupant in the frequency domain. w (ω):

[0088] (6),

[0089] In the formula, W f (ω) is the frequency weighting function.

[0090] Step (37): The instantaneous acceleration a after frequency weighting in the frequency domain w (ω) is converted into the instantaneous acceleration a in the time domain by inverse Fourier transform after frequency weighting. w (t):

[0091] (7),

[0092] In the formula, This is the inverse Fourier transform.

[0093] Step (38): Calculate the motion sickness dose value (MSDV):

[0094] (8),

[0095] In the formula, T is the total time under vibration.

[0096] Step (39): Obtain the relationship between the seat suspension stiffness value k and the seat suspension damping value c and the motion sickness dose value MSDV using equation (8):

[0097] (9),

[0098] In the formula, This is a Fourier transform.

[0099] Step (310): Determine the motion quantity a output by the preliminary model of the occupant motion sickness protection seat control using equation (9). t :

[0100] (10)

[0101] In the formula, k t c is the target stiffness value calculated by the algorithm at time t; t The target damping value is calculated by the algorithm at time t.

[0102] Step (4): Determine the policy network, Q network, target Q network, and entropy coefficient update strategy for the preliminary model of occupant motion sickness protection seat control;

[0103] Step (5): Set the reward function r(s) of the preliminary model for occupant motion sickness protection seat control. t ,a t ):

[0104] (11),

[0105] in, (12)

[0106] In the formula, ω1, ω2, ω3, ω4, ω5 are the weight coefficients of each item; C sickness C is the motion sickness assessment cost function, used to penalize actions that easily cause motion sickness in occupants. acceleration C is the acceleration mutation cost function, used to penalize actions that easily cause passengers to feel impact and jerking. smooth C is the smoothness cost function, used to evaluate the smoothness of the control itself; accuracy C is the accuracy cost function, used to evaluate the execution accuracy of control; energy α is the energy cost function, used to evaluate the economics of control methods. x , α y , α z , β x , β y , β z , α k , α c , β k , β c , κ k , κ c ΔMSDV represents the internal weighting coefficients for each item. x (t), ΔMSDV y (t), ΔMSDV z (t) represents the changes in longitudinal, transverse, and vertical motion dose values ​​at time t; Δa x (t), Δay (t), Δa z (t) represents the change in the three-axis acceleration of the seat at time t; Δk t Δc is the stiffness adjustment at time t. t k is the damping adjustment amount at time t; a c represents the actual stiffness value of the seat suspension system at time t; a k represents the actual damping value of the seat suspension system at time t. p c is the stiffness value under passive conditions. p This represents the damping value under passive conditions.

[0107] Step (6): Based on the update strategy of step (4) and the reward function r(s) set in step (5) t ,a t ), and collect the state variable s t The data was input into the preliminary control model for the occupant motion sickness protection seat for optimization, and an end flag d was set.

[0108] When the model has been updated and iterated a certain number of times, the end flag d1 is output, the round ends, and the optimized occupant motion sickness prevention seat control model based on maximum entropy reinforcement learning is obtained. During the optimization process, the state variable s at time t... t Action volume a t The state quantity s at the next moment t+1 The reward value r at time t is calculated using the reward function. t And the end marker d is stored in the experience pool. The optimized model can maximize the cumulative reward value J(π) with entropy.

[0109]

[0110] In the formula, π represents the policy selected by the policy network; This is the expected value; Let π be the marginal distribution of state and action quantities generated by policy π in environmental interactions; α is the entropy coefficient, used to balance reward and entropy. For strategy π in state variable s t The entropy function.

[0111] If the seat suspension travel exceeds its maximum travel or the motion sickness dose exceeds the set value, output the end flag d2, the round ends, and optimization is performed again.

[0112] Step (7): Real-time acquisition of the vehicle's state variables during motion is used as input to the optimized occupant motion sickness prevention seat control model based on maximum entropy reinforcement learning. The target stiffness value k output by the model is then used as the basis for further analysis. t and target damping value c tThe excitation current of the magnetorheological damper is controlled.

[0113] Example

[0114] This embodiment describes a scenario where a vehicle is traveling at 70 km / h on a straight road. The driver notices a bump on the road and brakes to reduce the speed to 40 km / h to pass over a 20-meter-long, 4-meter-high arc-shaped bump. The duration of this scenario is 10 seconds, with the vehicle starting to brake at 2.2 seconds, starting to drive onto the bump at 5.3 seconds, passing the top of the bump at 6.7 seconds, and leaving the bump at 8.2 seconds.

[0115] The schematic diagrams of the training platform and implementation platform of this invention are as follows: Figures 3-4 As shown in Table 1, the relevant hyperparameters are used during the training process.

[0116] Table 1. Preliminary Model Training Hyperparameter Table for Examples

[0117] parameter value Experience pool capacity 100000 Batch size 256 Learning Steps 1000 Learning rate 0.0003 Discount factor 0.99 Soft update factor 0.005 Optimizer Adam

[0118] After implementation, the embodiments of the present invention, since the vertical motion sickness dose value has the greatest impact on the motion sickness of the occupants, output curves of the real-time stiffness and damping values ​​of the seat suspension system, the vertical seat acceleration and the vertical motion sickness dose value.

[0119] Comparative Example

[0120] The operating conditions are consistent with the example, and a PID algorithm is used to regulate the stiffness and damping values ​​of the seat suspension system. The PID parameters for this comparative example are determined by an automatic tuning method, and the relevant hyperparameters are shown in Table 2.

[0121] Table 2 Hyperparameter Table for Automatic Tuning of Comparative PID Parameters

[0122] parameter value Stiffness amplitude (N / m) 50 Damped amplitude (Ns / m) 5 Adjust the number of steps 1500 Transient neglect steps 500 Minimum number of peaks required 4 Setting rules Tyreus-Luyben

[0123] The PID parameters obtained after automatic tuning are shown in Table 3.

[0124] Table 3 Comparative PID Parameter Table

[0125] parameter value Stiffness proportionality factor 75875.27 Stiffness integral coefficient 38664.53 Stiffness differential coefficients 10742.97 Damping proportional coefficient 198.03 Damping integral coefficient 120.65 Damping differential coefficient 23.45

[0126] This comparative model, after being controlled by a PID controller, outputs curves of real-time stiffness and damping values ​​of the seat suspension system, vertical seat acceleration, and vertical motion sickness dose value, since the vertical motion sickness dose value has the greatest impact on occupant motion sickness.

[0127] A comparison of the real-time stiffness and damping values, vertical seat acceleration, and vertical motion sickness dose values ​​of the seat suspension systems in the embodiments and comparative examples is shown in the figure. Figures 5-8 As shown.

[0128] in, Figure 5 and Figure 6 The charts show a comparison of the real-time stiffness and damping values ​​of the seat suspension system. They illustrate the control strategies for the stiffness and damping values ​​of the seat suspension system in both the embodiment and the comparative example. The embodiment can predict the driver's intentions and coordinate the control of the stiffness and damping values ​​of the seat suspension system from a long-term perspective, while the comparative example adjusts the stiffness and damping values ​​of the seat suspension system to deal with short-term emergencies. Figure 7 The graph shows a comparison of seat vertical acceleration. The embodiment exhibits a lower seat vertical acceleration compared to the comparative example. The maximum seat vertical acceleration values ​​for the embodiment are 0.47 m / s² at 2.3 seconds and 7.1 seconds, respectively. 2 and 2.31m / s 2 The comparative examples are 0.76 m / s 2 and 3.33m / s 2 The maximum seat vertical acceleration of the embodiment was approximately 38.15% and 30.63% lower than that of the comparative example. Figure 8 The chart compares the vertical motion sickness dose values ​​experienced by the occupant after frequency-weighted calculation of the seat vertical acceleration obtained in the embodiment and the comparative example. The vertical motion sickness dose value generated by the embodiment for the occupant under this condition is 0.62 m / s². 1.5 The comparative example showed a vertical motion sickness dose of 0.44 m / s for the occupants under this condition. 1.5 The vertical motion sickness dose value of the example was about 29.03% lower than that of the comparative example.

Claims

1. A method for controlling occupant motion sickness prevention seats based on maximum entropy reinforcement learning, characterized in that, include: Step (1): Build a preliminary model for occupant motion sickness prevention seat control based on maximum entropy reinforcement learning; Step (2): Determine the input state variables s for the preliminary model of the occupant motion sickness protection seat control. t This includes vehicle driving information and seat movement information; Step (3): Determine the relationship between the seat suspension stiffness value k and the seat suspension damping value c and the motion sickness dose value MSDV, and set the target stiffness value k. t and target damping value c t The motion quantity 'a' output from the preliminary model of the occupant motion sickness protection seat control t ; Step (4): Determine the policy network, Q network, target Q network, and entropy coefficient update strategy for the preliminary model of occupant motion sickness protection seat control; Step (5): Set the reward function r(s) of the preliminary model for occupant motion sickness protection seat control. t ,a t ); Step (6): Based on the update strategy of step (4) and the reward function r(s) set in step (5) t ,a t ), and collect the state variable s t The data was input into the preliminary control model for the occupant motion sickness protection seat for optimization, and an end flag d was set. Step (7): Real-time acquisition of the state variables s during the vehicle's motion. t As input to the optimized occupant motion sickness prevention seat control model based on maximum entropy reinforcement learning, the target stiffness value k output by the model is used. t and target damping value c t Control the excitation current of the magnetorheological damper; The relationship between the seat suspension stiffness value k, the seat suspension damping value c, and the motion sickness dose value MSDV is determined as follows: ， In the formula, For Fourier transform; MSDV is the motion sickness dose value; t is time, i.e., time t; ω is the frequency, and the input frequency ω is fixed under the same operating conditions; W f (ω) is the frequency weighting function; a f (ω) represents the instantaneous acceleration of the vehicle floor in the frequency domain; k represents the seat suspension stiffness value; c represents the seat suspension damping value; and m represents the mass of the seat cushion and the occupant. Reward function r(s) t ,a t ) is set to: ， in, , In the formula, ω1, ω2, ω3, ω4, ω5 are the weight coefficients of each item; C sickness C is the cost function for motion sickness assessment; acceleration C is the acceleration mutation cost function; smooth C is the smoothness cost function; accuracy For the accuracy cost function; C energy Let α be the energy cost function; x , α y , α z , β x , β y , β z , α k , α c , β k , β c , κ k , κ c ΔMSDV represents the internal weighting coefficients for each item. x (t), ΔMSDV y (t), ΔMSDV z (t) represents the changes in longitudinal, transverse, and vertical motion dose values ​​at time t; Δa x (t), Δa y (t), Δa z (t) represents the change in the three-axis acceleration of the seat at time t; Δk t Δc is the stiffness adjustment at time t. t k is the damping adjustment amount at time t; a c represents the actual stiffness value of the seat suspension system at time t; a k represents the actual damping value of the seat suspension system at time t. p c represents the stiffness value of the seat suspension system under passive conditions. p This represents the damping value of the seat suspension system in a passive state.

2. The method as described in claim 1, characterized in that, State quantity s t Set to: s t =[a x , a y , a z , ω x , ω y , ω z , v x , v y , v z , P throttle , P break , θ steer , a sx , a sy, a sz ,v sx , v sy , v sz , d x , d y , d z ]; In the formula, a x , a y , a z For the vehicle's three-axis acceleration; ω x , ω y , ω z v is the angular velocity of the vehicle's three axles. x , v y , v z P represents the speed of the vehicle's three axes. throttle P represents the accelerator pedal opening. break The pressure of the master cylinder; θ steer The steering wheel angle; a sx , a sy, a sz The three-axis acceleration of the seat; v sx , v sy , v sz For the three-axis velocity of the seat; d x , d y , d z The seat suspension has a three-axis travel.

3. The method as described in claim 2, characterized in that, Motion sickness dose value (MSDV) is: ， In the formula, T is the total time under vibration, and a w (t) represents the instantaneous acceleration in the time domain after frequency weighting.

4. The method as described in claim 3, characterized in that, Determine the amount of motion a t for: ； In the formula, k t c is the target stiffness value calculated by the algorithm at time t; t The target damping value is calculated by the algorithm at time t.

5. The method as described in claim 4, characterized in that, The end marker d is set to: When the number of model update iterations reaches the set number of learning steps, the end flag d1 is output, the round ends, and the optimized occupant motion sickness prevention seat control model based on maximum entropy reinforcement learning is obtained. If the seat suspension travel exceeds its maximum travel or the motion sickness dose exceeds the set value, output the end flag d2, the round ends, and optimization is performed again.

Images