Cooperative reinforcement learning control method for riding comfort and operation safety of automobile

Abstract

The invention provides an automobile riding comfort and operation safety collaborative reinforcement learning control method. Belongs to the technical field of automobile vibration control and comprises the steps that # imgabs0 # is defined as a comfort index, St2 = [delta Z1, delta Z2 and delta Z3] is defined as a safety index, and St = (1-gamma) St1 + gamma St2 is used as a state quantity; ideal control force At = [ufl, ufr, url, urr] T provided by four vertical magneto-rheological dampers acting on an automobile body serves as an action quantity, and an 11-degree-of-freedom human-automobile ring coupling vibration double-delay depth deterministic strategy gradient reinforcement learning TD3 controller is established; the TD3 controller calculates ideal control force At = [ufl, ufr, url, urr] T in real time according to the state quantity St to act on the current controller, the current controller outputs current I to act on the vertical magneto-rheological damper, the vertical magneto-rheological damper provides actual control force Fa = [ufla, ufraa, urla, urr] T to act on the automobile, and the riding comfort and operation safety of the automobile are controlled. The riding comfort and the operation safety of the automobile can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of vehicle vibration control, and more specifically, to a collaborative reinforcement learning control method for vehicle ride comfort and operation safety. Background Art

[0002] Vehicles have become an important means of transportation for people's daily travel. With the rapid growth of vehicle usage, improving vehicle ride comfort and operation safety has become increasingly important.

[0003] In the prior art, in order to improve vehicle ride comfort and operation safety, the mainstream method is to use a quarter-vehicle model (single wheel) to carry out vehicle ride comfort and operation safety control methods. Such methods ignore the coupled vibration between the various components of the vehicle, and the actual application effect is not ideal. In addition, a small number of them establish a 7-degree-of-freedom vehicle whole vehicle model including the vehicle body and 4 wheels, and use this model to carry out research on the whole vehicle ride comfort and operation safety control method. This method ignores the most critical comfort index of the vehicle occupants for vehicle ride comfort.

[0004] And the control algorithms for vehicle vibration mainly include skyhook damping control, sliding mode control, PID control algorithms, etc. These algorithms all have certain defects. For example, improper design of the skyhook damping value in the skyhook damping control algorithm will deteriorate the vibration damping effect, the parameters of the PID control algorithm are difficult to determine, and the chattering problem of the sliding mode control algorithm is difficult to effectively solve. Summary of the Invention

[0005] The purpose of the present application is to provide a collaborative reinforcement learning control method for vehicle ride comfort and operation safety, which can improve the vehicle ride comfort and operation safety.

[0006] The present application is implemented as follows:

[0007] The present application provides a collaborative reinforcement learning control method for vehicle ride comfort and operation safety, including:

[0008] S101: Obtain the vertical accelerations at 4 measurement points of the front and rear seats of the vehicle The vertical accelerations at 4 vehicle body measurement points The vertical accelerations at 4 wheel measurement points The road surface unevenness information Q = [q fl , q fr , q rl , q rr ; where the subscripts fl , fr, rl ,rr respectively represent the front left, front right, rear left and rear right positions;

[0009] S102: Obtain the vehicle body center acceleration according to a preset method and the vertical acceleration at the connection between the seat and the vehicle body wherein; respectively represent the vertical acceleration, sideslip angular acceleration, and pitch angular acceleration of the vehicle body center;

[0010] S103: Integrate the acceleration twice to obtain the displacement Z s =[Z sfl , Z sfr , Z srl , Z srr , Z = [Z fl , Z fr , Z rl , Z rr , Z u =[Z ufl , Z ufr , Z url , Z urr , X c =[Z c , β c , φ c , Z cs =[Z cfl , Z cfr , Z crl , Z crr ;

[0011] S104: According to the displacements Z s , Z, Z u , X c , Z cs and the road surface roughness information Q, obtain the vertical displacement differences ΔZ1 = Z s -Z cs , ΔZ2 = Z - Z u , ΔZ3 = Z u -Q;

[0012] S105: Define as the comfort index, S t2 =[ΔZ1, ΔZ2, ΔZ3] as the safety index, and take S t =(1 - γ)S t1 +γS t2 as the state variable, where γ (0 ≤ γ ≤ 1) is the weight coefficient and is dynamically adjusted by a fuzzy controller; take the ideal control forces A t =[u fl , u fr , u rl , u rrT As the action quantity, an 11-degree-of-freedom human-vehicle-environment coupled vibration double-delay deep deterministic policy gradient reinforcement learning TD3 controller is established;

[0013] S106: The fuzzy controller adjusts the weight coefficient γ in real time according to the state quantity S t The TD3 controller calculates the ideal control force A in real time according to the state quantity S t The ideal control force A t =[u fl ,u fr ,u rl ,u rr T acts on the current controller, the current controller outputs the current I which acts on the vertical magnetorheological damper, and the vertical magnetorheological damper provides the actual control force F a =[u fla ,u fra ,u rla ,u rra T acts on the vehicle to control the ride comfort and handling safety of the vehicle.

[0014] Compared with the prior art, the present application has at least the following advantages or beneficial effects:

[0015] 1. Different from the existing quarter-car model and the vehicle whole-body model without considering the driver and passengers, the present invention establishes a "human-vehicle-environment" coupled vibration model including the driver and passengers, the vehicle and the road environment, solves the problems such as poor actual application effect of the control algorithm caused by ignoring the vehicle coupled vibration in the existing research, and improves the accuracy of the vehicle whole-body model;

[0016] 2. Different from the existing vehicle vibration control algorithms with fixed weight coefficients, the present invention dynamically adjusts the weight coefficient γ (0≤γ≤1) through the fuzzy control algorithm, constructs the fuzzy control rules based on the vehicle comfort index S t1 and the handling safety index S t2 , solves the problems such as the difficulty in effectively applying the parameter-fixed control algorithm to complex working conditions, and improves the adaptability of the algorithm;

[0017] 3. Different from the existing vehicle vibration reinforcement learning control algorithms, the present invention designs a normalized reward function R=-(1-γ)(R1+R2)-γ(R3+R4+R5) containing the weight coefficient γ (0≤γ≤1), solves the problems such as inconsistent dimensions of the reward function parameters and poor training effect of the reinforcement learning controller caused by fixed parameters, and improves the training efficiency and accuracy of the reinforcement learning controller;

[0018] ​​​4. Compared with the existing research on vehicle ride comfort or operation safety, the difference lies in that the present invention aims to simultaneously improve vehicle ride comfort and operation safety. By using a fuzzy controller to dynamically optimize the weight coefficient γ, and guiding the magnetorheological damper to provide effective control force through the optimal control force calculated by the TD3 controller to reduce the vibration of the entire vehicle, the contradiction between vehicle ride comfort and operation safety is solved, and the simultaneous improvement of vehicle ride comfort and safety can be achieved.

[0019] 5. Compared with the existing vehicle ride comfort or operation safety measurement devices, the difference lies in that the present invention only requires measurement elements such as acceleration sensors and ultrasonic ranging sensors, and has the advantages of easy operation, low cost, and high control accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application, and thus should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0021] Figure 1 It is a flowchart of a collaborative reinforcement learning control method for vehicle ride comfort and operation safety of the present application;

[0022] Figure 2 It is a front view of an 11-degree-of-freedom vehicle-human-loop coupling vibration model in a collaborative reinforcement learning control method for vehicle ride comfort and operation safety of the present application;

[0023] Figure 3 It is a left view of an 11-degree-of-freedom vehicle-human-loop coupling vibration model in a collaborative reinforcement learning control method for vehicle ride comfort and operation safety of the present application;

[0024] Figure 4 It is a mounting position diagram of acceleration sensors and ultrasonic ranging sensors in a collaborative reinforcement learning control method for vehicle ride comfort and operation safety of the present application;

[0025] Figure 5 It is a block diagram of the working principle in a collaborative reinforcement learning control method for vehicle ride comfort and operation safety of the present application;

[0026] Figure 6 It is a structure diagram of the Actor network of the TD3 controller in a collaborative reinforcement learning control method for vehicle ride comfort and operation safety of the present application;

[0027] Figure 7This is the structure diagram of the Critic network of the TD3 controller in a collaborative reinforcement learning control method for vehicle ride comfort and operation safety of this application.

[0028] Reference signs:

[0029] 1. Rear seat; 2. Vehicle body; 3. Magnetorheological damper; 4. Wheel; 5. Front seat; 6. Left rear wheel acceleration sensor; 7. Left rear vehicle body acceleration sensor; 8. Right rear vehicle body acceleration sensor; 9. Right rear wheel acceleration sensor; 10. Ultrasonic ranging sensor for road unevenness at the right rear wheel; 11. Acceleration sensor for the right rear seat in the back row; 12. Acceleration sensor for the right front seat in the front row; 13. Right front vehicle body acceleration sensor; 14. Right front wheel acceleration sensor; 15. Ultrasonic ranging sensor for road unevenness at the right front wheel; 16. Ultrasonic ranging sensor for road unevenness at the left front wheel; 17. Left front wheel acceleration sensor; 18. Left front wheel acceleration sensor; 19. Acceleration sensor for the left front seat in the front row; 20. Acceleration sensor for the left rear seat in the back row; 21. Ultrasonic ranging sensor for road unevenness at the left rear wheel. Detailed implementation manners

[0030] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are some but not all of the embodiments of this application. Usually, the components of the embodiments of this application described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations.

[0031] Below, some implementation manners of this application will be described in detail with reference to the accompanying drawings. Without conflict, the following various embodiments and the various features in the embodiments can be combined with each other.

[0032] Embodiment

[0033] The embodiment of this application provides a collaborative reinforcement learning control method for vehicle ride comfort and operation safety, which can improve the ride comfort and operation safety of the vehicle.

[0034] To facilitate the understanding of the 11-degree-of-freedom vehicle-human-loop coupling vibration model, please refer to Figure 2-3 , Figure 2 which is the front view of the 11-degree-of-freedom vehicle-human-loop coupling vibration model; Figure 3It is the left view of the 11-degree-of-freedom vehicle-human-environment coupling vibration model. The vehicle-human-environment in this application refers to the driver and passengers, the vehicle, and the road environment. The 11-degree-of-freedom vehicle-human-environment coupling vibration model consists of the driver and passengers, the vehicle, and the road surface. Among them, the vehicle structure includes the rear seat 1, the front seat 5, the vehicle body 2, the magnetorheological damper 3, and the wheels 4. In the present invention, the driver and passengers and the seats they contact are regarded as a whole, and the vertical vibration of the seats is used to characterize the riding comfort of the driver and passengers. The rear seat 1, the front seat 5, and the vehicle body 2 are connected together by shock absorbers, and the vehicle body 2 and the wheels 4 are connected together by the magnetorheological damper 3. The present invention studies the vertical vibration coupling problem of the rear seat 1, the front seat 5, the vehicle body 2, and the wheels 4, and by dynamically adjusting the damping force provided by the 4 magnetorheological dampers installed between the vehicle body 2 and the wheels 4, the purpose of simultaneously improving the riding comfort and operation safety of the vehicle is achieved.

[0035] Please refer to Figure 1 and Figure 5 , Figure 1 which is the flowchart of this application; Figure 5 which is the block diagram of the working principle of this application; The collaborative reinforcement learning control method for vehicle riding comfort and operation safety includes the following steps:

[0036] S101: Obtain the vertical accelerations at 4 measurement points on the front and rear seats of the vehicle the vertical accelerations at 4 vehicle body measurement points the vertical accelerations at 4 wheel measurement points the road surface unevenness information Q = [q fl , q fr , q rl , q rr at 4 wheel measurement points; where the subscripts fl, fr, rl, rr represent the front left, front right, rear left, and rear right positions respectively;

[0037] Specifically, multiple vibration acceleration sensors are used to measure the vertical accelerations The multiple vibration acceleration sensors are respectively installed at the bottoms of the front and rear seats of the vehicle, the connection points between the vehicle body and the vertical shock absorbers, and the centers of the wheel rims; the ultrasonic ranging sensors are used to measure the road surface unevenness information Q = [q fl , q fr , q rl , q rr . Please refer to Figure 4 , Figure 4 which is the installation position diagram of the acceleration sensors and the ultrasonic ranging sensors. In this embodiment, the left rear wheel acceleration sensor 6, the right rear wheel acceleration sensor 9, the right front wheel acceleration sensor 14, and the left front wheel acceleration sensor 17 are respectively installed at the centers of the 4 wheel rims to collect the vertical accelerations at 4 wheel measurement points Install the left rear body acceleration sensor 7, the right rear body acceleration sensor 8, the right front body acceleration sensor 13, and the left front wheel acceleration sensor 18 at the joints of the four magnetorheological dampers and the four wheels respectively, for collecting the vertical accelerations at the four body measurement points. Install the right rear seat acceleration sensor 11, the right front seat acceleration sensor 12, the left front seat acceleration sensor 19, and the left rear seat acceleration sensor 20 at the centers of the lower parts of the front and rear seats respectively, for collecting the vertical accelerations at the four measurement points of the front and rear seats of the vehicle. Install the right rear wheel road roughness ultrasonic ranging sensor 10, the right front wheel road roughness ultrasonic ranging sensor 15, the left front wheel road roughness ultrasonic ranging sensor 16, and the left rear wheel road roughness ultrasonic ranging sensor 21 on the vehicle chassis 2 cm directly in front of the wheels respectively, for collecting the road roughness Q = [q fl , q fr , q rl , q rr at the four wheels.

[0038] S102: Obtain the vehicle body center acceleration and the vertical acceleration at the connection between the seat and the vehicle body where; respectively represent the vertical acceleration of the vehicle body center, the sideslip angular acceleration, and the pitch angular acceleration;

[0039] Specifically, calculate the vehicle body center acceleration according to the vehicle body structure, the vertical magnetorheological damper, and the seat installation dimensions The calculation formula is as follows:

[0040]

[0041] where, l1 and l2 respectively represent the distances from the front and rear seat measurement points to the vehicle body center along the vehicle body length direction; l3 and l4 respectively represent the distances from the centers of the front and rear vertical magnetorheological dampers of the vehicle to the vehicle body center along the vehicle body length direction; l c represents the distance from the left and right seat measurement points of the vehicle to the vehicle body center along the vehicle body width direction; l represents the distance from the centers of the left and right vertical magnetorheological dampers of the vehicle to the vehicle body center along the vehicle body width direction.

[0042] Calculate the vertical acceleration at the connection between the seat and the vehicle body according to the vehicle body structure and the seat installation dimensions The calculation formula is as follows:

[0043]

[0044] S103: For the acceleration Integrate twice to obtain the displacement Z s , Z, Z u , X c and Z cs ; Z s = [Z sfl , Z sfr , Z srl , Z srr , Z = [Z fl , Z fr , Z rl , Z rr , Z u = [Z ufl , Z ufr , Z url , Z urr , X c = [Z c , β c , φ c , Z cs = [Z cfl , Z cfr , Z crl , Z crr ;

[0045] Specifically, acceleration refers to the rate of change of an object's velocity with respect to time, with the unit of m / s 2 . Integrating the acceleration once gives the velocity, which represents the rate of change of displacement with respect to time, with the unit of m / s. Integrating the velocity once (i.e., integrating the acceleration twice) represents the change in the object's position, with the unit of m.

[0046] S104: Obtain the vertical displacement differences ΔZ1 = Z s - Z u , ΔZ2 = Z - Z c , and ΔZ3 = Z cs - Q between the vehicle body and the seat, the vehicle body and the wheels, and the wheels and the ground based on the displacement Z s , Z cs , ΔZ2 = Z - Z u , ΔZ3 = Z u - Q;

[0047] Specifically, through the comprehensive processing of displacement and road surface data, the vertical relative movement between key components of the vehicle body is quantified.

[0048] S105: Define as the comfort index, S t2 = [ΔZ1, ΔZ2, ΔZ3] as the safety index, and let S t = (1 - γ)S t1 + γS t2As a state variable, where γ (0 ≤ γ ≤ 1) is a weight coefficient and is dynamically adjusted by a fuzzy controller; the ideal control forces A provided by the 4 vertical magneto-rheological dampers acting on the vehicle body t =[u fl ,u fr ,u rl ,u rr T As an action variable, an 11-degree-of-freedom vehicle-human loop coupling vibration double-delay deep deterministic policy gradient reinforcement learning TD3 controller is established;

[0049] Specifically, the weight coefficient γ (0 ≤ γ ≤ 1) is dynamically adjusted by a fuzzy controller. The fuzzy sets of the input variables S t1 and S t2 both contain 8 elements, and the fuzzy set of the output variable γ contains 9 elements, forming a total of 64 fuzzy rules. The input variables of the fuzzy controller are S t1 and S t2 , and the output variable is the weight coefficient γ. By dynamically adjusting the weight coefficient γ (0 ≤ γ ≤ 1) through a fuzzy control algorithm, fuzzy control rules based on the vehicle comfort index S t1 and the operation safety index S t2 are constructed, solving the problem that it is difficult for parameter-fixed control algorithms to be effectively applied to complex working conditions, etc., and improving the adaptability of the algorithm.

[0050] Furthermore, the logical algebra method is used to define the elements in as follows:

[0051] Specifically, when the absolute value of the element in is greater than or equal to the vertical seat acceleration threshold , the element takes the logical value "1", otherwise it takes the logical value "0"; for example takes Conversely, when , take

[0052] The definition is as follows: when all 4 elements in are logical 0, When only 1 element in is logical 1, When only 2 elements in are logical 1, When only 3 elements in

[0053] Similarly, the logical algebra method is used to define the elements in ​The elements in are defined as follows:

[0054] Specifically, when take When at that time, take When take When at that time, take When take When take

[0055] Furthermore, it is defined as follows: When all 3 elements in are logical 0, When only 1 element in is logical 1, When only 2 elements in are logical 1, When all 3 elements in are logical 1,

[0056] Furthermore, for the following definition is made:

[0057] Specifically, considering that the seat is the bridge connecting the driver and passengers to the vehicle, to ensure the riding comfort of the vehicle, it is defined as: and at that time, S t1 = 1; and at that time, S t1 = 2; and at that time, S t1 = 3; and at that time, S t1 = 4; at that time, S t1 = 5; at that time, S t1 = 6; at that time, S t1 = 7; at that time, S t1 = 8.

[0058] Similarly, using the logical algebra method, the elements in ΔZ1 = Z s - Z cs are defined as follows:

[0059] Specifically, when the absolute value of the elements in ΔZ1 is greater than or equal to the vertical displacement difference threshold ΔZ between the vehicle body and the seat 10When this occurs, this element takes the logical value "1", and vice versa, it takes the logical value "0"; for example, |Z sfl -Z cfl |≥Z 10 , take Z sfl -Z cfl =1, and conversely, when |Z sfl -Z cfl |<Z 10 When this occurs, take Z sfl -Z cfl =0.

[0060] Furthermore, the following is defined: when all 4 elements in ΔZ1 are logical 0, ΔZ1 = 0; when only 1 element in ΔZ1 is logical 1, ΔZ1 = 1; when only 2 elements in ΔZ1 are logical 1, ΔZ1 = 2; when only 3 elements in ΔZ1 are logical 1, ΔZ1 = 3; when all 4 elements in ΔZ1 are logical 1, ΔZ1 = 4.

[0061] Similarly, using the logical algebra method, the elements in ΔZ2 = Z - Z u are defined as follows:

[0062] Specifically, when the absolute value of the element in ΔZ2 is greater than or equal to the vertical displacement difference threshold ΔZ 20 between the vehicle body and the wheel, this element takes the logical value "1", and vice versa, it takes the logical value "0".

[0063] Furthermore, the following is defined: when all 4 elements in ΔZ2 are logical 0, ΔZ2 = 0; when only 1 element in ΔZ2 is logical 1, ΔZ2 = 1; when only 2 elements in ΔZ2 are logical 1, ΔZ2 = 2; when only 3 elements in ΔZ2 are logical 1, ΔZ2 = 3; when all 4 elements in ΔZ2 are logical 1, ΔZ2 = 4.

[0064] Similarly, using the logical algebra method, the elements in ΔZ3 = Z u -Q are defined as follows:

[0065] Specifically, when the absolute value of the element in ΔZ3 is greater than or equal to the vertical displacement difference threshold ΔZ 30 between the wheel and the road surface, this element takes the logical value "1", and vice versa, it takes the logical value "0".

[0066] Furthermore, the following is defined: when all 4 elements in ΔZ3 are logical 0, ΔZ3 = 0; when only 1 element in ΔZ3 is logical 1, ΔZ3 = 1; when only 2 elements in ΔZ2 are logical 1, ΔZ2 = 2; when only 3 elements in ΔZ3 are logical 1, ΔZ3 = 3; when all 4 elements in ΔZ3 are logical 1, ΔZ3 = 4.

[0067] Furthermore, for St2 is defined as = [ΔZ1, ΔZ2, ΔZ3] as follows:

[0068] Specifically, considering that the contact between the wheel and the ground is a prerequisite for the normal driving of the vehicle, to ensure the operational safety of the vehicle, it is defined that: when ΔZ3 = 0, both ΔZ1 and ΔZ2 are 0, or one is 1 and one is 0, or two are 1, S t2 = 1; when ΔZ3 = 0, both ΔZ1 and ΔZ2 are 2, or one is 2 and one is less than 2, S t2 = 2; when ΔZ3 = 0, both ΔZ1 and ΔZ2 are 3, or one is 3 and one is less than 3, S t2 = 3; when ΔZ3 = 0, both ΔZ1 and ΔZ2 are 4, or one is 4 and one is less than 4, S t2 = 4; when ΔZ3 = 1, S t2 = 5; when ΔZ3 = 2, S t2 = 6; when ΔZ3 = 3, S t2 = 7; when ΔZ3 = 4, S t2 = 8.

[0069] Furthermore, the physical universes of discourse of the input variables S t1 and S t2 are both set to [1, 8], and the physical universe of discourse of the output variable γ is [0, 1]; the fuzzy sets of the input variables S t1 and S t2 are both set to {EB, ES, GB, GS, MB, MS, PB, PS}, and the fuzzy set of the output variable γ is set to {SS, SM, SB, MS, MM, MB, BS, BM, BB}; all elements in the input and output fuzzy sets adopt Gaussian membership functions, and the fuzzy universes of discourse are both set to [-5, 5].

[0070] Specifically, according to the fuzzy sets of the input variables S t1 and S t2 of the fuzzy controller, 64 fuzzy control rules are formed, as shown in the following table:

[0071]

[0072] S106: The fuzzy controller adjusts the weight coefficient γ in real time according to the state variable S t , and the TD3 controller calculates the ideal control force A t in real time according to the state variable S t = [u fl , u fr , u rl , u rr T ​Act on the current controller, and the current I output by the current controller acts on the vertical magnetorheological damper, which provides the actual control force F a =[u fla ,u fra ,u rla ,u rra T Act on the vehicle to control the ride comfort and handling safety of the vehicle.

[0073] Specifically, the TD3 controller adopts an 11-degree-of-freedom vehicle vibration state space model, the reward function is a normalized reward function including vehicle ride comfort and handling safety indicators, and both the Actor-Critic networks are designed as network structures including an input layer, three hidden layers and an output layer, and the ReLU activation function is adopted between each network layer.

[0074] The purpose of such setting is to dynamically optimize the weight coefficient γ by using the fuzzy controller, and guide the magnetorheological damper to provide an effective control force to reduce the vibration of the whole vehicle by the optimal control force calculated by the TD3 controller, solve the contradiction problem between vehicle ride comfort and handling safety, and realize the simultaneous improvement of vehicle ride comfort and safety.

[0075] Furthermore, the steps of establishing an 11-degree-of-freedom vehicle-human loop coupling vibration double-delay deep deterministic policy gradient reinforcement learning TD3 controller specifically include:

[0076] Establishing an 11-degree-of-freedom vehicle-human loop coupling vibration model includes the vertical movement of the front and rear row seats, the vertical, lateral and pitching movements of the vehicle body, and the vertical movement of the wheels; establishing the dynamic equations of each seat, vehicle body and wheel according to Newton's second law, and the calculation formulas are as follows:

[0077] Vertical dynamic equation of the seat:[[]]

[0078]

[0079] Vertical dynamic equation of the vehicle body:[[]]

[0080]

[0081] Lateral movement equation of the vehicle body:[[]]

[0082]

[0083] Nodding movement equation of the vehicle body:[[]]

[0084]

[0085] Vertical movement equation of the wheel:[[]]

[0086] ​

[0087] In the formula, the subscripts fl, fr, rl, and rr represent the front left, front right, rear left, and rear right positions respectively; m sf represents the mass of a single front-row seat and the occupants in the vehicle, m sr represents half of the mass of the rear-row seat and the occupants in the vehicle, m c represents the mass of the vehicle body excluding the seats, m u represents the mass of a single vehicle wheel; I cx represents the body sideslip moment, I cy represents the body pitching moment; K s represents the seat spring stiffness, C s represents the seat damping coefficient, K u represents the body spring stiffness, K r represents the equivalent stiffness of the wheel; l1 and l2 respectively represent the distances from the measuring points of the front and rear-row seats to the vehicle body center along the vehicle body length direction; l3 and l4 respectively represent the distances from the centers of the front and rear vertical magnetorheological dampers of the vehicle to the vehicle body center along the vehicle body length direction; l c represents the distances from the measuring points of the left and right seats of the vehicle to the vehicle body center along the vehicle body width direction; l represents the distances from the centers of the left and right vertical magnetorheological dampers of the vehicle to the vehicle body center along the vehicle body width direction; u fl , u fr , u rl , u rr respectively represent the forces provided by the 4 vertical magnetorheological dampers acting on the vehicle body; Z sfl , Z sfr , Z srl , Z srr respectively represent the vertical displacements at the measuring points of the front and rear-row seats of the vehicle; Z c , φ c , β c respectively represent the vertical displacement, sideslip angle, and pitch angle of the vehicle body center; Z cfl , Z cfr , Z crl , Z crr represents the vertical displacements at the 4 connections between the seat and the vehicle body; Z fl , Z fr , Z rl , Z rr represents the vertical displacements at the 4 connections between the vehicle body and the vertical magnetorheological dampers; Z ufl , Z ufr , Z url , Z urr represents the vertical displacements of the 4 vehicle wheels; q fl , q fr , q rl , q rr represents the road surface roughness displacements at the 4 vehicle wheels.

[0088] Furthermore, it also includes a 11-degree-of-freedom vehicle-human-environment coupled vibration state space model derived from the 11-degree-of-freedom vehicle-human-environment coupled vibration model, and its expression is as follows:

[0089]

[0090] In the formula U = [u; w], x = [Z sfl , Z sfr , Z srl , Z srr , Z c , φ c , β c , Z ufl , Z ufr , Z url , Z urr T , u = [u fl , u fr , u rl , u rr T , w = [q fl , q fr , q rl , q rr T ; A, B, C, D are coefficient matrices, and their expressions are as follows:

[0091] A1 = 0 11×11 , A2 = I 11×11 ,

[0092]

[0093] A6 = 0 4×11 ,

[0094] B1 = 0 15×8 ,

[0095] C = I 22×22 ; D = 0 22×8 .

[0096] The purpose of such a setting is to solve the problems such as poor actual application effect of the control algorithm caused by ignoring the vehicle coupling vibration in the existing research by establishing a "vehicle-human-environment" coupled vibration model including the driver and passengers, the vehicle and the road environment, and improve the accuracy of the vehicle whole vehicle model.

[0097] ​​​Furthermore, it also includes designing a TD3 controller based on the 11-degree-of-freedom vehicle-human coupling vibration state space model. Specifically, it includes designing the normalized reward function R of the TD3 controller, and its expression is as follows:

[0098] R = -(1 - γ)(R1 + R2) - γ(R3 + R4 + R5);

[0099] In the formula, γ (0 ≤ γ ≤ 1) is the weight coefficient, which is used to characterize the vertical acceleration at the seat measurement point; which is used to characterize the vertical, nodding, and lateral acceleration at the vehicle body center; R1 and R2 together characterize the vehicle ride comfort; R3 = [(Z sf l - Z cf l) N 2 + [(Z sfr - Z cfr ) N 2 + [(Z srl - Z crl ) N 2 + [(Z srr - Z crr ) N 2 , which is used to characterize the seat dynamic stroke; R4 = [(Z f l - Z uf l) N 2 + [(Z fr - Z ufr ) N 2 + [(Z rl - Z url ) N 2 + [(Z rr - Z urr ) N 2 , which is used to characterize the vehicle body dynamic stroke; R5 = [(Z ufl - q fl ) N 2 + [(Z ufr - q fr ) N 2 + [(Z url - q rl ) N 2 + [(Z urr - q rr ) N 2 ​​​​​​​​​​​​, which is used to characterize the dynamic stroke of the tire; R3, R4, and R5 together characterize the vehicle operation safety; the subscript N represents normalization, and the minimum-maximum normalization method is adopted. After normalization, the value range is [0, 1].

[0100] The purpose of such a setting is to solve the problems such as the inconsistent dimension of the reward function parameters and the poor training effect of the reinforcement learning controller caused by the fixed parameters by designing a normalized reward function R = -(1 - γ)(R1 + R2) - γ(R3 + R4 + R5) with a weight coefficient γ (0 ≤ γ ≤ 1), and improve the training efficiency and accuracy of the reinforcement learning controller.

[0101] Furthermore, it also includes designing the Actor-Critic network structure of the TD3 controller. Please refer to Figure 6-7 , Figure 6 This is the structure diagram of the Actor network; Figure 7 This is the structure diagram of the Critic network. The Actor network has 19 inputs, 4 outputs, and 3 hidden layers. The number of nodes in the hidden layers is 64, 32, and 8 respectively, and the ReLU activation function is adopted; the Critic network has 23 inputs, 1 output, and 3 hidden layers. The number of nodes in the hidden layers is 128, 64, and 8 respectively, and the ReLU activation function is adopted.

[0102] For those skilled in the art, it is obvious that the present application is not limited to the details of the above exemplary embodiments, and the present application can be implemented in other specific forms without departing from the spirit or basic characteristics of the present application. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present application is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application. Any reference signs in the claims should not be regarded as limiting the claimed rights.

Claims

1. A collaborative reinforcement learning control method for automobile riding comfort and operational safety, characterized in that: include: S101: Obtain the vertical acceleration at the four measuring points of the front and rear seats of the vehicle respectively Vertical acceleration at 4 measuring points on the vehicle body Vertical acceleration at the four wheel measuring points The road surface roughness information Q at the four wheel measuring points is [q fl ,q fr ,q rl ,q rr ]; where the subscript fl ,fr,rl ,rr Respectively represent the front left, front right, rear left and rear right positions; S102: Obtaining the vehicle body center acceleration according to a preset method and vertical acceleration at the connection between the seat and the body in; They represent the vertical acceleration of the vehicle body center, the sideslip angular acceleration and the pitch angular acceleration respectively; S103: Acceleration Integrate twice to get the displacement Z s =[Z sfl ,Z sfr ,Z srl ,Z srr ],Z=[Z fl ,Z fr ,Z rl ,Z rr ],Z u =[Z ufl ,Z ufr ,Z url ,Z urr ], X c =[Z c ,β c ,φ c ],Z cs =[Z cfl ,Z cfr ,Z crl ,Z crr ]; S104: According to the displacement Z s , Z, Z u , X c , Z cs The vertical displacement difference ΔZ1=Z between the vehicle body and the seat, between the vehicle body and the wheel, and between the wheel and the ground is obtained by combining the road surface roughness information Q. s -Z cs , ΔZ2=ZZ u , ΔZ3=Z u -Q; S105: Definition is the comfort index, S t2 =[ΔZ1, ΔZ2, ΔZ3] is the safety index, and S t =(1-γ)S t1 +γS t2 As the state variable, γ (0≤γ≤1) is the weight coefficient and is dynamically adjusted by the fuzzy controller; the ideal control force A provided by the four vertical magnetorheological dampers acting on the car body t =[u fl ,u fr ,u rl ,u rr ] T As the action quantity, a double-delay deep deterministic policy gradient reinforcement learning TD3 controller with 11 degrees of freedom human-vehicle-loop coupled vibration is established; S106: The fuzzy controller is based on the state quantity S t The weight coefficient γ is adjusted in real time. The TD3 controller adjusts the weight coefficient γ according to the state quantity S t Calculate the ideal control force A in real time t =[u fl ,u fr ,u rl ,u rr ] T Acting on the current controller, the current controller output current I acts on the vertical magnetorheological damper, and the vertical magnetorheological damper provides the actual control force F a =[u fla ,u fra ,u rla ,u rra ] T Acts on the car to control the car's riding comfort and operating safety.

2. The method for collaborative reinforcement learning control of vehicle riding comfort and operational safety according to claim 1 is characterized in that: In step S101, a plurality of vibration acceleration sensors are used to measure the vertical acceleration The vibration acceleration sensors are respectively installed at the bottom of the front and rear seats, the connection between the vehicle body and the vertical shock absorber, and the center of the wheel rim; the ultrasonic ranging sensor is used to measure the road surface roughness information Q = [q fl ,q fr ,q rl ,q rr ].

3. The method for collaborative reinforcement learning control of vehicle riding comfort and operational safety according to claim 2 is characterized in that: In step S102, the vehicle body center acceleration is obtained according to a preset method. The steps include: The vehicle body center acceleration is calculated based on the vehicle body structure, vertical magnetorheological damper and seat installation dimensions. The calculation formula is as follows: Wherein, l1, l2 represent the distance between the front and rear seat measuring points and the center of the vehicle body along the length direction of the vehicle body; l3, l4 represent the distance between the center of the vertical magnetorheological damper at the front and rear ends of the vehicle and the center of the vehicle body along the length direction of the vehicle body; l c It represents the distance between the left and right seat measuring points and the center of the vehicle body along the width direction of the vehicle body; l represents the distance between the center of the vertical magnetorheological damper at the left and right ends of the vehicle and the center of the vehicle body along the width direction of the vehicle body.

4. The method for collaborative reinforcement learning control of vehicle riding comfort and operational safety according to claim 3 is characterized in that: In step S102, the vertical acceleration of the connection between the seat and the vehicle body is obtained according to a preset method. The steps include: The vertical acceleration at the connection between the seat and the body is calculated based on the body structure and seat installation dimensions. The calculation formula is as follows:

5. The method for collaborative reinforcement learning control of vehicle riding comfort and operational safety according to claim 1, characterized in that: In step S105, the weight coefficient γ (0≤γ≤1) is dynamically adjusted by the fuzzy controller; the fuzzy controller input S t1 and S t2 The fuzzy sets of contain 8 elements, and the fuzzy set of the output quantity γ contains 9 elements, forming a total of 64 fuzzy rules.

6. The method for collaborative reinforcement learning control of vehicle riding comfort and operational safety according to claim 5 is characterized in that: In step S106, the TD3 controller adopts an 11-degree-of-freedom vehicle vibration state space model, the reward function is a normalized reward function that includes vehicle ride comfort and operational safety indicators, and the Actor-Critic network is designed to have a network structure including an input layer, three hidden layers and an output layer, and ReLU activation function is used between each network layer.

7. The method for collaborative reinforcement learning control of vehicle riding comfort and operational safety according to claim 6 is characterized in that: In step S106, establishing an 11-DOF human-vehicle-environment coupled vibration double-delay deep deterministic policy gradient reinforcement learning TD3 controller includes: An 11-DOF human-vehicle-environment coupled vibration model is established, including the vertical motion of the front and rear seats, the vertical, lateral and pitching motions of the vehicle body, and the vertical motion of the wheels. The dynamic equations of the seats, vehicle body and wheels are established according to Newton’s second law, and the calculation formula is as follows: Vertical dynamic equation of the seat: The vertical dynamics equation of the vehicle body: The equation of motion for vehicle body lateral deviation is: Car body nodding motion equation: The vertical motion equation of the wheel: Wherein, subscripts fl, fr, rl, and rr represent the front left, front right, rear left, and rear right positions, respectively; m sf Indicates the mass of a single front seat and its passengers, m sr Represents half of the mass of the rear seats and passengers in the car, m c Indicates the mass of the car body excluding the seats, m u Indicates the mass of a single wheel of a car; I cx is the body cornering moment, I cy Indicates the vehicle body nodding moment; K s Indicates the seat spring stiffness, C s Indicates the seat damping coefficient, K u Indicates the body spring stiffness, K r represents the equivalent stiffness of the wheel; l1, l2 represent the distances between the front and rear seat measuring points and the center of the vehicle body along the length direction of the vehicle body; l3, l4 represent the distances between the center of the vertical magnetorheological damper at the front and rear ends of the vehicle and the center of the vehicle body along the length direction of the vehicle body; l c It represents the distance between the left and right seat measuring points and the center of the vehicle body along the width direction of the vehicle body; l represents the distance between the center of the vertical magnetorheological damper at the left and right ends of the vehicle and the center of the vehicle body along the width direction of the vehicle body; u fl ,u fr ,u rl ,u rr They represent the forces provided by the four vertical magnetorheological dampers acting on the vehicle body; Z sfl ,Z sfr ,Z srl ,Z srr Respectively represent the vertical displacement of the front and rear seats of the car; Z c ,φ c ,β c Respectively represent the vertical displacement of the vehicle body center, the sideslip angle and the pitch angle; Z cfl ,Z cfr ,Z crl ,Z crr Indicates the vertical displacement of the four connections between the seat and the body; Z fl ,Z fr ,Z rl ,Z rr represents the vertical displacement of the connection between the vehicle body and the four vertical MR dampers; Z ufl ,Z ufr ,Z url ,Z urr represents the vertical displacement of the four wheels; q fl ,q fr ,q rl ,q rr Indicates uneven road surface displacement at the four wheels.

8. The method for collaborative reinforcement learning control of vehicle riding comfort and operational safety according to claim 7 is characterized in that: It also includes the derivation of an 11-DOF human-vehicle-environment coupled vibration state space model based on the 11-DOF human-vehicle-environment coupled vibration model, and its expression is as follows: In the formula U=[u;w],x=[Z sfl ,Z sfr ,Z srl ,Z srr ,Z c ,φ c ,β c ,Z ufl ,Z ufr ,Z url ,Z urr ] T ,u=[u fl ,u fr ,u rl ,u rr ] T , w=[q fl ,q fr ,q rl ,q rr ] T ; A, B, C, D are coefficient matrices, and their expressions are as follows: <h2 style=";text-align:left;direction:ltr"> <h2 style=";text-align:left;direction:ltr"> A1=0<h2 style=";text-align:left;direction:ltr"> 11×11 <h2 style=";text-align:left;direction:ltr"> A2=I<h2 style=";text-align:left;direction:ltr"> 11×11 <h2 style=";text-align:left;direction:ltr"> , A6＝0 4×11 ， B1=0 15×8 , C=I 22×22 ;D=0 22×8 。 9. The method for collaborative reinforcement learning control of vehicle riding comfort and operational safety according to claim 8, characterized in that: It also includes designing a TD3 controller based on an 11-DOF human-vehicle-ring coupled vibration state space model. Specifically, it includes designing a normalized reward function R of the TD3 controller, and its expression is as follows: R=-(1-γ)(R1+R2)-γ(R3+R4+R5); In the formula, γ (0≤γ≤1) is the weight coefficient, It is used to characterize the vertical acceleration at the seat measuring point; It is used to characterize the vertical, nodding and lateral acceleration of the vehicle body center; R1 and R2 are used together to characterize the ride comfort of the vehicle; R3 = [(Z sfl -Z cfl ) N ] 2 +[(Z sfr -Z cfr ) N ] 2 +[(Z srl -Z crl ) N ] 2 +[(Z srr -Z crr ) N ] 2 , which is used to characterize the seat travel; R4 = [(Z fl -Z ufl ) N ] 2 +[(Z fr -Z ufr ) N ] 2 +[(Z rl -Z url ) N ] 2 +[(Z rr -Z urr ) N ] 2 , which is used to characterize the vehicle body dynamic travel; R5 = [(Z ufl -q fl ) N ] 2 +[(Z ufr -q fr ) N ] 2 +[(Z url -q rl ) N ] 2 +[(Z urr -q rr ) N ] 2 , which is used to characterize the tire travel; R3, R4 and R5 together characterize the vehicle's operational safety; The subscript N indicates normalization, and the minimum-maximum normalization method is adopted. The value range after normalization is [0,1].

10. The method for collaborative reinforcement learning control of vehicle riding comfort and operational safety according to claim 9, characterized in that: It also includes the design of the Actor-Critic network structure of the TD3 controller, in which the Actor network has 19 inputs, 4 outputs, and 3 hidden layers. The number of hidden layer nodes is 64, 32, and 8 respectively, and the ReLU activation function is used; the Critic network has 23 inputs, 1 output, and 3 hidden layers. The number of hidden layer nodes is 128, 64, and 8 respectively, and the ReLU activation function is used.

Images