Distributed driving vehicle composite steering control method based on reinforcement learning interaction MPC

Abstract

The invention discloses a distributed driving vehicle composite steering control method based on reinforcement learning interaction MPC. The method comprises the following steps: step 1, distributed driving vehicle steering control based on MPC; 2, composite steering control parameter optimization based on reinforcement learning; thirdly, compensation control of the modular composite steering system is carried out; and 4, performing redundancy control on the composite steering system. The method has the beneficial effects that the accurate model is established under the sudden turn working condition of the multi-axis distributed driving vehicle; optimal stability adjustment of the vehicle under the sudden turning working condition is achieved, and the rollover risk is greatly reduced. The control stability and the adaptability of the vehicle under the sudden turning working condition in the changeable environment are greatly improved; the smoothness of the vehicle during steering mode switching is improved; dependence on a prior fault model is eliminated, and accurate and rapid fault positioning of a single round is realized.

Description

Technical Field

[0001] The present invention relates to a distributed drive vehicle compound steering control method, in particular to a distributed drive vehicle compound steering control method based on reinforcement learning interactive MPC. Background Art

[0002] At present, the attention and technology accumulation of new energy passenger vehicles at home and abroad are much higher than that of new energy commercial vehicles, especially heavy-duty vehicles. In order to meet the great demand of transportation and accelerate the technological innovation of new energy vehicles, many research institutions have begun to conduct system control research on distributed electric drive heavy-duty vehicles. In this process, the combination of reinforcement learning and traditional control algorithms has further expanded the possibility of new energy vehicle control.

[0003] At present, there are few invention patents on the steering control of distributed electric drive vehicles. Chinese patent CN202410214451.9 introduces a lateral and longitudinal coordinated control method of a distributed corner module chassis configuration. The patent introduces a control method for introducing differential steering when a distributed drive vehicle is turning, but the control object is not suitable for heavy-loaded multi-axle vehicles. Invention patent CN202311175192.5 discloses a multi-axle distributed chassis stability control system and control method. The system control model in the invention patent still follows the Ackerman steering model, and the optimal vehicle body posture and lateral stability cannot be achieved during the steering process. Invention patent CN202311518186.5 discloses a vehicle stability coordination control method based on reinforcement learning. By constructing a reinforcement learning network model, multiple reinforcement learning network models are optimized respectively, and the additional front wheel steering angle and additional yaw moment of the optimal coordinated control vehicle are selected, but it is not suitable for short-term precise control under sharp turning conditions.

[0004] The patent of this invention is in line with the national new energy vehicle development trend and the shortcomings of the existing technology, and proposes a distributed drive vehicle compound steering control method based on reinforcement learning interactive MPC. In order to ensure that the new configuration of distributed drive heavy-duty vehicles can have better lateral stability in sharp turning conditions and good adaptability in variable road environments, the present invention uses a three-axis distributed drive vehicle as a platform, with the goal of keeping the vehicle stability parameters within a reasonable range, divides different working conditions, and adopts a reinforcement learning interactive MPC control method under sharp turning conditions, achieving good obstacle avoidance performance of the vehicle under emergency conditions and adaptive performance under various working conditions. Summary of the invention

[0005] The purpose of the present invention is to solve the problems of poor stability and weak adaptability of multi-axle distributed heavy-load vehicles under sharp turning conditions, overcome the poor adaptability of traditional model predictive control due to reliance on accurate vehicle models, and make full use of the outstanding advantages of modular configuration such as single-wheel steering and independent drive, and provide a distributed drive vehicle compound steering control method based on reinforcement learning interactive MPC.

[0006] The distributed drive vehicle compound steering control method based on reinforcement learning interactive MPC provided by the present invention comprises the following steps:

[0007] Step 1: Distributed drive vehicle steering control based on MPC;

[0008] Step 2: Optimization of compound steering control parameters based on reinforcement learning;

[0009] Step 3: Modular composite steering system compensation control;

[0010] Step 4: Redundant control of compound steering system.

[0011] The process of MPC-based distributed drive vehicle steering control in the first step is as follows:

[0012] Step 1: Establish a linear two-degree-of-freedom model for compound steering of a three-axle vehicle. Considering that the load on the two steering wheels of the front axle is large under sharp turning conditions, the steering angle and tire lateral stiffness of the left and right tires of the front axle are taken into account to improve the accuracy of the model. Considering the influence of differential steering, the torque acting on each axle generated by differential steering and the additional lateral force caused by differential steering are taken into account. The obtained linear two-degree-of-freedom model is shown in the following equations (1) and (2):

[0013]

[0014] Where m is the total mass of the vehicle, a y is the lateral acceleration of the vehicle, k f1 , k f2 , k m , k r are the cornering stiffness of the left and right tires on the front axle, the tires on the middle axle, and the tires on the rear axle, δ f1 , δ f2 , δ m , δ r is the steering angle of the left and right tires on the front axle, the tires on the intermediate axle, and the tires on the rear axle, v y is the lateral velocity of the vehicle, l f , l m , l r are the distances from the front axle, middle axle, and rear axle to the center of mass, ω r is the vehicle's yaw rate, u is the vehicle's forward speed, Ffl 、F fr 、F ml 、F mr 、F rl 、F rr The driving forces of the front, middle and rear axles are the left and right wheels, d f d m d r They are the front axle, middle axle, and rear axle track respectively;

[0015] Step 2: Judgment of the limit of the rapid-turning condition and calculation of the target state. The specific three steps are as follows:

[0016] Step 1: Driver's intention to turn sharply is evaluated. The current vehicle speed and steering wheel angular velocity are used to determine whether the vehicle is in an emergency condition. The defined limits are shown in equations (3) and (4):

[0017] v≤v boundary ω≥ω low (3)

[0018] v≥v boundary ω≥ω high (4)

[0019] Where v is the longitudinal velocity of the vehicle, ω is the angular velocity of the steering wheel, and v boundary is the speed limit, ω low is the steering wheel angular velocity limit under low speed conditions, ω high It is the angular velocity limit of the steering wheel under high-speed conditions;

[0020] Step 2: Vehicle emergency status assessment: obtain the current ω through the vehicle yaw angular velocity sensor and lateral acceleration sensor r with a y , by constructing a stability index to comprehensively consider the impact of the two on vehicle stability, as shown in formula (5), and finally establish a functional relationship between the mode switching limit and the stability index, and define the stability index and mode switching limit function, as shown in formula (6):

[0021] S=k1|a y |+k2|ω r | (5)

[0022] L=A·e -B×S (6)

[0023] In the formula, S is the stability index, a y is the lateral acceleration, k1 and k2 are weight coefficients, and L is the mode switching limit;

[0024] Step 3: Design of stable target state decision controller. By defining the distribution ratio coefficient k between wire control steering and differential steering, the driver's steering wheel input is assigned tasks. The pure wire control steering module of the three-axis distributed drive vehicle is designed according to the steering task, taking into account the longitudinal speed of the vehicle and the Ackerman steering relationship to change the transmission ratio design and calculate the target turning angle δ of each wheel. f1 ,δ f2 ,δ m ,δ r The differential steering module performs hierarchical control according to the steering task, and the upper controller calculates the target yaw rate ω through the equivalent wire control steering model r , the dynamic output feedback controller is used to obtain the desired yaw moment, and the lower controller uses a direct allocation algorithm to obtain the target driving force difference F of each axis fl -F fr ,F ml -F mr ,F rl -F rr , where the design of the dynamic output feedback controller is shown in equations (7) and (8):

[0025]

[0026] In the formula, Indicates the controller status, A kij ,B kj ,C kj ,D k is the controller matrix to be designed, A k ,B k ,C k ,D k is the final controller matrix considering that the parameter h(t) varies with speed;

[0027] Step 3: Design of coordinated controller for steer-by-wire and differential steering based on MPC. The specific four steps of step 3 are as follows:

[0028] Step 1: State space modeling of the compound steering system. By discretizing the linear two-degree-of-freedom model of the three-axis vehicle compound steering in step 1 of the first step, the vehicle state vector is defined as:

[0029]

[0030] The control input vector of the compound steering system is defined as:

[0031] u k =[δ f1 δ f2 δ m δ r F fl -F frF ml -F mr F rl -F rr ] T

[0032] The obtained state space model is shown in formula (9):

[0033] x k+1 =Ax k +Bu k (9)

[0034] in:

[0035]

[0036] Step 2: Establishment of the performance index function for compound steering system prediction in the time domain optimization. By defining the compound steering system input increment and the vehicle augmented state vector Δu [k] =u [k] -u [k-1] With x a[k+1] =[x [k] x d[k] u [k-1] ], the state space equation of the compound steering system under the input increment and the performance index function of the compound steering system under the input increment are obtained, as shown in equations (10) and (11), respectively:

[0037] x ak+1] =A a x a[k] +B a Δu [k] (10)

[0038] in:

[0039] B a =[B 0 n×p I p×p ] T

[0040]

[0041] Among them, A D is the transition matrix of the target state, I p×p is the unit matrix, S, Q, R are weight matrices;

[0042] k, k+1,...k+N p The state space equations at each moment are iterated sequentially to obtain the prediction equation of the composite steering system:

[0043] X [k] =Φ x[k|k]+ΓU [k] (12)

[0044] The vehicle performance index matrix is ​​defined as:

[0045]

[0046] At time k, the prediction interval will be N p When , the performance index function within this prediction interval is sorted out to obtain the prediction time domain optimization performance index function:

[0047]

[0048] Let F = Γ T ΩΦ,H=Γ T ΩΓ+Ψ, transforming the performance index function into a canonical form;

[0049] Step 3: Design of constraints for the composite steering system. Based on the motion characteristics of the three-axis distributed drive vehicle that can be controlled by the single wheel angle, the physical constraints of the system are constructed. The proportional relationship of the three-axis wheel angles is written as a set of linear equality constraints. The angle limit of the modular steering system and the difference limit of the wheel hub motor driving force are used as inequality constraints. The constraint matrix is ​​converted into a canonical form that can be solved by quadratic programming. The performance index function considering the system constraints is obtained, as shown in formula (14):

[0050]

[0051] Where H is a symmetric positive definite matrix consisting of the coefficients of the quadratic terms in the composite steering cost function, f is the coefficient of the linear term in the composite steering cost function, c is a constant, and δ min , δ max is the angle limit of the modular steering system, ΔF rmin , ΔF rmax It is the difference limit of the driving force of the hub motor;

[0052] By substituting the performance index function and constraint conditions (14) considering the system constraints into the quadratic programming solver, the optimal distribution of the control input vector can be obtained. The solver searches for the value that minimizes the objective function in the feasible domain. By performing such a solution process at each sampling time, real-time control of the composite steering allocation strategy based on quadratic programming can be achieved.

[0053] The process of optimizing the compound steering control parameters based on reinforcement learning in the second step is as follows:

[0054] Step 1: Establish vehicle status and select compound steering system actions. The vehicle status includes lateral acceleration, yaw rate, steering response time, steering motor energy consumption, vehicle current steering angle vector, and steering angle adjustment of each wheel. The compound steering system action selection includes prediction time domain, control time domain, allocation ratio coefficient of wire control steering and differential steering in step 2, link 3 of the first step, and angle limit of modular steering system and wheel hub motor driving force difference limit in step 3, link 4 of the first step;

[0055] S = {a y ,ω y ,t delay ,E consumed ,θ(t),Δθ(t)}

[0056] A={T p ,T c ,k,δ min ,δ max ,ΔF rmin ,ΔF rmax}

[0057] In the formula, S is the vehicle state, A is the steering system action, T p is the prediction time domain, T c To control the time domain;

[0058] Step 2: Design of the four-dimensional vehicle stability reward function, including lateral stability reward (15), yaw rate stability reward (16), steering response timeliness reward (17), and energy efficiency reward (18), as shown in the following formulas:

[0059]

[0060] R=ω1r1+ω2r2+ω3r3+ω4r4 (19)

[0061] Among them, a y is the actual lateral acceleration, a yd is the ideal lateral acceleration, a ymax is the maximum lateral acceleration threshold, w r is the actual yaw rate, w rd is the ideal yaw rate, w rmax is the maximum yaw rate threshold, t delay is the time delay, t delaymax is the maximum acceptable delay time, E consumed is the energy currently consumed, E max is the maximum energy consumption threshold, ω1, ω2, ω3, ω4 are weight coefficients;

[0062] Step 3: Design of composite steering system parameter optimizer based on LSTM. The specific two links in step 3 are as follows:

[0063] Step 1: Construction of the strategy network of the composite steering system parameter optimizer. The strategy network of the composite steering system parameter optimizer consists of an LSTM layer and a fully connected layer. The input signal is the current state s t , the LSTM layer processes the state sequence and outputs the hidden state h t , the fully connected layer maps the LSTM output to the dimension of the action space and calculates the mean (20), standard deviation (21), and deterministic action (22) of the action, as shown in the following formulas:

[0064] μ(s t )=W μ h t +b μ (20)

[0065] σ(s t )=W σ h t +b σ (twenty one)

[0066] α t =μ(s t )+∈⊙σ(s t ) (twenty two)

[0067] Where W μ ,W σ is the weight matrix, b μ ,b σ is the bias term, ⊙ is the element-wise multiplication, and ∈ is the random noise sampled from the standard normal distribution;

[0068] Step 2: Construction of the value network of the composite steering system parameter optimizer. The value network of the composite steering system parameter optimizer consists of an LSTM layer and a fully connected layer. The input signal is the current state s t With the current action a t , the LSTM layer processes the state sequence and outputs the hidden state h t , the input signal of the fully connected layer is the concatenated vector of state and action and the output of LSTM, and the output signal is the state value function estimation, as shown in formula (23):

[0069] Q(s t ,a t )=W Q ([s t ,a t ],h t )+b Q (twenty three)

[0070] Where WQ is the weight matrix, b Q is the bias term;

[0071] Step 4: Optimization of control parameters of the composite steering system based on the deep deterministic policy gradient algorithm. The specific four steps in step 4 are as follows:

[0072] Step 1: Establish and initialize the composite steering system parameter optimizer architecture, randomly initialize the composite steering system parameter optimizer strategy network in step 2, step 3, step 1, and the parameters θ of the composite steering system parameter optimizer value network in step 2, step 3, step 2 μ ,θ Q , copy and create a composite steering system parameter optimizer target strategy network and a composite steering system parameter optimizer target value network respectively;

[0073] Step 2: Calculation of target parameters and target parameter values. For each time step t, according to the current state s t , the action output of the compound steering system parameter optimizer strategy network is a t =μ(s t |θ μ )+∈, where ∈ is noise, and performs action a t , get the vehicle state feedback s at the next moment t+1 , the reward value r is obtained from the four-dimensional vehicle stability reward function in step 2 of the second step t , store the transfer samples in the experience replay buffer D, and then randomly sample a batch of transfer samples from the experience replay buffer. The target parameters and target parameter value calculations are shown in (24) and (25) respectively:

[0074] a t+1 =μ'(s t+1 |θ μ ′ ) (twenty four)

[0075] y t =r t +γQ'(s t+1 ,a t+1 |θ Q′ ) (25)

[0076] In the formula, γ is the discount factor;

[0077] Step 3: Update the network parameters of the composite steering system parameter optimizer based on the DDPG algorithm. The value network parameters of the composite steering system parameter optimizer in step 1 of step 4 of step 2 are updated by minimizing the loss function, as shown in formula (26). The sampled policy gradient is used to update the policy network parameters of the composite steering system parameter optimizer in step 1 of step 4 of step 2, as shown in formula (27):

[0078]

[0079] In the formula, L(θ Q ) is the loss function of the composite steering system parameter optimizer value network, y i is the target value, Q(s i ,a i |θ Q ) is the vehicle state s i Take parameter a i When, based on θ Q The value of is the parameter network parameter of the compound steering system optimizer strategy θ μ About J(θ μ ), u(s i |θ u ) is the μ Next, the vehicle state s i The action output by the system parameter optimizer strategy network;

[0080] Step 4: Soft update of the target composite steering system parameter optimizer network. As shown in formula (28):

[0081] θ μ′ ←τθ μ +(1-τ)θ μ′ ,θ Q′ ←τθ Q +(1-τ)θ Q′ (28)

[0082] Where τ is the soft update coefficient.

[0083] The process of the compensation control of the modular composite steering system in the third step is as follows:

[0084] Step 1: Kingpin motor steering system model construction. The specific two steps in step 1 are as follows:

[0085] Step 1: Establishment of the motion equation of the kingpin motor steering system. The kingpin steering motor uses a brushless DC motor. The voltage balance equation of the stator winding, the electromagnetic torque equation, and the mechanical motion equation of the steering system are established. The motion equation of the kingpin motor steering system is obtained through joint calculation, as shown in formula (29):

[0086]

[0087] in:

[0088] T e =K t i

[0089]

[0090] In the formula, U is the stator winding terminal voltage, R is the stator winding resistance, i is the winding current, L is the winding inductance, e is the back electromotive force, K t is the torque constant, J is the moment of inertia of the steering system, including the motor rotor and the kingpin, B is the damping coefficient of the steering system, T r is the electromagnetic torque, T r is the steering resistance torque, θ is the rotation angle of the motor output shaft, and ω is the angular velocity of the motor output shaft;

[0091] Step 2: Establishment of the state space equation of the kingpin motor steering system. Define the state variable: x = [θ, ω, i]T, input variable: u = [U], output variable y = [θ, nθ], and the state equation is shown in equation (30):

[0092]

[0093] in:

[0094] Step 2: Construct the wheel hub motor differential steering system model. The motion equation of the wheel hub motor can be obtained through the voltage equation and electromagnetic torque equation of the wheel hub motor, as shown in formula (31). Similarly, its state space equation can be obtained.

[0095]

[0096] Where, L d , L q are the d-axis and q-axis inductances, respectively, and i d 、i q are the d-axis and q-axis stator currents, ω e is the electrical angular velocity, ψ f is the permanent magnet flux. J is the moment of inertia of the hub motor and related rotating parts, B is the viscous damping coefficient of the rotating parts, T L is the load torque;

[0097] Step 3: Dual system compensation control under emergency conditions. The specific four links in step 3 are as follows:

[0098] Link 1: The kingpin steering motor enters control. In the time period [0-t1], an exponential function steering angle command is set to make the current rise quickly, so that the kingpin steering motor can respond quickly and the steering angle can be established quickly. The steering angle command and the kingpin steering motor control equation are shown in equations (32) and (33) respectively:

[0099] θ d (t) = θ0(1-e -kt ) (32)

[0100]

[0101] Where θ0 is the maximum steering angle required for a sharp turn, k is a constant that determines the speed at which the kingpin steering motor enters the sharp turn state, and t is a time variable;

[0102] Step 2: Hub motor entry control. [0~t2] is the time interval when the hub motor enters rapid rotation. The target yaw rate segmented function is used to make the hub motor enter the working state relatively quickly. The yaw rate command and the hub motor differential control equation are shown in equations (34) and (35) respectively:

[0103]

[0104] Where θ0 is the maximum steering angle required for a sharp turn, k is a constant that determines the speed at which the kingpin steering motor enters the sharp turn state, and t is a time variable;

[0105] Link 3: Wheel hub motor exit control. At the end of the sharp turn, the vehicle's driving state gradually stabilizes, and the yaw rate and lateral acceleration approach normal driving values. Therefore, the output of the wheel hub motor's differential torque attenuation speed function adopts a sinusoidal decreasing form, and the output torque is stopped first. The process takes tens of milliseconds to one hundred milliseconds. The wheel hub motor exit control method is shown in formula (36):

[0106]

[0107] Where t is the time variable, [t5, t6] is the time interval when the wheel hub motor exits;

[0108] Link 4: Kingpin steering motor exit control. When the sharp turn is about to end, the driver gradually returns the steering wheel to the center. However, due to the inertia of the vehicle during the sharp turn and the mechanical return characteristics of the steering system, the kingpin steering motor continues to adjust the output torque while the steering angle of the vehicle gradually returns to the normal driving state to ensure the smoothness of the steering. Therefore, the steering torque attenuation speed function should adopt a quadratic decreasing form. The whole process lasts for several hundred milliseconds to longer. The kingpin steering motor exit control method is shown in equations (37)-(40):

[0109]

[0110] Wherein, [t3, t4] is the time interval when the kingpin steering motor exits.

[0111] The process of redundant control of the composite steering system in the fourth step is as follows:

[0112] Step 1: Multi-criteria fault diagnosis of compound steering system of multi-axis electric drive vehicle. The three specific steps in step 1 are as follows:

[0113] Link 1: Fault judgment of sudden turning condition based on reinforcement learning. According to the state space and action space defined in step 1 of the second step, the current steering angle vector of the vehicle θ(t) = [θ 11 (t),θ 12 (t),…θ nm (t)] and the steering angle adjustment of each wheel Δθ(t). According to the stability reward function defined in step 2 of the second step, the reward item related to the vehicle lateral acceleration is already included. The fault judgment algorithm is shown in formula (41):

[0114]

[0115] In the formula, a y (t) is the ideal lateral acceleration calculated based on reinforcement learning, Δa y is the lateral acceleration deviation limit value;

[0116] Step 2: Fault judgment based on motor response consistency algorithm, by defining the consistency index η of the kingpin steering motor m (t) and weight coefficient w ij , as shown in equations (42) and (43), and then the fault diagnosis is performed through the consistency of each motor by comparing the set consistency threshold. The consistency index calculation is shown in equation (46):

[0117]

[0118] By expanding the consistency index, we get:

[0119]

[0120] In the formula, is the average current of all kingpin steering motors at time t, W is the total weight coefficient, d i is the distance from the center of gravity of the vehicle to the i-th axis, A1 is the weighted sum of the squares of the kingpin steering motor currents of all wheels, reflecting the fluctuation of the motor current, A2 is related to the average current and the current of each wheel, reflecting the degree of current deviation from the average value, and A3 is the weighted sum of the squares of the average current;

[0121] Step 3: Single wheel fault judgment based on K-means clustering algorithm. Suppose the parameter vector x of the jth wheel on the i-th axis is ij is a d-dimensional vector, then x ij =[θ ij (t),I mij (t),I hij (t),a yij (t),ω ij (t),T ij (t),Pij (t)], randomly select K initial cluster centers c k , for each sample point c k , use the Euclidean distance to calculate the distance from it to each cluster center, and then convert the sample x ij Assign to meet In the clustering, the cluster center is updated until the cluster center no longer changes significantly. The fault judgment method is: set the threshold β, when d ij,k ≥β and x ij When any of the parameters exceeds the normal range, it is judged that the relevant parts of the wheel are faulty;

[0122] Step 2: Failure redundancy control of the steer-by-wire system. The specific two steps in step 2 are as follows:

[0123] Link 1: Failure backup control algorithm for steer-by-wire system under sharp turning conditions. Assuming that the kingpin steering motor of the left wheel on the front axle fails, the ideal steering angle θ before failure of the front axle wheel is obtained based on the vehicle state input and the composite steering vehicle state model established in step 3 of the first step. wish , and then take the Ackerman steering relationship of the three-axle vehicle as the constraint to obtain the target turning angle θ of the right wheel when the left wheel on the front axle fails desire , the calculation of the compensation process is shown in equations (47)-(49):

[0124] θ desire =θ wish +Δθ (47)

[0125]

[0126] Where Δθ is the compensation angle, d is the steering center offset, y is the displacement of the center of mass, ω is the instantaneous yaw rate of the vehicle, l is the distance from the center of mass of the vehicle to the front axle, t is the small time period of the vehicle driving after the left wheel fails, and W is the wheelbase;

[0127] Link 2: Dual kingpin motor redundant control: When all kingpin steering motors fail under sharp turning conditions, the two backup kingpin steering motors of one axle are started to enter the front axle steering mode. The redundant control unit of the wire-controlled steering system updates the vehicle's linear two-degree-of-freedom model to only have the front wheel steering angle input and differential steering input, and considers the initial steering angles of the remaining wheels;

[0128] Step 3: Failure redundancy control of differential steering system. The specific two links in step 3 are as follows:

[0129] Link 1: Failure backup control algorithm for differential steering system under sharp turning conditions. Assuming that the left wheel hub motor of the front axle fails, the ideal differential torque of the front axle is obtained according to the vehicle state input and the composite steering vehicle state model established in step 3 of the first step, and the driving torque of the wheel on the other side is recalculated. The calculation of the compensation process is shown in equations (50)-(52):

[0130]

[0131] In the formula, α desired is the ideal angular acceleration, T left , T right is the ideal driving torque of the left and right wheels before failure, r is the wheel radius, is the target driving torque of the right wheel, ΔT is the compensation torque of the right wheel;

[0132] Step 2: Entering the pure wire-controlled steering mode, the redundant control unit of the differential steering system updates the vehicle's linear two-degree-of-freedom model to have only three-axis angle inputs and ignores the differential torque.

[0133] Beneficial effects of the present invention:

[0134] 1) The distributed drive vehicle compound steering control method based on reinforcement learning interactive MPC provided by the present invention obtains a reliable linear two-degree-of-freedom model under this state by comprehensively considering the driving state of the three-axis distributed drive vehicle entering the compound steering mode under the sharp turning condition and the situation that the load on one axle is large under the vehicle sharp turning condition, thereby realizing the establishment of an accurate model for the multi-axis distributed drive vehicle under the sharp turning condition;

[0135] 2) The distributed drive vehicle compound steering control method based on reinforcement learning interactive MPC provided by the present invention combines the advantages of precise wire-controlled steering and high stability of differential steering for distributed drive vehicles by designing an upper-level stable target state decision controller and introducing a proportional factor of wire-controlled steering and differential steering. By designing a lower-level MPC-based wire-controlled steering and differential steering coordination controller, the optimal stability adjustment of the vehicle under sharp turning conditions is achieved, greatly reducing the risk of rollover.

[0136] 3) The distributed drive vehicle compound steering control method based on reinforcement learning interactive MPC provided by the present invention optimizes the prediction time domain, control time domain, proportional factor of wire control steering and differential steering, upper and lower limits of wheel angle, and upper and lower limits of left and right wheel longitudinal force difference in the MPC module by constructing a compound steering system parameter optimizer based on LSTM and a four-dimensional vehicle stability reward function, combined with a deep deterministic policy gradient algorithm, thereby realizing the adaptive ability of the vehicle under variable environmental conditions and greatly improving the handling stability and adaptability of the vehicle when performing sharp turns under variable environments;

[0137] 4) The distributed drive vehicle compound steering control method based on reinforcement learning interactive MPC provided by the present invention constructs a kingpin steering motor steering system model and a hub motor model, analyzes the internal mechanism of the DC brushless motor and the permanent magnet synchronous motor from the perspective of electromechanics, and formulates a dual-system compensation control strategy under a sharp turn condition, which realizes the reasonable entry and exit of the kingpin steering motor and the hub motor at the beginning and end of the sharp turn condition, thereby improving the smoothness of the vehicle steering mode switching;

[0138] 5) The distributed drive vehicle compound steering control method based on reinforcement learning interactive MPC provided by the present invention comprehensively evaluates the compound steering system by establishing a multi-criteria fault diagnosis criterion for the compound steering system of a multi-axis electric drive vehicle; through the sudden turn condition fault judgment based on reinforcement learning, it is achieved that the vehicle can achieve optimal fault diagnosis based on experience in a changing environment; through the fault judgment based on the motor response consistency algorithm, the coordination of the multi-motor system is quantified; through the single-wheel fault judgment based on the K-means clustering algorithm, the problem that the fault forms of distributed drive vehicles are multiple and difficult to predict is solved, and the dependence on the priori fault model is broken, and the accurate and rapid fault location of a single wheel is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0139] Figure 1 The figure is a schematic diagram of the overall process of the distributed drive vehicle compound steering control method described in the present invention.

[0140] Figure 2 The figure is a schematic diagram of the overall logic of the distributed drive vehicle compound steering control method according to the present invention.

[0141] Figure 3 This is a logic block diagram of the MPC-based distributed drive vehicle steering control described in the present invention.

[0142] Figure 4 This is a logic block diagram of the composite steering control parameter optimization based on reinforcement learning described in the present invention.

[0143] Figure 5 This is a logic block diagram of the compensation control of the modular composite steering system described in the present invention.

[0144] Figure 6 The figure is a logic block diagram of redundant control under sudden turning conditions of the compound steering system described in the present invention.

[0145] Figure 7 This is a flow chart of the coordinated control algorithm of MPC-based steer-by-wire and differential steering described in the present invention.

[0146] Figure 8 It is a schematic diagram of the control parameter optimization algorithm flow of the present invention.

[0147] Fig. 9 It is a schematic diagram of the flow of the dual-system compensation control algorithm under the rapid turning condition described in the present invention.

[0148] Fig.10 It is a schematic diagram of the vehicle lateral acceleration curve according to the present invention.

[0149] Fig.11 This is the torque curve of the kingpin steering motor described in the present invention. DETAILED DESCRIPTION

[0150] See also Figures 1 to 11 As shown:

[0151] The distributed drive vehicle compound steering control method based on reinforcement learning interactive MPC provided by the present invention comprises the following steps:

[0152] Step 1: Distributed drive vehicle steering control based on MPC;

[0153] Step 2: Optimization of compound steering control parameters based on reinforcement learning;

[0154] Step 3: Modular composite steering system compensation control;

[0155] Step 4: Redundant control of compound steering system.

[0156] The process of MPC-based distributed drive vehicle steering control in the first step is as follows:

[0157] Step 1: Establish a linear two-degree-of-freedom model for compound steering of a three-axle vehicle. Considering that the load on the two steering wheels of the front axle is large under sharp turning conditions, the steering angle and tire lateral stiffness of the left and right tires of the front axle are taken into account to improve the accuracy of the model. Considering the influence of differential steering, the torque acting on each axle generated by differential steering and the additional lateral force caused by differential steering are taken into account. The obtained linear two-degree-of-freedom model is shown in the following equations (1) and (2):

[0158]

[0159] Where m is the total mass of the vehicle, a y is the lateral acceleration of the vehicle, k f1 , k f2 , k m , k r are the cornering stiffness of the left and right tires on the front axle, the tires on the middle axle, and the tires on the rear axle, δ f1 ,δ f2 ,δ m ,δ r is the steering angle of the left and right tires on the front axle, the tires on the intermediate axle, and the tires on the rear axle, v yis the lateral velocity of the vehicle, l f , l m , l r are the distances from the front axle, middle axle, and rear axle to the center of mass, ω r is the vehicle's yaw rate, u is the vehicle's forward speed, F fl 、F fr 、F ml 、F mr 、F rl 、F rr The driving forces of the front, middle and rear axles are the left and right wheels, d f d m d r They are the front axle, middle axle, and rear axle track respectively;

[0160] Step 2: Judgment of the limit of the rapid-turning condition and calculation of the target state. The specific three steps are as follows:

[0161] Step 1: Driver's intention to turn sharply is evaluated. The current vehicle speed and steering wheel angular velocity are used to determine whether the vehicle is in an emergency condition. The defined limits are shown in equations (3) and (4):

[0162] v≤v boundary ω≥ω low (3)

[0163] v≥v boundary ω≥ω high (4)

[0164] Where v is the longitudinal velocity of the vehicle, ω is the angular velocity of the steering wheel, and v boundary is the speed limit, ω log is the steering wheel angular velocity limit under low speed conditions, ω high It is the angular velocity limit of the steering wheel under high-speed conditions;

[0165] Step 2: Vehicle emergency status assessment: obtain the current ω through the vehicle yaw angular velocity sensor and lateral acceleration sensor r with a y , by constructing a stability index to comprehensively consider the impact of the two on vehicle stability, as shown in formula (5), and finally establish a functional relationship between the mode switching limit and the stability index, and define the stability index and mode switching limit function, as shown in formula (6):

[0166] S=k1|a y |+k2|ω r | (5)

[0167] L=A·e -B×S (6)

[0168] In the formula, S is the stability index, a yis the lateral acceleration, k1 and k2 are weight coefficients, and L is the mode switching limit;

[0169] Step 3: Design of stable target state decision controller. By defining the distribution ratio coefficient k between wire control steering and differential steering, the driver's steering wheel input is assigned tasks. The pure wire control steering module of the three-axis distributed drive vehicle is designed according to the steering task, taking into account the longitudinal speed of the vehicle and the Ackerman steering relationship to change the transmission ratio design and calculate the target turning angle δ of each wheel. f1 ,δ f2 ,δ m ,δ r The differential steering module performs hierarchical control according to the steering task, and the upper controller calculates the target yaw rate ω through the equivalent wire control steering model r , the dynamic output feedback controller is used to obtain the desired yaw moment, and the lower controller uses a direct allocation algorithm to obtain the target driving force difference F of each axis fl -F fr ,F ml -F mr ,F rl -F rr , where the design of the dynamic output feedback controller is shown in equations (7) and (8):

[0170]

[0171] In the formula, Indicates the controller status, A kij ,B kj ,C kj ,D k is the controller matrix to be designed, A k ,B k ,C k ,D k is the final controller matrix considering that the parameter h(t) varies with speed;

[0172] Step 3: Design of coordinated controller for steer-by-wire and differential steering based on MPC. The specific four steps of step 3 are as follows:

[0173] Step 1: State space modeling of the compound steering system. By discretizing the linear two-degree-of-freedom model of the three-axis vehicle compound steering in step 1 of the first step, the vehicle state vector is defined as:

[0174]

[0175] The control input vector of the compound steering system is defined as:

[0176] u k =[δ f1 δ f2 δm δ r F fl -F fr F ml -F mr F rl -F rr ] T

[0177] The obtained state space model is shown in formula (9):

[0178] x k+1 =Ax k +Bu k (9)

[0179] in:

[0180]

[0181] Step 2: Establishment of the performance index function for compound steering system prediction in the time domain optimization. By defining the compound steering system input increment and the vehicle augmented state vector Δu [k] =u [k] -u [k-1] With x a[k+1] =[x [k] x d[k] u [k-1] ], the state space equation of the compound steering system under the input increment and the performance index function of the compound steering system under the input increment are obtained, as shown in equations (10) and (11), respectively:

[0182] x a[k+1] =A a x a[k] +B a Δu [k] (10)

[0183] in:

[0184] B a =[B 0 n×p I p×p ] T

[0185]

[0186] Among them, A D is the transition matrix of the target state, I p×p is the unit matrix, S, Q, R are weight matrices;

[0187] k, k+1,...k+N pThe state space equations at each moment are iterated sequentially to obtain the prediction equation of the composite steering system:

[0188] X [k] =Φx [k|k] +ΓU [k] (12)

[0189] The vehicle performance index matrix is ​​defined as:

[0190]

[0191] At time k, the prediction interval will be N p When , the performance index function within this prediction interval is sorted out to obtain the prediction time domain optimization performance index function:

[0192]

[0193] Let F = Γ T ΩΦ,H=Γ T ΩΓ+Ψ, transforming the performance index function into a canonical form;

[0194] Step 3: Design of constraints for the composite steering system. Based on the motion characteristics of the three-axis distributed drive vehicle that can be controlled by the single wheel angle, the physical constraints of the system are constructed. The proportional relationship of the three-axis wheel angles is written as a set of linear equality constraints. The angle limit of the modular steering system and the difference limit of the wheel hub motor driving force are used as inequality constraints. The constraint matrix is ​​converted into a canonical form that can be solved by quadratic programming. The performance index function considering the system constraints is obtained, as shown in formula (14):

[0195]

[0196] Where H is a symmetric positive definite matrix consisting of the coefficients of the quadratic terms in the composite steering cost function, f is the coefficient of the linear term in the composite steering cost function, c is a constant, and δ min , δ max is the angle limit of the modular steering system, ΔF rmin , ΔF rmax It is the difference limit of the driving force of the hub motor;

[0197] By substituting the performance index function and constraint conditions (14) considering the system constraints into the quadratic programming solver, the optimal distribution of the control input vector can be obtained. The solver searches for the value that minimizes the objective function in the feasible domain. By performing such a solution process at each sampling time, real-time control of the composite steering allocation strategy based on quadratic programming can be achieved.

[0198] The process of optimizing the compound steering control parameters based on reinforcement learning in the second step is as follows:

[0199] Step 1: Establish vehicle status and select compound steering system actions. The vehicle status includes lateral acceleration, yaw rate, steering response time, steering motor energy consumption, vehicle current steering angle vector, and steering angle adjustment of each wheel. The compound steering system action selection includes prediction time domain, control time domain, allocation ratio coefficient of wire control steering and differential steering in step 2, link 3 of the first step, and angle limit of modular steering system and wheel hub motor driving force difference limit in step 3, link 4 of the first step;

[0200] S = {a y ,ω y ,t delay ,E consumed ,θ(t),Δθ(t)}

[0201] A={T p ,T c ,k,δ min ,δ max ,ΔF rmin ,ΔF rmax}

[0202] In the formula, S is the vehicle state, A is the steering system action, T p is the prediction time domain, T c To control the time domain;

[0203] Step 2: Design of the four-dimensional vehicle stability reward function, including lateral stability reward (15), yaw rate stability reward (16), steering response timeliness reward (17), and energy efficiency reward (18), as shown in the following formulas:

[0204]

[0205] R=ω1r1+ω2r2+ω3r3+ω4r4 (19)

[0206] Among them, a y is the actual lateral acceleration, a yd is the ideal lateral acceleration, a ymax is the maximum lateral acceleration threshold, w r is the actual yaw rate, w rd is the ideal yaw rate, w rmax is the maximum yaw rate threshold, t delay is the time delay, t delaymax is the maximum acceptable delay time, E consumed is the energy currently consumed, E max is the maximum energy consumption threshold, ω1, ω2, ω3, ω4 are weight coefficients;

[0207] Step 3: Design of composite steering system parameter optimizer based on LSTM. The specific two links in step 3 are as follows:

[0208] Step 1: Construction of the strategy network of the composite steering system parameter optimizer. The strategy network of the composite steering system parameter optimizer consists of an LSTM layer and a fully connected layer. The input signal is the current state s t , the LSTM layer processes the state sequence and outputs the hidden state h t , the fully connected layer maps the LSTM output to the dimension of the action space and calculates the mean (20), standard deviation (21), and deterministic action (22) of the action, as shown in the following formulas:

[0209] μ(s t )=W μ h t +b μ (20)

[0210] σ(s t )=W σ h t +b σ (twenty one)

[0211] s t =μ(s t )+∈⊙σ(s t ) (twenty two)

[0212] Where W μ ,W σ is the weight matrix, b μ ,b σ is the bias term, ⊙ is the element-wise multiplication, and ∈ is the random noise sampled from the standard normal distribution;

[0213] Step 2: Construction of the value network of the composite steering system parameter optimizer. The value network of the composite steering system parameter optimizer consists of an LSTM layer and a fully connected layer. The input signal is the current state s t With the current action a t , the LSTM layer processes the state sequence and outputs the hidden state h t , the input signal of the fully connected layer is the concatenated vector of state and action and the output of LSTM, and the output signal is the state value function estimation, as shown in formula (23):

[0214] Q(s t ,a t )=W Q ([s t ,a t ],h t )+b Q (twenty three)

[0215] Where WQ is the weight matrix, b Q is the bias term;

[0216] Step 4: Optimization of control parameters of the composite steering system based on the deep deterministic policy gradient algorithm. The specific four steps in step 4 are as follows:

[0217] Step 1: Establish and initialize the composite steering system parameter optimizer architecture, randomly initialize the composite steering system parameter optimizer strategy network in step 2, step 3, step 1, and the parameters θ of the composite steering system parameter optimizer value network in step 2, step 3, step 2 μ ,θ Q , copy and create a composite steering system parameter optimizer target strategy network and a composite steering system parameter optimizer target value network respectively;

[0218] Step 2: Calculation of target parameters and target parameter values. For each time step t, according to the current state s t , the action output of the compound steering system parameter optimizer strategy network is a t =μ(s t |θ u )+∈, where ∈ is noise, and performs action a t , get the vehicle state feedback s at the next moment t+1 , the reward value r is obtained from the four-dimensional vehicle stability reward function in step 2 of the second step t , store the transfer samples in the experience replay buffer D, and then randomly sample a batch of transfer samples from the experience replay buffer. The target parameters and target parameter value calculations are shown in (24) and (25) respectively:

[0219] a t+1 =μ'(s t+1 |θ μ′ ) (twenty four)

[0220] y t =r t +γQ'(s t+1 ,a t+1 |θ Q′ ) (25)

[0221] In the formula, γ is the discount factor;

[0222] Step 3: Update the network parameters of the composite steering system parameter optimizer based on the DDPG algorithm. The value network parameters of the composite steering system parameter optimizer in step 1 of step 4 of step 2 are updated by minimizing the loss function, as shown in formula (26). The sampled policy gradient is used to update the policy network parameters of the composite steering system parameter optimizer in step 1 of step 4 of step 2, as shown in formula (27):

[0223]

[0224] In the formula, L(θ Q ) is the loss function of the composite steering system parameter optimizer value network, y i is the target value, Q(s i ,a i |θ Q ) is the vehicle state s i Take parameter a i When, based on θ Q The value of is the parameter network parameter of the compound steering system optimizer strategy θ μ About J(θ μ ), u(s i |θ u ) is the μ Next, the vehicle state s i The action output by the system parameter optimizer strategy network;

[0225] Step 4: Soft update of the target composite steering system parameter optimizer network. As shown in formula (28):

[0226] θ μ′ ←τθ μ +(1-τ)θ μ′ ,θ Q′ ←τθ Q +(1-τ)θ Q′ (28)

[0227] Where τ is the soft update coefficient.

[0228] exist Fig.10 and Fig.11 An exemplary operation result of the second step, step four, is shown in FIG.

[0229] The process of the compensation control of the modular composite steering system in the third step is as follows:

[0230] Step 1: Kingpin motor steering system model construction. The specific two steps in step 1 are as follows:

[0231] Step 1: Establishment of the motion equation of the kingpin motor steering system. The kingpin steering motor uses a brushless DC motor. The voltage balance equation of the stator winding, the electromagnetic torque equation, and the mechanical motion equation of the steering system are established. The motion equation of the kingpin motor steering system is obtained through joint calculation, as shown in formula (29):

[0232]

[0233] in:

[0234] T e =K t i

[0235]

[0236] In the formula, U is the stator winding terminal voltage, R is the stator winding resistance, i is the winding current, L is the winding inductance, e is the back electromotive force, K t is the torque constant, J is the moment of inertia of the steering system, including the motor rotor and the kingpin, B is the damping coefficient of the steering system, T r is the electromagnetic torque, T r is the steering resistance torque, θ is the rotation angle of the motor output shaft, and ω is the angular velocity of the motor output shaft;

[0237] Step 2: Establish the state space equation of the kingpin motor steering system. Define the state variable: x = [θ, ω, i] T , input variable: u = [U], output variable y = [θ, nθ], the state equation is shown in formula (30):

[0238]

[0239] in:

[0240] Step 2: Construct the wheel hub motor differential steering system model. The motion equation of the wheel hub motor can be obtained through the voltage equation and electromagnetic torque equation of the wheel hub motor, as shown in formula (31). Similarly, its state space equation can be obtained.

[0241]

[0242] Where, L d , L q are the d-axis and q-axis inductances, respectively, and i d 、i q are the d-axis and q-axis stator currents, ω e is the electrical angular velocity, ψ f is the permanent magnet flux. J is the moment of inertia of the hub motor and related rotating parts, B is the viscous damping coefficient of the rotating parts, T L is the load torque;

[0243] Step 3: Dual system compensation control under emergency conditions. The specific four links in step 3 are as follows:

[0244] Link 1: The kingpin steering motor enters control. In the time period [0-t1], an exponential function steering angle command is set to make the current rise quickly, so that the kingpin steering motor can respond quickly and the steering angle can be established quickly. The steering angle command and the kingpin steering motor control equation are shown in equations (32) and (33) respectively:

[0245] θ d (t) = θ0(1-e -kt ) (32)

[0246]

[0247] Where θ0 is the maximum steering angle required for a sharp turn, k is a constant that determines the speed at which the kingpin steering motor enters the sharp turn state, and t is a time variable;

[0248] Step 2: Hub motor entry control. [0~t2] is the time interval when the hub motor enters rapid rotation. The target yaw rate segmented function is used to make the hub motor enter the working state relatively quickly. The yaw rate command and the hub motor differential control equation are shown in equations (34) and (35) respectively:

[0249]

[0250] Where θ0 is the maximum steering angle required for a sharp turn, k is a constant that determines the speed at which the kingpin steering motor enters the sharp turn state, and t is a time variable;

[0251] Link 3: Wheel hub motor exit control. At the end of the sharp turn, the vehicle's driving state gradually stabilizes, and the yaw rate and lateral acceleration approach normal driving values. Therefore, the output of the wheel hub motor's differential torque attenuation speed function adopts a sinusoidal decreasing form, and the output torque is stopped first. The process takes tens of milliseconds to one hundred milliseconds. The wheel hub motor exit control method is shown in formula (36):

[0252]

[0253] Where t is the time variable, [t5, t6] is the time interval when the wheel hub motor exits;

[0254] Link 4: Kingpin steering motor exit control. When the sharp turn is about to end, the driver gradually returns the steering wheel to the center. However, due to the inertia of the vehicle during the sharp turn and the mechanical return characteristics of the steering system, the kingpin steering motor continues to adjust the output torque while the steering angle of the vehicle gradually returns to the normal driving state to ensure the smoothness of the steering. Therefore, the steering torque attenuation speed function should adopt a quadratic decreasing form. The whole process lasts for several hundred milliseconds to longer. The kingpin steering motor exit control method is shown in equations (37)-(40):

[0255]

[0256] Wherein, [t3, t4] is the time interval when the kingpin steering motor exits.

[0257] The process of redundant control of the composite steering system in the fourth step is as follows:

[0258] Step 1: Multi-criteria fault diagnosis of compound steering system of multi-axis electric drive vehicle. The three specific steps in step 1 are as follows:

[0259] Link 1: Fault judgment of sudden turning condition based on reinforcement learning. According to the state space and action space defined in step 1 of the second step, the current steering angle vector of the vehicle θ(t) = [θ 11 (t),θ 12 (t),…θ nm (t)] and the steering angle adjustment of each wheel Δθ(t). According to the stability reward function defined in step 2 of the second step, the reward item related to the vehicle lateral acceleration is already included. The fault judgment algorithm is shown in formula (41):

[0260]

[0261] In the formula, a y (t) is the ideal lateral acceleration calculated based on reinforcement learning, Δa y is the lateral acceleration deviation limit value;

[0262] Step 2: Fault judgment based on motor response consistency algorithm, by defining the consistency index η of the kingpin steering motor m (t) and weight coefficient w ij , as shown in equations (42) and (43), and then the fault diagnosis is performed through the consistency of each motor by comparing the set consistency threshold. The consistency index calculation is shown in equation (46):

[0263]

[0264] By expanding the consistency index, we get:

[0265]

[0266] In the formula, is the average current of all kingpin steering motors at time t, W is the total weight coefficient, d i is the distance from the center of gravity of the vehicle to the i-th axis, A1 is the weighted sum of the squares of the kingpin steering motor currents of all wheels, reflecting the fluctuation of the motor current, A2 is related to the average current and the current of each wheel, reflecting the degree of current deviation from the average value, and A3 is the weighted sum of the squares of the average current;

[0267] Step 3: Single wheel fault judgment based on K-means clustering algorithm. Suppose the parameter vector x of the jth wheel on the i-th axis is ij is a d-dimensional vector, then x ij =[θ ij (t),I mij (t),Ihij (t),a yij (t),ω ij (t),T ij (t),P ij (t)], randomly select K initial cluster centers c k , for each sample point c k , use the Euclidean distance to calculate the distance from it to each cluster center, and then convert the sample x ij Assign to meet In the clustering, the cluster center is updated until the cluster center no longer changes significantly. The fault judgment method is: set the threshold β, when d ij,k ≥β and x ij When any of the parameters exceeds the normal range, it is judged that the relevant parts of the wheel are faulty;

[0268] Step 2: Failure redundancy control of the steer-by-wire system. The specific two steps in step 2 are as follows:

[0269] Link 1: Failure backup control algorithm for steer-by-wire system under sharp turning conditions. Assuming that the kingpin steering motor of the left wheel on the front axle fails, the ideal steering angle θ before failure of the front axle wheel is obtained based on the vehicle state input and the composite steering vehicle state model established in step 3 of the first step. wish , and then take the Ackerman steering relationship of the three-axle vehicle as the constraint to obtain the target turning angle θ of the right wheel when the left wheel on the front axle fails desire , the calculation of the compensation process is shown in equations (47)-(49):

[0270] θ desire =θ wish +Δθ (47)

[0271]

[0272] Where Δθ is the compensation angle, d is the steering center offset, y is the displacement of the center of mass, ω is the instantaneous yaw rate of the vehicle, l is the distance from the center of mass of the vehicle to the front axle, t is the small time period of the vehicle driving after the left wheel fails, and W is the wheelbase;

[0273] Link 2: Dual kingpin motor redundant control: When all kingpin steering motors fail under sharp turning conditions, the two backup kingpin steering motors of one axle are started to enter the front axle steering mode. The redundant control unit of the wire-controlled steering system updates the vehicle's linear two-degree-of-freedom model to only have the front wheel steering angle input and differential steering input, and considers the initial steering angles of the remaining wheels;

[0274] Step 3: Failure redundancy control of differential steering system. The specific two links in step 3 are as follows:

[0275] Link 1: Failure backup control algorithm for differential steering system under sharp turning conditions. Assuming that the left wheel hub motor of the front axle fails, the ideal differential torque of the front axle is obtained according to the vehicle state input and the composite steering vehicle state model established in step 3 of the first step, and the driving torque of the wheel on the other side is recalculated. The calculation of the compensation process is shown in equations (50)-(52):

[0276]

[0277]

[0278] In the formula, α desired is the ideal angular acceleration, T left , T right is the ideal driving torque of the left and right wheels before failure, r is the wheel radius, is the target driving torque of the right wheel, ΔT is the compensation torque of the right wheel;

[0279] Step 2: Entering the pure wire-controlled steering mode, the redundant control unit of the differential steering system updates the vehicle's linear two-degree-of-freedom model to have only three-axis angle inputs and ignores the differential torque.

Claims

1. A distributed drive vehicle compound steering control method based on reinforcement learning interactive MPC, characterized by: The method comprises the following steps: Step 1: Distributed drive vehicle steering control based on MPC; Step 2: Optimization of compound steering control parameters based on reinforcement learning; Step 3: Modular composite steering system compensation control; Step 4: Redundant control of compound steering system.

2. The distributed drive vehicle compound steering control method based on reinforcement learning interactive MPC according to claim 1 is characterized in that: The process of the MPC-based distributed drive vehicle steering control in the first step is as follows: Step 1: Establish a linear two-degree-of-freedom model for compound steering of a three-axle vehicle. Considering that the load on the two steering wheels of the front axle is large under sharp turning conditions, the steering angle and tire lateral stiffness of the left and right tires of the front axle are taken into account to improve the accuracy of the model. Considering the influence of differential steering, the torque acting on each axle generated by differential steering and the additional lateral force caused by differential steering are taken into account. The obtained linear two-degree-of-freedom model is shown in the following equations (1) and (2): Where m is the total mass of the vehicle, a y is the lateral acceleration of the vehicle, k f1 , k f2 , k m , k r are the cornering stiffness of the left and right tires on the front axle, the tires on the middle axle, and the tires on the rear axle, δ f1 , δ f2 , δ m , δ r is the steering angle of the left and right tires on the front axle, the tires on the intermediate axle, and the tires on the rear axle, v y is the lateral velocity of the vehicle, l f , l m , l r are the distances from the front axle, middle axle, and rear axle to the center of mass, ω r is the vehicle's yaw rate, u is the vehicle's forward speed, F fl 、F fr 、F ml 、F mr 、F rl 、F rr The driving forces of the front, middle and rear axles are the left and right wheels, d f ,d m ,d r They are the front axle, middle axle, and rear axle track respectively; Step 2: Judgment of the limit of the rapid-turning condition and calculation of the target state. The specific three steps are as follows: Step 1: Driver's intention to turn sharply is evaluated. The current vehicle speed and steering wheel angular velocity are used to determine whether the vehicle is in an emergency condition. The defined limits are shown in equations (3) and (4): v≤v boundary ω≥ω low (3) v≥v boundary ω≥ω h igh (4) Where v is the longitudinal velocity of the vehicle, ω is the angular velocity of the steering wheel, and v boundary is the speed limit, ω low is the steering wheel angular velocity limit under low speed conditions, ω h igh It is the angular velocity limit of the steering wheel under high-speed conditions; Step 2: Vehicle emergency status assessment: obtain the current ω through the vehicle yaw angular velocity sensor and lateral acceleration sensor r with a y , by constructing a stability index to comprehensively consider the impact of the two on vehicle stability, as shown in formula (5), and finally establish a functional relationship between the mode switching limit and the stability index, and define the stability index and mode switching limit function, as shown in formula (6): S=k1|a y |+k2|ω r | (5) L=A、e -B×S (6) In the formula, S is the stability index, a y is the lateral acceleration, k1 and k2 are weight coefficients, and L is the mode switching limit; Step 3: Design of stable target state decision controller. By defining the distribution ratio coefficient k between wire control steering and differential steering, the driver's steering wheel input is assigned tasks. The pure wire control steering module of the three-axis distributed drive vehicle is designed according to the steering task, taking into account the longitudinal speed of the vehicle and the Ackerman steering relationship to change the transmission ratio design and calculate the target turning angle δ of each wheel. f1 ,δ f2 ,δ m ,δ r The differential steering module performs hierarchical control according to the steering task, and the upper controller calculates the target yaw rate ω through the equivalent wire control steering model r , the dynamic output feedback controller is used to obtain the desired yaw moment, and the lower controller uses a direct allocation algorithm to obtain the target driving force difference F of each axis fl -F fr ,F ml -F mr ,F rl -F rr , where the design of the dynamic output feedback controller is shown in equations (7) and (8): In the formula, Indicates the controller status, A kij ,B kj ,C kj ,D k is the controller matrix to be designed, A k ,B k ,C k ,D k is the final controller matrix considering that the parameter h(t) varies with speed; Step 3: Design of coordinated controller for steer-by-wire and differential steering based on MPC. The specific four steps of step 3 are as follows: Step 1: State space modeling of the compound steering system. By discretizing the linear two-degree-of-freedom model of the three-axis vehicle compound steering in step 1 of the first step, the vehicle state vector is defined as: The control input vector of the compound steering system is defined as: you k =[δ f1 d f2 d m d r F fl -F fr F ml -F mr F rl -F rr ] T The obtained state space model is shown in formula (9): x k+1 =Ax k +Bu k (9) in: Step 2: Establishment of the performance index function for compound steering system prediction in the time domain optimization. By defining the compound steering system input increment and the vehicle augmented state vector Δu [k] =u [k] -u [k-1] With x a[k+1] =[x d[k] x d[k] u [k-1] ], the state space equation of the compound steering system under the input increment and the performance index function of the compound steering system under the input increment are obtained, as shown in equations (10) and (11), respectively: x a[k+1] =A a x a[k] +B a Δu [k] (10) in: B a =[B 0 n×p I p×p ] T Among them, A D is the transition matrix of the target state, I p×p is the unit matrix, S, Q, R are weight matrices; k, k+1,...k+N p The state space equations at each moment are iterated sequentially to obtain the prediction equation of the composite steering system: X [k] =Φ x[k|k] +G U [k] (12) The vehicle performance index matrix is ​​defined as: At time k, the prediction interval will be N p When , the performance index function within this prediction interval is sorted out to obtain the prediction time domain optimization performance index function: Let F = Γ T ΩΦ,H=Γ T ΩΓ+Ψ, transforming the performance index function into a canonical form; Step 3: Design of constraints for the composite steering system. Based on the motion characteristics of the three-axis distributed drive vehicle that can be controlled by the single wheel angle, the physical constraints of the system are constructed. The proportional relationship of the three-axis wheel angles is written as a set of linear equality constraints. The angle limit of the modular steering system and the difference limit of the wheel hub motor driving force are used as inequality constraints. The constraint matrix is ​​converted into a canonical form that can be solved by quadratic programming. The performance index function considering the system constraints is obtained, as shown in formula (14): Where H is a symmetric positive definite matrix consisting of the coefficients of the quadratic terms in the composite steering cost function, f is the coefficient of the linear term in the composite steering cost function, c is a constant, and δ min , δ max is the angle limit of the modular steering system, ΔF rmin , ΔF rmax It is the difference limit of the driving force of the hub motor; By substituting the performance index function and constraint conditions (14) considering the system constraints into the quadratic programming solver, the optimal distribution of the control input vector can be obtained. The solver searches for the value that minimizes the objective function in the feasible domain. By performing such a solution process at each sampling time, real-time control of the composite steering allocation strategy based on quadratic programming can be achieved.

3. The distributed drive vehicle compound steering control method based on reinforcement learning interactive MPC according to claim 1 is characterized in that: The process of optimizing the composite steering control parameters based on reinforcement learning in the second step is as follows: Step 1: Establish vehicle status and select compound steering system actions. The vehicle status includes lateral acceleration, yaw rate, steering response time, steering motor energy consumption, vehicle current steering angle vector, and steering angle adjustment of each wheel. The compound steering system action selection includes prediction time domain, control time domain, allocation ratio coefficient of wire control steering and differential steering in step 2, link 3 of the first step, and angle limit of modular steering system and wheel hub motor driving force difference limit in step 3, link 4 of the first step; S={a y ,oh y ,t delay ,E consumed ,θ(t),Δθ(t)} A={T p ,T c ,k,δ min ,δ max ,ΔF rmin ,ΔF rmax } In the formula, S is the vehicle state, A is the steering system action, T p is the prediction time domain, T c To control the time domain; Step 2: Design of the four-dimensional vehicle stability reward function, including lateral stability reward (15), yaw rate stability reward (16), steering response timeliness reward (17), and energy efficiency reward (18), as shown in the following formulas: R=ω1r1+ω2r2+ω3r3+ω4r4 (19) Among them, a y is the actual lateral acceleration, a yd is the ideal lateral acceleration, a ymax is the maximum lateral acceleration threshold, w r is the actual yaw rate, w rd is the ideal yaw rate, w rmax is the maximum yaw rate threshold, t delay is the time delay, t dekatnax is the maximum acceptable delay time, E consumed is the energy currently consumed, E max is the maximum energy consumption threshold, ω1, ω2, ω3, ω4 are weight coefficients; Step 3: Design of composite steering system parameter optimizer based on LSTM. The specific two links in step 3 are as follows: Step 1: Construction of the strategy network of the composite steering system parameter optimizer. The strategy network of the composite steering system parameter optimizer consists of an LSTM layer and a fully connected layer. The input signal is the current state s t , the LSTM layer processes the state sequence and outputs the hidden state h t , the fully connected layer maps the LSTM output to the dimension of the action space and calculates the mean (20), standard deviation (21), and deterministic action (22) of the action, as shown in the following formulas: μ(s t )=W μ h t +b μ (20) σ(s t )=W σ h t +b σ (21) a t =μ(s t )+∈⊙σ(s t ) (22) Where W μ ,W σ is the weight matrix, b μ ,b σ is the bias term, ⊙ is the element-wise multiplication, and ∈ is the random noise sampled from the standard normal distribution; Step 2: Construction of the value network of the composite steering system parameter optimizer. The value network of the composite steering system parameter optimizer consists of an LSTM layer and a fully connected layer. The input signal is the current state s t With the current action a t , the LSTM layer processes the state sequence and outputs the hidden state h t , the input signal of the fully connected layer is the concatenated vector of state and action and the output of LSTM, and the output signal is the state value function estimation, as shown in formula (23): Q(s t ,a t )=W Q ([s t ,s t ],h t )+b Q (23) Where W Q is the weight matrix, b Q is the bias term; Step 4: Optimization of control parameters of the composite steering system based on the deep deterministic policy gradient algorithm. The specific four steps in step 4 are as follows: Step 1: Establish and initialize the composite steering system parameter optimizer architecture, randomly initialize the composite steering system parameter optimizer strategy network in step 2, step 3, step 1, and the parameters θ of the composite steering system parameter optimizer value network in step 2, step 3, step 2 μ ,θ Q , copy and create a composite steering system parameter optimizer target strategy network and a composite steering system parameter optimizer target value network respectively; Step 2: Calculation of target parameters and target parameter values. For each time step t, according to the current state s t , the action output of the compound steering system parameter optimizer strategy network is a t =μ(s t |θ u )+∈, where ∈ is noise, and performs action a t , get the vehicle state feedback s at the next moment t+1 , the reward value r is obtained from the four-dimensional vehicle stability reward function in step 2 of the second step t , store the transfer samples in the experience replay buffer D, and then randomly sample a batch of transfer samples from the experience replay buffer. The target parameters and target parameter value calculations are shown in (24) and (25) respectively: a t+1 =μ'(s t+1 |θ μ ′) (24) y t =r t +γQ'(s t+1 ,a t+1 |θ Q ′) (25) In the formula, γ is the discount factor; Step 3: Update the network parameters of the composite steering system parameter optimizer based on the DDPG algorithm. The value network parameters of the composite steering system parameter optimizer in step 1 of step 4 of step 2 are updated by minimizing the loss function, as shown in formula (26). The sampled policy gradient is used to update the policy network parameters of the composite steering system parameter optimizer in step 1 of step 4 of step 2, as shown in formula (27): In the formula, L(θ Q ) is the loss function of the composite steering system parameter optimizer value network, y i is the target value, Q(s i ,a i |θ Q ) is the vehicle state s i Take parameter a i When, based on θ Q The value of is the parameter network parameter of the compound steering system optimizer strategy θ μ About J(θ μ ), u(s i |θ u ) is the μ Next, the vehicle state s i The action output by the system parameter optimizer strategy network; Step 4: Soft update of the target composite steering system parameter optimizer network, as shown in formula (28): i μ′ ←tth μ +(1-τ)θ μ′ ,i Q′ ←tth Q +(1-τ)θ Q′ (28) Where τ is the soft update coefficient.

4. The distributed drive vehicle compound steering control method based on reinforcement learning interactive MPC according to claim 1 is characterized in that: The process of the compensation control of the modular composite steering system in the third step is as follows: Step 1: Kingpin motor steering system model construction. The specific two steps in step 1 are as follows: Step 1: Establishment of the motion equation of the kingpin motor steering system. The kingpin steering motor uses a brushless DC motor. The voltage balance equation of the stator winding, the electromagnetic torque equation, and the mechanical motion equation of the steering system are established. The motion equation of the kingpin motor steering system is obtained through joint calculation, as shown in formula (29): in: T e =K t i In the formula, U is the stator winding terminal voltage, R is the stator winding resistance, i is the winding current, L is the winding inductance, e is the back electromotive force, K t is the torque constant, J is the moment of inertia of the steering system, including the motor rotor and the kingpin, B is the damping coefficient of the steering system, T r is the electromagnetic torque, T r is the steering resistance torque, θ is the rotation angle of the motor output shaft, and ω is the angular velocity of the motor output shaft; Step 2: Establish the state space equation of the kingpin motor steering system and define the state variables: x = [θ, ω, i] T , input variable: u = [U], output variable y = [θ, nθ], the state equation is shown in formula (30): in: Step 2: Construct the wheel hub motor differential steering system model. The motion equation of the wheel hub motor can be obtained through the voltage equation and electromagnetic torque equation of the wheel hub motor, as shown in formula (31). Similarly, its state space equation can be obtained. Where, L d , L q are the d-axis and q-axis inductances, respectively, and i d 、i q are the d-axis and q-axis stator currents, ω e is the electrical angular velocity, ψ f is the permanent magnet flux, J is the moment of inertia of the hub motor and related rotating parts, B is the viscous damping coefficient of the rotating parts, T L is the load torque; Step 3: Dual system compensation control under emergency conditions. The specific four links in step 3 are as follows: Link 1: The kingpin steering motor enters control. In the time period [0-t1], an exponential function steering angle command is set to make the current rise quickly, so that the kingpin steering motor can respond quickly and the steering angle can be established quickly. The steering angle command and the kingpin steering motor control equation are shown in equations (32) and (33) respectively: i d (t)=θ0(1-e -kt ) (32) Where θ0 is the maximum steering angle required for a sharp turn, k is a constant that determines the speed at which the kingpin steering motor enters the sharp turn state, and t is a time variable; Step 2: Hub motor entry control. [0~t2] is the time interval when the hub motor enters rapid rotation. The target yaw rate segmented function is used to make the hub motor enter the working state relatively quickly. The yaw rate command and the hub motor differential control equation are shown in equations (34) and (35) respectively: Where θ0 is the maximum steering angle required for a sharp turn, k is a constant that determines the speed at which the kingpin steering motor enters the sharp turn state, and t is a time variable; Link 3: Wheel hub motor exit control. At the end of the sharp turn, the vehicle's driving state gradually stabilizes, and the yaw rate and lateral acceleration approach normal driving values. Therefore, the output of the wheel hub motor's differential torque attenuation speed function adopts a sinusoidal decreasing form, and the output torque is stopped first. The process takes tens of milliseconds to one hundred milliseconds. The wheel hub motor exit control method is shown in formula (36): Where t is the time variable, [t5, t6] is the time interval when the wheel hub motor exits; Link 4: Kingpin steering motor exit control. When the sharp turn is about to end, the driver gradually returns the steering wheel to the center. However, due to the inertia of the vehicle during the sharp turn and the mechanical return characteristics of the steering system, the kingpin steering motor continues to adjust the output torque while the steering angle of the vehicle gradually returns to the normal driving state to ensure the smoothness of the steering. Therefore, the steering torque attenuation speed function should adopt a quadratic decreasing form. The whole process lasts for several hundred milliseconds to longer. The kingpin steering motor exit control method is shown in equations (37)-(40): Wherein, [t3, t4] is the time interval when the kingpin steering motor exits.

5. The distributed drive vehicle compound steering control method based on reinforcement learning interactive MPC according to claim 1 is characterized by: The process of redundant control of the composite steering system in the fourth step is as follows: Step 1: Multi-criteria fault diagnosis of compound steering system of multi-axis electric drive vehicle. The three specific steps in step 1 are as follows: Link 1: Fault judgment of sudden turning condition based on reinforcement learning. According to the state space and action space defined in step 1 of the second step, the current steering angle vector of the vehicle θ(t) = [θ 11 (t),θ 12 (t),…θ nm (t)] and the steering angle adjustment of each wheel Δθ(t). According to the stability reward function defined in step 2 of the second step, the reward item related to the vehicle lateral acceleration is already included. The fault judgment algorithm is shown in formula (41): In the formula, a y (t) is the ideal lateral acceleration calculated based on reinforcement learning, Δa y is the lateral acceleration deviation limit value; Step 2: Fault judgment based on motor response consistency algorithm, by defining the consistency index η of the kingpin steering motor m (t) and weight coefficient w ij , as shown in equations (42) and (43), and then the fault diagnosis is performed through the consistency of each motor by comparing the set consistency threshold. The consistency index calculation is shown in equation (46): By expanding the consistency index, we get: In the formula, is the average current of all kingpin steering motors at time t, W is the total weight coefficient, d i is the distance from the center of gravity of the vehicle to the i-th axis, A1 is the weighted sum of the squares of the kingpin steering motor currents of all wheels, reflecting the fluctuation of the motor current, A2 is related to the average current and the current of each wheel, reflecting the degree of current deviation from the average value, and A3 is the weighted sum of the squares of the average current; Step 3: Single wheel fault judgment based on K-means clustering algorithm. Suppose the parameter vector x of the jth wheel on the i-th axis is ij is a d-dimensional vector, then x ij =[θ ij (t),I mij (t),I hij (t),a yij (t),ω ij (t),T ij (t),P ij (t)], randomly select K initial cluster centers c k , for each sample point c k , use the Euclidean distance to calculate the distance from it to each cluster center, and then convert the sample x ij Assign to meet In the clustering, the cluster center is updated until the cluster center no longer changes significantly. The fault judgment method is: set the threshold β, when d ij,k ≥β and x ij When any of the parameters exceeds the normal range, it is judged that the relevant parts of the wheel are faulty; Step 2: Failure redundancy control of the steer-by-wire system. The specific two steps in step 2 are as follows: Link 1: Failure backup control algorithm for steer-by-wire system under sharp turning conditions. Assuming that the kingpin steering motor of the left wheel on the front axle fails, the ideal steering angle θ before failure of the front axle wheel is obtained based on the vehicle state input and the composite steering vehicle state model established in step 3 of the first step. wish , and then take the Ackerman steering relationship of the three-axle vehicle as the constraint to obtain the target turning angle θ of the right wheel when the left wheel on the front axle fails desire , the calculation of the compensation process is shown in equations (47)-(49): i desire =θ wish +Δθ (47) Where Δθ is the compensation angle, d is the steering center offset, y is the displacement of the center of mass, ω is the instantaneous yaw rate of the vehicle, l is the distance from the center of mass of the vehicle to the front axle, t is the small time period of the vehicle driving after the left wheel fails, and W is the wheelbase; Link 2: Dual kingpin motor redundant control: When all kingpin steering motors fail under sharp turning conditions, the two backup kingpin steering motors of one axle are started to enter the front axle steering mode. The redundant control unit of the wire-controlled steering system updates the vehicle's linear two-degree-of-freedom model to only have the front wheel steering angle input and differential steering input, and considers the initial steering angles of the remaining wheels; Step 3: Failure redundancy control of differential steering system. The specific two links in step 3 are as follows: Link 1: Failure backup control algorithm for differential steering system under sharp turning conditions. Assuming that the left wheel hub motor of the front axle fails, the ideal differential torque of the front axle is obtained according to the vehicle state input and the composite steering vehicle state model established in step 3 of the first step, and the driving torque of the wheel on the other side is recalculated. The calculation of the compensation process is shown in equations (50)-(52): In the formula, α desired is the ideal angular acceleration, T left , T righ t is the ideal driving torque of the left and right wheels before failure, r is the wheel radius, is the target driving torque of the right wheel, ΔT is the compensation torque of the right wheel; Step 2: Entering the pure wire-controlled steering mode, the redundant control unit of the differential steering system updates the vehicle's linear two-degree-of-freedom model to have only three-axis angle inputs and ignores the differential torque.

Images