A method and device for optimizing transmission ratio control of a steer-by-wire system

Abstract

The present invention discloses a method and device for optimizing the transmission ratio control of a wire-controlled steering system, comprising: obtaining vehicle parameters of a vehicle when traveling on a road; preprocessing the vehicle parameters to obtain preprocessed vehicle parameters; inputting the preprocessed vehicle parameters into a pre-trained deep reinforcement learning algorithm to obtain a real-time wire-controlled steering transmission ratio, and adjusting the value of the wire-controlled steering system in real time based on the calculated transmission ratio value. The present invention determines a basic transmission ratio selection range based on a traditional vehicle dynamics algorithm, and then performs optimization learning based on the vehicle state based on a deep learning reinforcement algorithm, dynamically adjusting the steering transmission ratio, and optimizing the performance of the wire-controlled steering system so that the driver can have the best experience regardless of whether the vehicle is in various states such as low-speed turning, high-speed turning, empty or fully loaded.

Description

Technical Field

[0001] The present invention relates to a transmission ratio optimization control method and device for a wire-controlled steering system, belonging to the technical field of vehicle control. Background Art

[0002] The steer-by-wire system has gradually begun to be installed in actual vehicles. In the steer-by-wire system, the adjustment of the steering ratio has the greatest impact on the handling performance.

[0003] Since the steer-by-wire system eliminates the mechanical control connection between the steering wheel and the steering wheel, the entire steering transmission ratio can be freely adjusted by software. However, the steering transmission ratio has a very direct impact on the stability and maneuverability of the vehicle. Currently, it is more common to use preset different transmission ratios for manual selection or automatic selection at a certain speed. This lacks personalization and makes it difficult to take advantage of the stepless adjustment of the transmission ratio. Summary of the Invention

[0004] The purpose of the present invention is to overcome the deficiencies in the prior art and to provide a method and device for optimizing the transmission ratio of a steer-by-wire system to automatically select the steering transmission ratio so as to optimize the vehicle's handling performance and stability.

[0005] To achieve the above object, the present invention is implemented by adopting the following technical solutions:

[0006] In a first aspect, the present invention provides a method for optimizing the transmission ratio of a steer-by-wire system, comprising:

[0007] Obtain vehicle parameters when the car is driving on the road;

[0008] Preprocessing the vehicle parameters to obtain preprocessed vehicle parameters;

[0009] The preprocessed vehicle parameters are input into a pre-trained deep reinforcement learning algorithm to obtain a real-time steer-by-wire transmission ratio, and the value of the steer-by-wire system is adjusted in real time based on the calculated transmission ratio.

[0010] Furthermore, the vehicle parameters include any one or more of vehicle speed, steering wheel angle, steering wheel angular velocity, vehicle body pitch angle, vehicle body yaw angular velocity, vehicle body roll angle, front axle ground clearance, and rear axle ground clearance.

[0011] Furthermore, the preprocessing of the vehicle parameters includes:

[0012] Filter each vehicle parameter to eliminate noise;

[0013] The various vehicle parameters are compared with each other to determine the availability of the parameters. For each parameter, the corresponding range limit is set. If it is exceeded, an alarm will be issued and the function will be downgraded to the initially set transmission ratio value.

[0014] Furthermore, the training of the deep reinforcement learning algorithm includes:

[0015] Obtain vehicle parameters when the car is driving on the road;

[0016] Preprocessing the vehicle parameters to obtain preprocessed vehicle parameters;

[0017] Using the preprocessed vehicle parameters as input to a deep reinforcement learning algorithm to obtain an optimal transmission ratio;

[0018] The optimal transmission ratio obtained by the deep reinforcement learning algorithm is used as the input signal of the steer-by-wire system. As the car continues to move, the steer-by-wire system inputs the transmission ratio according to the value calculated by the algorithm and obtains feedback values ​​of vehicle parameters.

[0019] The feedback value is pre-processed and used as input to the deep reinforcement learning algorithm again; multiple cycles are performed, and data on vehicle parameters is collected during each cycle, and the data is used to train the deep reinforcement learning algorithm;

[0020] The above steps are iterated multiple times to obtain a trained deep reinforcement learning algorithm; wherein the output of the deep reinforcement learning algorithm is the transmission ratio of the wire-controlled steering system.

[0021] Furthermore, the implementation steps of the deep reinforcement learning algorithm include:

[0022] Algorithm step 1: Use formula (1) to define the vehicle state parameter v of the deep reinforcement learning method:

[0023] v={w,α,β,wd,αd,βd} (1)

[0024] Wherein, wd is the ideal yaw rate; w is the measured yaw rate; αd is the ideal body roll angle; α is the measured body roll angle; βd is the ideal body pitch angle; β is the measured body pitch angle;

[0025] Algorithm step 2: Use formula (2) to define the action parameter a of the deep reinforcement learning method:

[0026] a={γ} (2)

[0027] Where: γ is the angular transmission ratio of the steer-by-wire system, unitless;

[0028] Algorithm step 3: Use formula (3) to establish the reward function r of the deep reinforcement learning method:

[0029] r=rβ+rw+rα+rt (3)

[0030] Among them: rβ is the vehicle body pitch angle reward function, rw is the vehicle body yaw angular velocity reward function, rα is the vehicle body roll angle reward function, and rt is the lateral acceleration response time;

[0031] Algorithm step 4: Construct a computational model for deep reinforcement learning methods;

[0032] Algorithm step 5: Generate N vehicle data samples using the initial data;

[0033] Algorithm step 6: Use the N vehicle data samples obtained in step 5 to train the model of the deep reinforcement learning method to obtain the optimized neural network model;

[0034] Algorithm step 7: During vehicle operation, the vehicle's current state parameter vt is measured in real time, and the state parameter is input into the optimal neural network model obtained above, which directly outputs the most optimized wire-controlled steering gear ratio under the current state.

[0035] Furthermore, the reward function is set to three priorities. The higher the priority, the more important the principle. The design principles are:

[0036] Level 1: Ensures that the yaw rate control and lateral acceleration response time are within the set range;

[0037] Level 2: Ensure that the roll control of the vehicle body is higher than the pitch control of the vehicle body;

[0038] Level 3: Ensure that the vehicle's pitch is within the set range;

[0039] Among them, the body roll angle reward function rα is shown in formula (4), corresponding to the level 2 design principle, the formula is as follows:

[0040]

[0041] Wherein, roll is the vehicle body pitch angle; Δroll = |roll – roll_normal|, which represents the change in the vehicle body pitch angle; roll_normal represents the vehicle body roll angle on a flat road; max_pitch is the maximum value of Δpitch, roll_th is the threshold of Δpitch, and max_roll ≥ roll_th ≥ 0;

[0042] The reward function for the vehicle body pitch angle rβ is shown in Equation (5), which corresponds to the three-level design principle and is as follows:

[0043]

[0044] Wherein, pitch is the vehicle body pitch angle; Δph = |ph – ph_normal|, which represents the change in the vehicle body pitch angle; ph_normal represents the vehicle body pitch angle on a flat road; max_ph is the maximum value of Δpitch, ph_th is the threshold of Δph, and max_ph ≥ ph_th ≥ 0;

[0045] The rw vehicle body yaw rate reward function is shown in Equation (6). Corresponding to the level 1 design principle, a quadratic function is designed as the level 1 reward function, and the formula is as follows:

[0046]

[0047] Wherein, yaw is the yaw angular velocity of the vehicle body; Δyaw = |yaw – yaw_normal|, which represents the change in the yaw angular velocity of the vehicle body; yaw_normal represents the pitch angle of the vehicle body on a flat road; max_yaw is the maximum value of Δyaw, yaw_th is the threshold value of Δyaw, and max_yaw ≥ yaw_th ≥ 0;

[0048] The reward function of the lateral acceleration response time of the rt vehicle body is shown in formula (7). Corresponding to the level 1 design principle, a quadratic function is designed as the level 1 reward function, and the formula is as follows:

[0049]

[0050] Wherein, yt is the vehicle body lateral acceleration response time; Δyt = |yt – yt_normal|, which represents the change in the vehicle body lateral acceleration response time; yt_normal represents the vehicle body lateral acceleration response time on a flat road; max_yt is the maximum value of Δyt, yt_th is the threshold value of Δyaw, and max_yt ≥ yt_th ≥ 0.

[0051] Furthermore, the computational model for constructing the deep reinforcement learning method includes:

[0052] Construct an initial action neural network model. The model is a three-layer neural network model including: the first layer is the input layer, which contains one neuron; the second layer is the hidden layer, which contains x1 neurons; the third layer is the output layer, which contains 2 neurons; the initialization action parameter is ina_first;

[0053] Construct an evaluation neural network model. The model is a three-layer neural network model, including: the first input layer contains two neurons, the hidden layer contains x2 neurons, the output layer contains 1 neuron, and the initial evaluation parameter is inev_first;

[0054] Construct a target neural network model, whose structure is the same as that of the initial neural network model, and set the target action parameter ina_target = ina_first; construct a target evaluation neural network model, whose structure is the same as that of the evaluation neural network model, and set the target evaluation parameter inev_target = inev_first.

[0055] Furthermore, the generating N vehicle data samples using the initial data includes:

[0056] Assign the initial vehicle state parameter v_inital using empirical values ​​or actual vehicle measurement values, and use this vehicle state parameter v_inital as the input of the initial neural network, and the model outputs f(v_inital|ina_first);

[0057] The nth vehicle action parameter an is obtained using formula (8):

[0058] an=f(v_inital|ina_first)(8)

[0059] According to formula (3), the nth vehicle reward value rn is obtained, and the updated nth vehicle state parameter vn_new is obtained; thus, the nth sample is obtained, which is recorded as (vn, an, rn, vn_new)

[0060] Repeat the above steps to obtain the required N data samples.

[0061] Furthermore, the N vehicle data samples obtained in step 5 are used to train the model of the deep reinforcement learning method to obtain the optimized neural network model, including:

[0062] Initialize n=1, use the nth vehicle state parameter vn as the input of the current nth action neural model, and output the nth output value f(vn|ina_first) from the current nth action neural model;

[0063] The nth vehicle state parameter vn, the nth vehicle action parameter an, and the nth output value f(vn|ina_first) of the action network are all used as inputs of the current nth evaluation model. The nth vehicle state parameter vn and the nth vehicle action parameter an are passed through the current nth evaluation model to output the nth output value Fn(an); the nth output value f(vn|ina_first) of the action neural model is passed through the current nth evaluation model to output the nth output value Fn(f(vn|ina_first));

[0064] Use the updated nth vehicle state parameter vn_new as the input of the current nth target action neural model, and output the nth output value f(vn_new|ina_target) from the current nth target action neural model;

[0065] Use the updated nth vehicle state parameter vn_new and the nth output value f(vn_new|ina_target) of the target action neural model as the input of the current nth target evaluation model, and output the nth output value F′n(an_vew) from the current nth target evaluation model;

[0066] Update the current nth action neural model according to the nth output value Fn(f(vn|ina_first)) of the current nth evaluation model using the natural gradient method, so as to obtain the action neural model after the nth update and use it as the nth + 1 action neural model;<0​​​​​​​​​​​​​​​​​​​​​​​​​2. The present invention provides a method and device for optimizing the transmission ratio of a wire-controlled steering system. By utilizing signals collected by vehicle sensors and performing deep learning reinforcement algorithm calculations, the present invention optimizes the real-time optimal selection of the steering transmission ratio, thereby providing the driver with the best experience regardless of whether the vehicle is turning at low speed, turning at high speed, empty, or fully loaded, while also optimizing the vehicle's stability and comfort. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 This is a flow chart of a transmission ratio optimization control method for a steer-by-wire system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0077] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention.

[0078] Example 1

[0079] This embodiment introduces a method for optimizing the transmission ratio of a steer-by-wire system, including:

[0080] Obtain vehicle parameters when the car is driving on the road;

[0081] Preprocessing the vehicle parameters to obtain preprocessed vehicle parameters;

[0082] The preprocessed vehicle parameters are input into a pre-trained deep reinforcement learning algorithm to obtain a real-time steer-by-wire transmission ratio, and the value of the steer-by-wire system is adjusted in real time based on the calculated transmission ratio.

[0083] The application process of the transmission ratio optimization control method of the steer-by-wire system provided in this embodiment specifically involves the following steps:

[0084] Step (1): obtaining vehicle parameters of the vehicle when it is driving on the road in real time; the parameters include: vehicle speed, steering wheel angle, steering wheel angular velocity, vehicle body pitch angle, vehicle body yaw angular velocity, vehicle body roll angle, front axle ground clearance, rear axle ground clearance

[0085] Step (2): pre-process the measured parameters, filter each parameter, eliminate noise, calibrate each parameter, determine the availability of the parameters, set the corresponding range limit for each parameter, and alarm when the limit is exceeded and downgrade the function to the initially set transmission ratio value;

[0086] Step (3): training the deep reinforcement learning algorithm: using the processed collected parameters as the input of the deep reinforcement learning algorithm, using the optimal transmission ratio obtained by the deep reinforcement learning algorithm as the input signal of the wire-controlled steering system, while the car continues to drive, the wire-controlled steering system inputs the transmission ratio according to the value calculated by the algorithm, and obtains the feedback value of the vehicle parameter, and uses the feedback value as the input of the deep reinforcement learning algorithm again after preprocessing; performing multiple cycles, collecting the parameter data in step (1) during each cycle, and using the data to train the deep reinforcement learning algorithm; performing multiple iterative training on step (3), thereby obtaining a trained deep reinforcement learning algorithm; the output of the deep reinforcement learning algorithm is the transmission ratio of the wire-controlled steering system;

[0087] Step (4): While the car is driving, the vehicle operating parameters in step (1) are collected in real time, and the parameters are preprocessed. The preprocessed parameters are input into the trained deep reinforcement learning algorithm to obtain a real-time steer-by-wire transmission ratio, and the value of the steer-by-wire system is adjusted in real time based on the calculated transmission ratio.

[0088] The deep reinforcement learning algorithm is the core point. The implementation steps of the deep reinforcement learning algorithm are as follows:

[0089] Algorithm step 1: Use formula (1) to define the vehicle state parameter v of the deep reinforcement learning method:

[0090] v={w,α,β,wd,αd,βd} (1)

[0091] Ideal yaw rate wd; measured yaw rate w; ideal body roll angle αd; measured body roll angle α; ideal body pitch angle βd; measured body pitch angle β;

[0092] Algorithm step 2: Use formula (2) to define the action parameter a of the deep reinforcement learning method:

[0093] a={γ} (2)

[0094] Where: γ, the angular transmission ratio of the steer-by-wire system, ranges from 10 to 20; unitless;

[0095] Algorithm step 3: Use formula (3) to establish the reward function r of the deep reinforcement learning method:

[0096] r=rβ+rw+rα+rt (3)

[0097] Where: rβ is the vehicle pitch angle reward function, rw is the vehicle yaw angular velocity reward function, rα is the vehicle roll angle reward function, and rt is the lateral acceleration response time

[0098] The reward function is the core of the entire deep reinforcement learning algorithm and guides the adjustment of deep neural network parameters. When designing, the design principles should be established first, and then the specific reward function should be designed based on these principles. In this example, the reward function is set to three priorities. The higher the priority, the more important the principle. The design principles are:

[0099] Level 1: The primary objective of this invention is to ensure the vehicle's handling stability, so ensuring yaw rate control and lateral acceleration response time is the primary task;

[0100] Level 2: Roll is more easily felt by the customer during steering, so the body roll control is more important than the body pitch control;

[0101] Level 3: Make the vehicle pitch as small as possible;

[0102] The body roll angle reward function rα is shown in formula (4), corresponding to the level 2 design principle, and the formula is as follows:

[0103]

[0104] Wherein, roll is the vehicle body pitch angle; Δroll = |roll – roll_normal|, which represents the change in the vehicle body pitch angle; roll_normal represents the vehicle body roll angle on a flat road; max_pitch is the maximum value of Δpitch, roll_th is the threshold of Δpitch, and max_roll ≥ roll_th ≥ 0;

[0105] The reward function for the vehicle body pitch angle rβ is shown in Equation (5), which corresponds to the three-level design principle and is as follows:

[0106]

[0107] Wherein, pitch is the vehicle body pitch angle; Δph = |ph – ph_normal|, which represents the change in the vehicle body pitch angle; ph_normal represents the vehicle body pitch angle on a flat road; max_ph is the maximum value of Δpitch, ph_th is the threshold of Δph, and max_ph ≥ ph_th ≥ 0;

[0108] The rw vehicle body yaw rate reward function is shown in Equation (6). Corresponding to the level 1 design principle, a quadratic function is designed as the level 1 reward function, and the formula is as follows:

[0109]

[0110] Wherein, yaw is the yaw angular velocity of the vehicle body; Δyaw = |yaw – yaw_normal|, which represents the change in the yaw angular velocity of the vehicle body; yaw_normal represents the pitch angle of the vehicle body on a flat road; max_yaw is the maximum value of Δyaw, yaw_th is the threshold value of Δyaw, and max_yaw ≥ yaw_th ≥ 0;

[0111] The reward function of the lateral acceleration response time of the rt vehicle body is shown in formula (7). Corresponding to the level 1 design principle, a quadratic function is designed as the level 1 reward function, and the formula is as follows:

[0112]

[0113] Where yt is the vehicle body lateral acceleration response time; Δyt = |yt – yt_normal|, which represents the change in the vehicle body lateral acceleration response time; yt_normal represents the vehicle body lateral acceleration response time on a flat road; max_yt is the maximum value of Δyt, yt_th is the threshold of Δyaw, and max_yt ≥ yt_th ≥ 0;

[0114] Algorithm step 4: Construct a computational model for deep reinforcement learning methods:

[0115] First, we construct the initial action neural network model. The model is a three-layer neural network model consisting of: the first layer is the input layer, which contains one neuron; the second layer is the hidden layer, which contains x1 neurons; the third layer is the output layer, which contains 2 neurons; the initial action parameter is ina_first;

[0116] Next, we construct an evaluation neural network model. The model is a three-layer neural network model, including: the first input layer, which contains two neurons, the hidden layer contains x2 neurons, and the output layer contains 1 neuron. The initial evaluation parameter is inev_first.

[0117] Finally, a target neural network model is constructed, whose structure is the same as that of the initial neural network model, and the target action parameter ina_target = ina_first is set. A target evaluation neural network model is constructed, whose structure is the same as that of the evaluation neural network model, and the target evaluation parameter inev_target = inev_first is set.

[0118] Algorithm step 5: Generate N vehicle data samples using the initial data:

[0119] Assign the initial vehicle state parameter v_inital using empirical values ​​or actual vehicle measurement values, and use this vehicle state parameter v_inital as the input of the initial neural network, and the model outputs f(v_inital|ina_first);

[0120] The nth vehicle action parameter an is obtained using formula (8):

[0121] an=f(v_inital|ina_first)(8)

[0122] According to formula (3), the nth vehicle reward value rn is obtained, and the updated nth vehicle state parameter vn_new is obtained; thus, the nth sample is obtained, which is recorded as (vn, an, rn, vn_new)

[0123] Repeat the above steps to obtain the required N data samples;

[0124] Algorithm step 6: Use the N vehicle data samples obtained in step 5 to train the model of the deep reinforcement learning method to obtain the optimized neural network model

[0125] Initialize n=1, use the nth vehicle state parameter vn as the input of the current nth action neural model, and output the nth output value f(vn|ina_first) from the current nth action neural model;

[0126] The nth vehicle state parameter vn, the nth vehicle action parameter an, and the nth output value f(vn|ina_first) of the action network are all used as inputs of the current nth evaluation model. The nth vehicle state parameter vn and the nth vehicle action parameter an are passed through the current nth evaluation model to output the nth output value Fn(an); the nth output value f(vn|ina_first) of the action neural model is passed through the current nth evaluation model to output the nth output value Fn(f(vn|ina_first));

[0127] The updated n-th vehicle state parameter vn_new is used as the input of the current n-th target action neural model, and the current n-th target action neural model outputs the n-th output value f(vn_new|ina_target);

[0128] The updated n-th vehicle state parameter vn_new and the n-th output value f(vn_new|ina_target) of the target action neural model are used as the input of the current n-th target evaluation model, and the current n-th target evaluation model outputs the n-th output value F′n(an_vew);

[0129] According to the nth output value Fn(f(vn|ina_first)) of the current nth evaluation model, the current nth action neural model is updated using the natural gradient method, so as to obtain the nth updated action neural model and serve as the n+1th action neural model;

[0130] Using the output Fn(an) of the current nth evaluation model and the output F′n(an_new) of the current nth target evaluation model, the current nth evaluation model is updated using the maximum likelihood loss, so as to obtain the evaluation model after the nth update and use it as the (n + 1)th evaluation model;

[0131] If n + 1 > N, it indicates that the optimal neural network model is obtained. If n < N, repeat the steps until n > N;

[0132] Algorithm step 7: During vehicle operation, the current state parameters vt of the vehicle are measured in real time, and the state parameters are input into the optimal neural network model obtained above to directly output the optimized by - wire steering transmission ratio in the current state.

[0133] Embodiment 2

[0134] This embodiment provides a device for optimizing the control of the by - wire steering system transmission ratio, including:

[0135] A vehicle parameter acquisition unit, configured to acquire vehicle parameters when the vehicle is driving on the road surface;

[0136] A pre - processing unit, configured to pre - process the vehicle parameters to obtain the pre - processed vehicle parameters;

[0137] A calculation and adjustment unit, configured to input the pre - processed vehicle parameters into a pre - trained deep reinforcement learning algorithm to obtain the real - time by - wire steering transmission ratio, and adjust the value of the by - wire steering system in real time based on the calculated value of the transmission ratio.

[0138] The above is only the preferred embodiment of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the technical principle of the present invention, several improvements and deformations can be made, and these improvements and deformations should also be regarded as the protection scope of the present invention.

Claims

1. A method for optimizing the transmission ratio of a steer-by-wire system, characterized in that: include: Obtain vehicle parameters when the car is driving on the road; Preprocessing the vehicle parameters to obtain preprocessed vehicle parameters; Inputting the preprocessed vehicle parameters into a pre-trained deep reinforcement learning algorithm to obtain a real-time steer-by-wire transmission ratio, and adjusting the steer-by-wire system value in real time based on the calculated transmission ratio; The training of the deep reinforcement learning algorithm includes: Obtain vehicle parameters when the car is driving on the road; Preprocessing the vehicle parameters to obtain preprocessed vehicle parameters; Using the preprocessed vehicle parameters as input to a deep reinforcement learning algorithm to obtain an optimal transmission ratio; The optimal transmission ratio obtained by the deep reinforcement learning algorithm is used as the input signal of the steer-by-wire system. As the car continues to move, the steer-by-wire system inputs the transmission ratio according to the value calculated by the algorithm and obtains feedback values ​​of vehicle parameters. The feedback value is pre-processed and used as input to the deep reinforcement learning algorithm again; multiple cycles are performed, and data on vehicle parameters is collected during each cycle, and the data is used to train the deep reinforcement learning algorithm; Performing multiple iterations of training on the above steps to obtain a trained deep reinforcement learning algorithm; wherein the output of the deep reinforcement learning algorithm is the transmission ratio of the steer-by-wire system; The implementation steps of the deep reinforcement learning algorithm include: Algorithm step 1: Use formula (1) to define the vehicle state parameter v of the deep reinforcement learning method: v={w,α,β,wd,αd,βd} (1); Wherein, wd is the ideal yaw rate; w is the measured yaw rate; αd is the ideal body roll angle; α is the measured body roll angle; βd is the ideal body pitch angle; β is the measured body pitch angle; Algorithm step 2: Use formula (2) to define the action parameter a of the deep reinforcement learning method: a={γ}(2); Where: γ is the angular transmission ratio of the steer-by-wire system, unitless; Algorithm step 3: Use formula (3) to establish the reward function r of the deep reinforcement learning method: r=rβ + rw + rα+ rt (3); Where: rβ is the vehicle pitch angle reward function, rw is the vehicle yaw angular velocity reward function, rα is the vehicle roll angle reward function, and rt is the lateral acceleration response time; Algorithm step 4: Construct a computational model for deep reinforcement learning methods; Algorithm step 5: Generate N vehicle data samples using the initial data; Algorithm step 6: Use the N vehicle data samples obtained in step 5 to train the model of the deep reinforcement learning method to obtain the optimized neural network model; Algorithm step 7: During vehicle operation, the vehicle's current state parameter vt is measured in real time. The state parameter is input into the optimal neural network model obtained above, and the optimal steer-by-wire transmission ratio is directly outputted for the current state. The reward function is set to 3 priorities. The higher the priority, the more important the principle. The design principles are: Level 1: Ensures that the yaw rate control and lateral acceleration response time are within the set range; Level 2: Ensure that the roll control of the vehicle body is higher than the pitch control of the vehicle body; Level 3: Ensure that the vehicle's pitch is within the set range; Among them, the body roll angle reward function rα is shown in formula (4), corresponding to the level 2 design principle, the formula is as follows: (4)； Wherein, roll is the roll angle of the vehicle body; Δroll = |roll – roll_normal|, which represents the change in the roll angle of the vehicle body; roll_normal represents the roll angle of the vehicle body on a flat road; maxroll is the maximum value of Δroll, roll_th is the threshold value of Δroll, and maxroll ≥ roll_th ≥ 0; The reward function for the vehicle body pitch angle rβ is shown in Equation (5), which corresponds to the 3-level design principle and is as follows: (5)； Wherein, ph is the vehicle body pitch angle; Δph = |ph – ph_normal|, which represents the change in the vehicle body pitch angle; ph_normal represents the vehicle body pitch angle on a flat road; maxph is the maximum value of Δph, ph_th is the threshold value of Δph, and maxph ≥ ph_th ≥ 0; The rw vehicle body yaw rate reward function is shown in Equation (6). Corresponding to the level 1 design principle, a quadratic function is designed as the level 1 reward function, and the formula is as follows: (6)； Wherein, yaw is the yaw angular velocity of the vehicle body; Δyaw = |yaw – yaw_normal|, which represents the change in the yaw angular velocity of the vehicle body; yaw_normal represents the yaw angular velocity of the vehicle body on a flat road; maxyaw is the maximum value of Δyaw, yaw_th is the threshold value of Δyaw, and maxyaw ≥ yaw_th ≥ 0; The reward function of the lateral acceleration response time of the rt vehicle body is shown in Equation (7). Corresponding to the level 1 design principle, a quadratic function is designed as the level 1 reward function, and the formula is as follows: (7)； Wherein, yt is the vehicle body lateral acceleration response time; Δyt = |yt – yt_normal|, which represents the change in vehicle body lateral acceleration response time; yt_normal represents the vehicle body lateral acceleration response time on a flat road; maxyt is the maximum value of Δyt, yt_th is the threshold value of Δyaw, and maxyt ≥ yt_th ≥ 0; The computational model for constructing the deep reinforcement learning method includes: Construct an initial action neural network model. The model is a three-layer neural network model including: the first layer is the input layer, which contains one neuron; the second layer is the hidden layer, which contains x1 neurons; the third layer is the output layer, which contains 2 neurons; the initial action parameter is ina_first; Construct an evaluation neural network model. The model is a three-layer neural network model, including: the first input layer contains two neurons, the hidden layer contains x2 neurons, the output layer contains 1 neuron, and the initial evaluation parameter is inev_first; Construct a target neural network model, whose structure is the same as that of the initial neural network model, and set the target action parameter ina_target = ina_first; construct a target evaluation neural network model, whose structure is the same as that of the evaluation neural network model, and set the target evaluation parameter inev_target = inev_first; The generating N vehicle data samples by using the initial data includes: Assign the initial vehicle state parameter v_inital using empirical values ​​or actual vehicle measurements, and use this vehicle state parameter v_inital as the input of the initial neural network, which outputs f(v_inital | ina_first); The nth vehicle action parameter an is obtained using formula (8): an=f(v_inital|ina_first) (8) According to formula (3), the nth vehicle reward value rn is obtained, and the updated nth vehicle state parameter vn_new is obtained; thus, the nth sample is obtained, which is recorded as (vn, an, rn, vn_new); Repeat the above steps to obtain the required N data samples; The N vehicle data samples obtained in step 5 are used to train the model of the deep reinforcement learning method to obtain an optimized neural network model, including: Initialize n=1, use the nth vehicle state parameter vn as the input of the current nth action neural model, and output the nth output value f(vn | ina_first ); The nth vehicle state parameter vn, the nth vehicle action parameter an, and the nth output value f(vn|ina_first) of the action network are all used as the input of the current nth evaluation model. The nth vehicle state parameter vn and the nth vehicle action parameter an are passed through the current nth evaluation model to output the nth output value Fn(an); the nth output value f(vn|ina_first) of the action neural model is passed through the current nth evaluation model to output the nth output value Fn(f(vn|ina_first)); The updated n-th vehicle state parameter vn_new is used as the input of the current n-th target action neural model, and the current n-th target action neural model outputs the n-th output value f(vn_new | ina_target); The updated n-th vehicle state parameter vn_new and the n-th output value f(vn_new|ina_target) of the target action neural model are used as the input of the current n-th target evaluation model, and the current n-th target evaluation model outputs the n-th output value F′n(an_vew); Update the current n-th action neural model using the natural gradient method according to the n-th output value Fn(f(vn| ina_first )) of the current n-th evaluation model, thereby obtaining the n-th updated action neural model and using it as the n+1-th action neural model; Using the output Fn(an) of the current nth evaluation model and the output F′n(an_new) of the current nth target evaluation model, the current nth evaluation model is updated using the maximum likelihood loss, so as to obtain the evaluation model after the nth update and use it as the (n + 1)th evaluation model; If n + 1 > N, it indicates that the optimal neural network model is obtained. If n < N, repeat the steps until n > N.

2. The method for optimizing the transmission ratio of a steer-by-wire system according to claim 1, wherein: The vehicle parameters include any one or more of vehicle speed, steering wheel angle, steering wheel angular velocity, body pitch angle, body yaw angular velocity, body roll angle, front axle ground clearance, and rear axle ground clearance.

3. The method for optimizing the transmission ratio of a steer-by-wire system according to claim 1, wherein: The preprocessing of the vehicle parameters includes: Filtering each vehicle parameter to eliminate noise points; Cross-checking each vehicle parameter to determine the availability of the parameter. For each parameter, set a corresponding range limit. If it exceeds, an alarm is given and the function is degraded to the transmission ratio set initially.

4. A steer-by-wire system transmission ratio optimization control device, using the steer-by-wire system transmission ratio optimization control method according to claim 1, characterized in that: It includes: A vehicle parameter acquisition unit for acquiring vehicle parameters when the vehicle is driving on the road surface; A preprocessing unit for preprocessing the vehicle parameters to obtain the preprocessed vehicle parameters; A calculation and adjustment unit for inputting the preprocessed vehicle parameters into a pre-trained deep reinforcement learning algorithm to obtain a real-time steer-by-wire transmission ratio, and adjusting the value of the steer-by-wire system in real time based on the calculated value of the transmission ratio.

Images