A Q-learning hybrid RBF neural network tire side slip angle calculation method, storage medium, device and computer program product

Abstract

The invention discloses a tire slip angle calculation method of a Q-learning hybrid RBF neural network, a storage medium, a device and a computer program product, comprising: collecting vehicle dynamic parameters, tire parameters and tire slip angles under different steering conditions, and obtaining a first component of the tire slip angle through a Dugoff tire inverse model; constructing an RBF neural network, taking all the collected vehicle dynamic parameters and tire parameters as inputs of the RBF neural network, and training the RBF neural network through a reinforcement learning algorithm; taking a state with a maximum Q value as an optimal parameter of the RBF neural network, and calculating a weight matrix of the RBF neural network under the optimal parameter; collecting vehicle dynamic parameters in real time during vehicle driving, calculating a first component of the tire slip angle, predicting a second component of the tire slip angle through an optimal RBF neural network, and taking the sum of the first component of the tire slip angle and the second component of the tire slip angle as the tire slip angle.

Description

Technical Field

[0001] The present invention relates to the field of vehicle dynamics control, and in particular to a tire side slip angle calculation method, storage medium, device and computer program product of a Q-learning hybrid RBF neural network. Background Art

[0002] With the development of vehicle chassis control, its control architecture has gradually changed from a decentralized architecture to a centralized control architecture. In centralized control, the tire lateral force needs to be precisely controlled. The size of the tire slip angle is closely related to the vehicle lateral force, and the control of the tire lateral force is inseparable from the precise control of the tire slip angle.

[0003] Traditional tire models, such as the Dugoff tire inverse model, can better describe the corresponding relationship between tire slip angle and slip rate and tire lateral force and tire longitudinal force. However, due to the simplification and nonlinear factors of the Dugoff tire inverse model, the Dugoff tire inverse model may have certain errors in practical applications, resulting in inaccurate calculation of the expected tire slip angle. Therefore, how to improve the accuracy of the expected tire slip angle calculation is of great significance to improving vehicle handling stability. Summary of the invention

[0004] In view of the problems existing in the prior art, the present invention provides a tire slip angle calculation method, storage medium, device and computer program product based on a Q-learning hybrid RBF neural network, which compensates for the error of the Dugoff tire inverse model through the RBF neural network and improves the accuracy and stability of the tire slip angle calculation.

[0005] To achieve the above technical objectives, the present invention adopts the following technical solution: a tire side slip angle calculation method using a Q-learning hybrid RBF neural network, specifically comprising the following steps:

[0006] Step S1, establishing a Dugoff tire inverse model according to the Dugoff tire model;

[0007] Step S2, collecting vehicle dynamic parameters, tire parameters and tire slip angles under different steering conditions, inputting the vehicle dynamic parameters into the Dugoff tire inverse model, and obtaining the first component of the tire slip angle under different tire slip angles; the vehicle dynamic parameters include: the expected longitudinal force of the tire, the expected lateral force of the tire, the adhesion coefficient of the road surface on which the tire is located, and the vertical load of the tire; the tire parameters include the longitudinal stiffness of the tire and the lateral stiffness of the tire;

[0008] Step S3, constructing an RBF neural network, using all collected vehicle dynamic parameters and tire parameters as inputs of the RBF neural network, using the collected tire side slip angles as labels, and training the RBF neural network through a reinforcement learning algorithm until the round of training is completed, thereby completing the training of the RBF neural network;

[0009] Step S4, taking the state with the maximum Q value as the optimal parameter of the RBF neural network, inputting all the collected vehicle dynamic parameters and tire parameters into the RBF neural network under the optimal parameters, calculating the weight matrix of the RBF neural network, and obtaining the optimal RBF neural network model under the optimal parameters and the corresponding weight matrix;

[0010] Step S5, collecting vehicle dynamic parameters and tire parameters in real time during vehicle driving, calculating the first component of the tire slip angle by using the vehicle dynamic parameters through the Dugoff tire inverse model, and inputting the vehicle dynamic parameters and tire parameters into the optimal RBF neural network to predict the second component of the tire slip angle;

[0011] Step S6: taking the sum of the first component of the tire slip angle and the second component of the tire slip angle as the tire slip angle.

[0012] Furthermore, the establishment process of the Dugoff tire inverse model in step S1 is:

[0013] When the vehicle and road dynamic parameters D ≥ 1, the Dugoff tire inverse model is expressed as:

[0014]

[0015] When the vehicle and road dynamic parameter D < 1, the Dugoff tire inverse model is expressed as:

[0016]

[0017] in, λ is the tire slip rate, α is the tire side slip angle, F x is the desired longitudinal force of the tire, F y is the desired lateral force of the tire, μ is the tire adhesion coefficient on the road, F z is the vertical load of the tire, C λ is the longitudinal stiffness of the tire, C α is the cornering stiffness of the tire.

[0018] Furthermore, step S3 includes the following sub-steps:

[0019] Step S3.1, using the hidden layer node center vector and base width vector of the RBF neural network as the state of the reinforcement learning algorithm, using the increment of the hidden layer node center vector and the increment of the base width vector of the RBF neural network as the action of the reinforcement learning algorithm, and initializing the Q table, which is used to store the Q value of the state-action pair;

[0020] Step S3.2, select the corresponding action in the Q table according to the greedy algorithm, and update the hidden layer node center vector and base width vector of the RBF neural network;

[0021] Step S3.3, inputting all collected vehicle dynamic parameters and tire parameters into the RBF neural network for updating the hidden layer node center vector and base width vector for training, and predicting the second component of the tire side slip angle corresponding to all tire parameters;

[0022] Step S3.4, taking the sum of the second component of the predicted tire slip angle and the first component of the tire slip angle calculated by the Dugoff tire inverse model as the predicted tire slip angle, taking the sum of the mean square errors of all predicted tire slip angles and the corresponding actual tire slip angles as the reward function, updating the Q value through the Bellman equation, and storing it in the Q table;

[0023] Step S3.5: Repeat steps S3.2-S3.4 until the round of training is completed, thus completing the training of the RBF neural network.

[0024] Furthermore, the specific process of the corresponding action selection in step S3.2 is:

[0025]

[0026] Among them, a represents the corresponding action selected from the Q table, Q(s,a) represents the Q value of the state-action pair in the Q table, random() represents generating a random number between 0 and 1, max(Q(s,a)) represents the maximum Q value in the Q table, rand represents a random number in [0,1], and ε represents the probability of the exploration rate.

[0027] Furthermore, the updating process of the Q value in step S3.4 is specifically as follows:

[0028] Q(s,a)′=Q(s,a)+ζ(r(s,a)+ξmaxQ(s′,a′)-Q(s,a))

[0029] Among them, Q(s,a)′ represents the Q value updated by the Bellman equation, Q(s,a) represents the Q value of the state-action pair in the Q table, ζ represents the learning rate, ξ represents the discount rate, r(s,a) represents the reward function for a single training round, and maxQ(s′,a′) represents the maximum expected Q value corresponding to the new state s′ in the Q table under the current action.

[0030] Furthermore, the calculation process of the weight matrix of the RBF neural network in step S4 is:

[0031] ω i =(H i T H i ) -1 H i T Y

[0032] Among them, ω i represents the weight matrix of the i-th node in the hidden layer of the RBF neural network, Y represents the difference matrix between the tire slip angle and the second component of the tire slip angle, H i The vector matrix representing the i-th node of the hidden layer in the RBF neural network.

[0033] Furthermore, the present invention also provides a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the tire side slip angle calculation method of the Q-learning hybrid RBF neural network.

[0034] Furthermore, the present invention also provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the tire side slip angle calculation method of the Q-learning hybrid RBF neural network is implemented.

[0035] Furthermore, the present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the tire side slip angle calculation method of the Q-learning hybrid RBF neural network.

[0036] Compared with the prior art, the present invention has the following beneficial effects: the tire slip angle calculation method of the Q-learning hybrid RBF neural network of the present invention combines the Dugoff tire inverse model and the RBF neural network, and obtains the optimal parameters of the RBF neural network through a reinforcement learning algorithm, so that the sum of the tire slip angle components output by the Dugoff tire inverse model and the RBF neural network can approach the desired tire slip angle, compensate for the calculation error of the Dugoff tire inverse model, improve the accuracy and stability of the tire slip angle calculation, and provide important support for the optimization of the vehicle control system. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a flow chart of the tire side slip angle calculation method of the Q learning hybrid RBF neural network of the present invention;

[0038] Figure 2 It is a framework diagram of the tire side slip angle calculation method of the Q learning hybrid RBF neural network in the present invention. DETAILED DESCRIPTION

[0039] The technical solution of the present invention is further explained below in conjunction with the accompanying drawings.

[0040] like Figure 1-2 The present invention provides a tire side slip angle calculation method using a Q-learning hybrid RBF neural network, which specifically includes the following steps:

[0041] Step S1, establish a Dugoff tire inverse model according to the Dugoff tire model; specifically, according to the tire slip rate λ, the tire side slip angle α, the tire adhesion coefficient μ on the road surface, the tire vertical load F z , longitudinal stiffness of the tire C λ , tire cornering stiffness C α Building the Dugoff Tire Model Calculate vehicle and road dynamic parameters The Dugoff tire inverse model is derived based on the vehicle and road dynamic parameters:

[0042] When the vehicle and road dynamic parameters D ≥ 1, the Dugoff tire inverse model is expressed as:

[0043]

[0044] When the vehicle and road dynamic parameter D < 1, the Dugoff tire inverse model is expressed as:

[0045]

[0046] Among them, F x is the desired longitudinal force of the tire, F y is the desired lateral force of the tire, H D Represents tire dynamic parameters,

[0047] f(H D ) represents a D The function of

[0048] Step S2, collecting vehicle dynamic parameters, tire parameters and tire slip angles under different steering conditions, and after removing noise and abnormal values, inputting the vehicle dynamic parameters into the Dugoff tire inverse model to obtain the tire slip angles under different tire slip angles. Due to the influence of the tire parameter errors and the tire dynamic characteristics, the tire slip angle calculated by the Dugoff tire inverse model has errors, resulting in inaccurate calculation of the expected tire slip angle. Therefore, the tire slip angle calculated by the Dugoff tire inverse model is used as the first component α of the tire slip angle. DugoffThe vehicle dynamic parameters in the present invention include: the expected longitudinal force of the tire, the expected lateral force of the tire, the adhesion coefficient of the road surface on which the tire is located, and the vertical load of the tire. The tire parameters include: the longitudinal stiffness of the tire and the lateral stiffness of the tire.

[0049] Step S3, constructing an RBF neural network. The RBF neural network in the present invention is a three-layer feedforward neural network. The structure of the RBF neural network is as follows: Figure 2 As shown, it includes: input layer, hidden layer, and output layer. The input longitudinal force F collected in the input layer x , lateral force F y , vertical load F z , road adhesion coefficient μ, tire longitudinal stiffness C λ 、Tire cornering stiffness C α , the collected tire slip angles are used as labels, and the collected tire slip angles are compared with the first component α of the tire slip angle Dugoff The difference between the input feature and the target value is used as the input feature; the RBF vector H in the hidden layer i =[h1,h2,h3,...,h m ] T , consisting of m radial centers, each weight value connecting the input layer to the hidden layer is uniform, and the output weight vector is ω i =[ω1,ω2,ω3,…,ω m ], the Gaussian basis function is used as the activation function in the RBF neural network, and the number of nodes is determined by experience or cross-validation. The output of the i-th node is Among them, C i =[C1,C2,C3,…,C m ] is the center vector of the i-th node in the hidden layer, B i =[B1,B2,B3,...,B m ] is the base width vector of the i-th node in the hidden layer; ‖x i -C i ‖ is the input x i With C i The Euclidean distance of the tire side slip angle is output from the output layer.

[0050] All collected vehicle dynamic parameters and tire parameters are used as inputs of the RBF neural network, and the collected tire side slip angles are used as labels. The RBF neural network is trained through a reinforcement learning algorithm until the round of training is completed. The training of the RBF neural network is completed, and the exploratory nature of reinforcement learning is used to obtain the optimal RBF neural network parameters, improve the training efficiency of the RBF neural network, and prevent overfitting of the RBF neural network. The specific sub-steps include the following:

[0051] Step S3.1: The hidden layer node center vector C of the RBF neural network i and base width vector B i As the state s of the reinforcement learning algorithm = (C i ,B i ), the increment ΔC of the center vector of the hidden layer node of the RBF neural network i and the increment of the base width vector ΔB i As the action of reinforcement learning algorithm a=(ΔC i ,ΔB i ) and initialize the Q table, which is used to store the Q value of the state-action pair. The Q value represents the expected cumulative reward for taking a specific action in a specific state. The initial Q table is set to zero and its dimension is 2m×2m.

[0052] Step S3.2, select the corresponding action in the Q table according to the greedy algorithm, update the hidden layer node center vector and base width vector of the RBF neural network, the greedy algorithm compromises exploration and utilization based on probability, that is, exploration is performed with a probability of the exploration rate ε, and utilization is performed with a probability of 1-ε, to prevent the selection of the corresponding action from falling into the local optimum. The specific process of the corresponding action selection is:

[0053]

[0054] Among them, a represents the corresponding action selected from the Q table, Q(s,a) represents the Q value of the state-action pair in the Q table, random() represents generating a random number between 0 and 1, max(Q(s,a)) represents the maximum Q value in the Q table, rand represents a random number in [0,1], and ε represents the probability of the exploration rate.

[0055] Step S3.3, inputting all collected vehicle dynamic parameters and tire parameters into the RBF neural network for updating the hidden layer node center vector and base width vector for training, and predicting the second component of the tire side slip angle corresponding to all tire parameters;

[0056] Step S3.4, taking the sum of the second component of the predicted tire slip angle and the first component of the tire slip angle calculated by the Dugoff tire inverse model as the predicted tire slip angle, taking the sum of the mean square errors of all predicted tire slip angles and the corresponding actual tire slip angles as the reward function, updating the Q value through the Bellman equation, and storing it in the Q table;

[0057] The updating process of the Q value in the present invention is based on the Bellman equation, which is simple, easy to implement and highly efficient, and is specifically as follows:

[0058] Q(s,a)=Q(s,a)+ζ(r(s,a)+ξmaxQ(s′,a′)-Q(s,a))

[0059] Among them, Q(s,a)′ represents the Q value updated by the Bellman equation, Q(s,a) represents the Q value of the state-action pair in the Q table, ζ represents the learning rate, ξ represents the discount rate, r(s,a) represents the reward function for a single training round, and maxQ(s′,a) represents the maximum expected Q value corresponding to the new state s′ in the Q table under the current action.

[0060] Step S3.5: Repeat steps S3.2-S3.4 until the round of training is completed, thus completing the training of the RBF neural network.

[0061] Step S4, taking the state with the maximum Q value as the optimal parameter of the RBF neural network, inputting all the collected vehicle dynamic parameters and tire parameters into the RBF neural network under the optimal parameters, calculating the weight matrix of the RBF neural network, and obtaining the optimal RBF neural network model under the optimal parameters and the corresponding weight matrix;

[0062] The calculation process of the weight matrix of the RBF neural network in the present invention is:

[0063] ω i =(H i T H i ) -1 H i T Y

[0064] Among them, ω i represents the weight matrix of the RBF neural network, Y represents the difference matrix between the tire slip angle and the second component of the tire slip angle, H i Represents the hidden layer vector matrix of the RBF neural network.

[0065] Step S5: Real-time acquisition of vehicle dynamic parameters and tire parameters during vehicle driving, and calculation of the first component α of the tire side slip angle by using the vehicle dynamic parameters through the Dugoff tire inverse model Dugoff The vehicle dynamic parameters and tire parameters are input into the optimal RBF neural network to predict the second component α of the tire side slip angle RBF ;

[0066] Step S6: The first component α of the tire side slip angle is Dugoff The second component of the tire slip angle α RBF The sum is the tire slip angle α final .

[0067] The tire slip angle α obtained by the tire slip angle calculation method of the Q learning hybrid RBF neural network of the present invention is finalThe tire slip angle is sent to the steering system of the vehicle for tire side slip angle control, which has the advantages of fast convergence speed, high accuracy and strong adaptability. Compared with the separate Dugoff tire inverse model for calculating the tire slip angle, this method greatly compensates for the tire slip angle error calculated near the lateral force peak, and the actual lateral force obtained is closer to the expected lateral force, thereby improving the vehicle response characteristics. This shows that the tire slip angle calculation method of the present invention improves the accuracy and stability of the tire slip angle calculation.

[0068] In a technical solution of the present invention, a computer-readable storage medium is also provided, storing a computer program, wherein the computer program enables a computer to execute the tire side slip angle calculation method of the Q-learning hybrid RBF neural network.

[0069] In one technical solution of the present invention, an electronic device is also provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the tire side slip angle calculation method of the Q-learning hybrid RBF neural network is implemented.

[0070] In a technical solution of the present invention, a computer program product is also provided, including a computer program, and when the computer program is executed by a processor, the tire side slip angle calculation method of the Q-learning hybrid RBF neural network is implemented.

[0071] In the embodiments disclosed in the present application, the computer storage medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. The computer storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the above. More specific examples of computer storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0072] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0073] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A tire slip angle calculation method using a Q-learning hybrid RBF neural network, characterized in that: The specific steps include: Step S1, establishing a Dugoff tire inverse model according to the Dugoff tire model; Step S2, collecting vehicle dynamic parameters, tire parameters and tire slip angles under different steering conditions, inputting the vehicle dynamic parameters into the Dugoff tire inverse model, and obtaining the first component of the tire slip angle under different tire slip angles; the vehicle dynamic parameters include: the expected longitudinal force of the tire, the expected lateral force of the tire, the adhesion coefficient of the road surface on which the tire is located, and the vertical load of the tire; the tire parameters include: the longitudinal stiffness of the tire and the lateral stiffness of the tire; Step S3, constructing an RBF neural network, using all collected vehicle dynamic parameters and tire parameters as inputs of the RBF neural network, using the collected tire side slip angles as labels, and training the RBF neural network through a reinforcement learning algorithm until the round of training is completed, thereby completing the training of the RBF neural network; Step S4, taking the state with the maximum Q value as the optimal parameter of the RBF neural network, inputting all the collected vehicle dynamic parameters and tire parameters into the RBF neural network under the optimal parameters, calculating the weight matrix of the RBF neural network, and obtaining the optimal RBF neural network model under the optimal parameters and the corresponding weight matrix; Step S5, collecting vehicle dynamic parameters and tire parameters in real time during vehicle driving, calculating the first component of the tire slip angle by using the vehicle dynamic parameters through the Dugoff tire inverse model, and inputting the vehicle dynamic parameters and tire parameters into the optimal RBF neural network to predict the second component of the tire slip angle; Step S6: taking the sum of the first component of the tire slip angle and the second component of the tire slip angle as the tire slip angle.

2. The tire side slip angle calculation method of a Q-learning hybrid RBF neural network according to claim 1, characterized in that: The establishment process of the Dugoff tire inverse model in step S1 is: When the vehicle and road dynamic parameters D ≥ 1, the Dugoff tire inverse model is expressed as: When the vehicle and road dynamic parameter D < 1, the Dugoff tire inverse model is expressed as: in, λ is the tire slip rate, α is the tire side slip angle, F x is the desired longitudinal force of the tire, F y is the desired lateral force of the tire, μ is the tire adhesion coefficient on the road, F z is the vertical load of the tire, C λ is the longitudinal stiffness of the tire, C α is the cornering stiffness of the tire.

3. The tire slip angle calculation method of a Q-learning hybrid RBF neural network according to claim 1 is characterized in that: Step S3 includes the following sub-steps: Step S3.1, using the hidden layer node center vector and base width vector of the RBF neural network as the state of the reinforcement learning algorithm, using the increment of the hidden layer node center vector and the increment of the base width vector of the RBF neural network as the action of the reinforcement learning algorithm, and initializing the Q table, which is used to store the Q value of the state-action pair; Step S3.2, select the corresponding action in the Q table according to the greedy algorithm, and update the hidden layer node center vector and base width vector of the RBF neural network; Step S3.3, inputting all collected vehicle dynamic parameters and tire parameters into the RBF neural network for updating the hidden layer node center vector and base width vector for training, and predicting the second component of the tire side slip angle corresponding to all tire parameters; Step S3.4, taking the sum of the second component of the predicted tire slip angle and the first component of the tire slip angle calculated by the Dugoff tire inverse model as the predicted tire slip angle, taking the sum of the mean square errors of all predicted tire slip angles and the corresponding actual tire slip angles in the label as the reward function, updating the Q value through the Bellman equation, and storing it in the Q table; Step S3.5: Repeat steps S3.2-S3.4 until the round of training is completed, thus completing the training of the RBF neural network.

4. The tire slip angle calculation method of a Q-learning hybrid RBF neural network according to claim 3 is characterized in that: The specific process of the corresponding action selection in step S3.2 is: Among them, a represents the corresponding action selected from the Q table, Q(s,a) represents the Q value of the state-action pair in the Q table, random() represents generating a random number between 0 and 1, max(Q(s,a)) represents the maximum Q value in the Q table, rand represents a random number in [0,1], and ε represents the probability of the exploration rate.

5. The tire side slip angle calculation method of a Q-learning hybrid RBF neural network according to claim 3 is characterized in that: The specific process of updating the Q value in step S3.4 is as follows: Q(s,a)′=Q(s,a)+ζ(r(s,a)+ξmaxQ(s′,a′)-Q(s,a)) Among them, Q(s,a)′ represents the Q value updated by the Bellman equation, Q(s,a) represents the Q value of the state-action pair in the Q table, ζ represents the learning rate, ξ represents the discount rate, r(s,a) represents the reward function for a single training round, and maxQ(s′,a′) represents the maximum expected Q value corresponding to the new state s′ in the Q table under the current action.

6. The tire side slip angle calculation method of a Q-learning hybrid RBF neural network according to claim 1, characterized in that: The calculation process of the weight matrix of the RBF neural network in step S4 is: ω i =(H i T H i ) -1 H i T Y Among them, ω i represents the weight matrix of the i-th node in the hidden layer of the RBF neural network, Y represents the difference matrix between the tire slip angle and the second component of the tire slip angle, H i The vector matrix representing the i-th node of the hidden layer in the RBF neural network.

7. A computer-readable storage medium storing a computer program, characterized in that: The computer program enables a computer to execute the tire side slip angle calculation method of the Q-learning hybrid RBF neural network as described in any one of claims 1-6.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the tire side slip angle calculation method of the Q learning hybrid RBF neural network as described in any one of claims 1 to 6 is implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the tire slip angle calculation method of the Q-learning hybrid RBF neural network described in any one of claims 1 to 6 is implemented.

Images