Path tracking control method and device based on intelligent driving, and vehicle

Abstract

The application relates to the technical field of intelligent driving, for example, to a path tracking control method and device based on intelligent driving and a vehicle. The control method comprises the following steps: acquiring running state information and planning path information of the vehicle; constructing a state space based on the running state information and the planning path information; inputting the state space into a reinforcement learning strategy model to obtain a vehicle speed sequence based on the planning path information; constructing and solving a control Lyapunov-barrier function based on the planning path information and the vehicle speed sequence to obtain an explicit control law; and controlling the vehicle to run based on the explicit control law. The present application uses a reinforcement learning algorithm to process road curvature and slope to output a globally optimized speed sequence. The lower layer constructs an explicit control target based on the speed reference sequence and the planning path information, and solves the explicit control law to generate a control instruction meeting the tracking accuracy and safety requirements, thereby realizing the autonomous driving function of the vehicle in a complex environment.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, such as a path tracking control method and device, vehicle, and readable storage medium based on intelligent driving. Background Technology

[0002] Autonomous driving is a cutting-edge field that deeply integrates artificial intelligence, automatic control, and vehicle engineering. In real-world traffic environments, autonomous vehicles must safely and smoothly complete path-tracking tasks amidst dense dynamic obstacles and varied road conditions. In practice, trajectory tracking involves not only precise following of the spatial path but also dynamic optimization of the reference speed. Especially in typical scenarios such as slopes, curves, and complex traffic areas, failure to rationally plan the speed based on road curvature and gradient information will directly impact the stability and safety of the control system. Furthermore, with increasing environmental complexity, path-tracking systems must also possess high real-time performance and safety assurance capabilities.

[0003] In related technologies, to enhance the global performance and intelligent decision-making capabilities of systems, some studies have introduced reinforcement learning methods for policy optimization. Specifically, the upper layer uses reinforcement learning algorithms for global optimization, while the lower layer uses MPC for constraint control. This approach combines intelligent policy optimization with traditional model control.

[0004] In the process of implementing the above embodiments, at least the following problems were found:

[0005] The above embodiments use MPC as the execution layer controller, which still cannot get rid of the high complexity of iterative calculation. At the same time, the reinforcement learning part does not fully consider the impact of geometric information such as road curvature and slope on speed planning, which limits the system's response capability and control accuracy in diverse road scenarios. Summary of the Invention

[0006] To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general commentary, nor is it intended to identify key / important components or describe the scope of protection of these embodiments, but rather as a prelude to the detailed description that follows.

[0007] This disclosure provides a path tracking control method and device, vehicle, and readable storage medium based on intelligent driving, in order to solve the technical problem that the high complexity of iterative calculation in related technologies limits the system's response capability and control accuracy.

[0008] In some embodiments, a path tracking control method based on intelligent driving is provided, comprising: acquiring vehicle operating state information and planned path information; constructing a state space based on the operating state information and planned path information; inputting the state space into a reinforcement learning policy model to obtain a vehicle speed sequence based on the planned path information; constructing and solving a control Lyapunov-obstacle function based on the planned path information and the vehicle speed sequence to obtain an explicit control law; and controlling vehicle operation based on the explicit control law.

[0009] The path tracking control method for intelligent driving disclosed herein adopts a hierarchical control architecture. The upper layer employs a reinforcement learning strategy model, which outputs a vehicle speed sequence based on the planned path information according to the vehicle's operating state information and the planned path information. The lower layer constructs an explicit control law based on the control Lyapunov-obstacle function, using the vehicle speed sequence generated by reinforcement learning as the tracking target, to achieve real-time control without iterative optimization and ensure that the system meets safety constraints.

[0010] Thus, compared to reinforcement learning methods in related technologies, the path control method provided in this disclosure utilizes a reinforcement learning policy model and leverages the autonomous learning capability of reinforcement learning algorithms to process complex geometric information such as road curvature and slope, outputting a globally optimized speed sequence. Furthermore, unlike related technologies that use MPC (Model Predictive Control) as the execution layer controller, the lower layer of this disclosure constructs an explicit control objective based on the speed reference and planned path data generated by reinforcement learning, and solves it using an explicit control law, thereby generating control commands that meet the requirements of tracking accuracy and safety, enabling autonomous driving of the vehicle in complex environments.

[0011] Optionally, the control Lyapunov-obstacle function is constructed, including: performing Taylor expansion on the vehicle nonlinear model based on the running state information corresponding to the reference point in the planned path information and the target vehicle speed to obtain a locally linear affine model; constructing a control Lyapunov function based on the tracking target; constructing a control obstacle function based on safety constraints; and weighting the control Lyapunov function and the control obstacle function to obtain the control Lyapunov-obstacle function.

[0012] In this embodiment, a Taylor series expansion of the vehicle's nonlinear dynamics model is performed, centered on the reference trajectory point. After linearization, a locally linear affine model is constructed as the basis for controller design. The path tracking accuracy requirement is modeled as a control Lyapunov function, and obstacle avoidance and safety constraints are uniformly modeled as a control obstacle function. These are then fused using a weighted approach to construct a control Lyapunov-obstacle function. For the discretized linearized controlled system, the theoretical range for weight selection is derived, avoiding the uncertainty of relying on empirical parameter adjustments. This ensures that the control law maintains millisecond-level real-time performance while providing stronger stability and theoretical guarantees for safety.

[0013] Optionally, a control Lyapunov function is constructed based on the tracking target, including: the control Lyapunov function V(x) is defined as: Where P is represented by the algebraic equation PA + A T P - PBB T P + Q = 0 n×n The calculations show that Q is a positive definite diagonal matrix of dimension n×n, and X is the vehicle state vector. ref To represent the state vector of the reference trajectory point, A and B are the Jacobian matrices at the reference point, respectively. The superscript "T" indicates the transpose of the matrix. n×n It is a zero matrix of dimension n×n.

[0014] In this embodiment, a control Lyapunov function is introduced to ensure the tracking stability of the system. The gradient of the control Lyapunov function in this method is less than zero, and the system trajectory moves towards the equilibrium point, thereby ensuring the tracking stability of the system trajectory.

[0015] Optionally, a control barrier function is constructed based on safety constraints, including: constructing the control barrier function using the Softplus function, wherein the expression of the control barrier function B(x) is: , where c is a parameter that adjusts the magnitude of the gradient of the function, and Hx-b represents the safety constraint for linearization.

[0016] In this embodiment, a control barrier function is introduced to ensure system safety. When the system approaches the state constraint boundary, the Softplus function value increases. At this time, the function gradient rapidly changes from a value close to zero to a negative value with a large absolute value, and the Softplus function value decreases, thus allowing the system to return to a safe region far from the constraint boundary. After the system moves away from the constraint boundary, the Softplus gradient approaches zero and does not affect the system's motion. The control barrier function of this disclosure has a uniform form and a large gradient at the constraint boundary, resulting in fast system convergence and stronger safety.

[0017] Optionally, the control Lyapunov function and the control barrier function are weighted to obtain the control Lyapunov-barrier function, including: adding the control Lyapunov function and the control barrier function with corresponding weights to construct the control Lyapunov-barrier function W(x), with the expression: W(x) = V(x) + λB(x) = Where λ is the weighting coefficient, and P is derived from the algebraic equation PA + A T P - PBB T P + Q = 0 n×n The calculations show that c is a parameter that adjusts the magnitude of the gradient of the function, Hx-b represents the safety constraint for linearization, and A and B are the Jacobian matrices at the reference point. The algebraic equation is PA + A T P - PBB T P + Q = 0 n×nz In the diagram, Q is a positive definite diagonal matrix of dimension n×n, and X is the vehicle state vector. ref To represent the state vector of the reference trajectory point, A and B are the Jacobian matrices at the reference point, respectively. The superscript "T" indicates the transpose of the matrix. n×n It is a zero matrix of dimension n×n.

[0018] Optionally, the conditions for the value of λ include: Let μ i Let m be the eigenvalues ​​of P, and let m = min{μ1, μ2, ..., μ...} n}, M=max{μ1,μ2,…,μ n}; for any, There is m‖x‖ 2 ≤V(x)≤M‖x‖ 2 For a set X that contains the unsafe set D, i.e. There is a balance point And B(x) = -ε, where ε is a constant and ε > 0. ; .

[0019] In this embodiment, the value of λ is limited by the above conditions to ensure that the gradient of the control Lyapunov-barrier function is less than zero. Its function value is greater than zero within the unsafe set and less than zero within the safe set. Due to the continuity of the control Lyapunov-barrier function, when the system approaches the safe constraint boundary, the control barrier function takes effect, pulling the system trajectory back to the safe region. Under the influence of the control Lyapunov function, the system trajectory will converge to the equilibrium point.

[0020] Optionally, the control Lyapunov-barrier function is solved to obtain the explicit control law, including: differentiating the control Lyapunov-barrier function to obtain the explicit control law; the explicit control law is expressed as:

[0021] ;

[0022] Where k represents the control input vector; α represents the Lie derivative of W(x) with respect to Ax; β represents the Lie derivative of W(x) with respect to B; and γ represents the parameter that adjusts the convergence of the function W(x).

[0023] In this embodiment, the derivative of the Lyapunov-barrier function W(x) over time is kept less than zero, ensuring that W(x) monotonically decreases and approaches 0, thereby guaranteeing the tracking stability and safety of the system. Furthermore, based on the fused analytical explicit control law of the Lyapunov-barrier function, no online optimization iteration is required, achieving millisecond-level computation speed and high real-time performance.

[0024] Optionally, based on an explicit control law, the vehicle operation is controlled, including converting the explicit control law into control signals for the accelerator pedal opening, brake pedal opening, and steering wheel angle, and sending them to the vehicle actuators to control the vehicle's movement.

[0025] In this embodiment, the explicit control law serves as the control input for the vehicle. By converting the explicit control law into control signals for the accelerator pedal opening, brake pedal opening, and steering wheel angle, the vehicle is controlled to achieve intelligent driving.

[0026] Optionally, the operating status information includes the vehicle's longitudinal position, lateral position, heading angle, longitudinal speed, lateral speed, and longitudinal acceleration; and / or, the planned path information includes the coordinates of path points, road curvature, and slope.

[0027] Optionally, a state space is constructed based on operational status information and planned path information, including: the state space is defined as follows: ,in, p i,x , p i,y For the first i The ordinate and abscissa of the location point K i For road curvature, θ i The slope angle is defined as a; the action space of the reinforcement learning policy model is defined as a. i =[ v i ], v i For the first i Target vehicle speed at the location point.

[0028] In this embodiment, the state space of the reinforcement learning policy model includes the first... iThe horizontal and vertical coordinates of the location point, road curvature, and slope angle are used to comprehensively describe the current driving environment. The motion space is the [missing information]. i The adaptive vehicle speed corresponding to the location point is used to output a speed decision that adapts to the current state.

[0029] Optionally, the reward function R of the reinforcement learning policy model i (s, a) is a multi-objective weighting function, expressed as:

[0030] R i (s, a)=λ1C energy + λ2C comfort + λ3C time Among them, C energy As an energy consumption bonus item, C comfort For comfort bonus items, C time For efficiency reward items, λ1, λ2, and λ3 are the weight coefficients of each reward item.

[0031] In this embodiment, the upper-layer reinforcement learning policy model constructs a reward function based on explicitly coupled road geometric features such as road slope and curvature, and designed based on vehicle tracking expected path information, to train the speed planning policy. The reward function comprehensively considers energy consumption, comfort, and travel time efficiency, and couples road curvature, slope, and other geometric features into the reward term to achieve adaptive speed decisions in complex environments. This design not only improves the policy's adaptability to unstructured scenarios but also enables the learning process to balance the multi-objective requirements of energy saving and efficiency, thereby significantly enhancing the generalization ability and robustness of the speed planning policy.

[0032] Optionally, the reinforcement learning policy model is trained using the Soft Actor-Critic algorithm. The training steps include: before performing the training loop steps, constructing a simulation environment; initializing the neural network parameters required by the Soft Actor-Critic algorithm and designing hyperparameters; creating an experience replay buffer for storing training data; and performing the training loop steps, including: collecting interaction data between the vehicle and the environment in the simulation environment and storing the interaction data in the experience replay buffer; randomly selecting a preset number of data from the experience replay buffer; updating the parameters of the Critic network and the Actor network using the loss function of the Soft Actor-Critic algorithm; and iterating the above steps until the policy performance of the Actor network converges.

[0033] In this embodiment, based on the requirement of adaptive speed optimization, the Soft Actor-Critic (SAC) algorithm is selected as the learning framework. Leveraging its policy optimization capabilities applicable to continuous action spaces and combined with maximum entropy theory, the learned policy maintains goal orientation while possessing a certain degree of randomness, thereby enhancing the generalization and robustness of the policy under different driving scenarios. Through interaction with the driving environment, the policy network parameters are continuously updated to achieve adaptive optimization of speed planning under diverse road conditions.

[0034] In some embodiments, a path tracking control device based on intelligent driving is provided, comprising:

[0035] The information acquisition module is configured to acquire the vehicle's operating status information and planned path information; the state space construction module is configured to construct the state space based on the operating status information and planned path information.

[0036] The reinforcement learning module is configured to input the state space into the reinforcement learning policy model to obtain a vehicle speed sequence based on the planned path information;

[0037] The explicit control module is configured to construct and solve the control Lyapunov-obstacle function based on the planned path information and vehicle speed sequence to obtain the explicit control law. The construction of the control Lyapunov-obstacle function includes: performing a Taylor expansion on the vehicle's nonlinear model based on the operating state information corresponding to the reference points in the planned path information and the target vehicle speed to obtain a locally linear affine model; constructing the control Lyapunov function based on the tracking target; constructing the control obstacle function based on safety constraints; and weighting the control Lyapunov function and the control obstacle function to obtain the control Lyapunov-obstacle function.

[0038] The control barrier function is constructed based on safety constraints, including: using the Softplus function to construct the control barrier function B(x), the expression of which is: , where c is a parameter that adjusts the magnitude of the gradient of the function, and Hx-b represents the safety constraint for linearization;

[0039] The control Lyapunov function and the control barrier function are weighted to obtain the control Lyapunov-barrier function, which includes: adding the control Lyapunov function and the control barrier function with corresponding weights to construct the control Lyapunov-barrier function W(x), with the expression:

[0040] W(x) = V(x) + λB(x) = ,

[0041] Where λ is the weighting coefficient, and P is derived from the algebraic equation PA + A T P - PBBT P + Q = 0 n×n The calculation shows that c is the parameter for adjusting the gradient of the function, Hx-b represents the safety constraint for linearization, and A and B are the Jacobian matrices at the reference point, respectively.

[0042] The conditions for λ to take values ​​include: Let μ i Let m be the eigenvalues ​​of P, and let m = min{μ1, μ2, ..., μ...} n}, M=max{μ1,μ2,…,μ n}; for any, There is m‖x‖ 2 ≤V(x)≤M‖x‖ 2 For a set X that contains the unsafe set D, i.e. There is a balance point And B(x) = -ε, where ε is a constant and ε > 0. ; ;

[0043] The execution module is configured to control vehicle operation based on explicit control laws.

[0044] In some embodiments, a path tracking control device based on intelligent driving is provided, including a processor and a memory storing program instructions, the processor being configured to execute the path tracking control method based on intelligent driving as described in the above embodiments when running the program instructions.

[0045] In some embodiments, a vehicle is provided, including: a vehicle body; and a path tracking control device based on intelligent driving as described in any of the above embodiments, installed on the vehicle body.

[0046] In some embodiments, a readable storage medium is provided storing program instructions that, when executed, cause a computer to perform the path tracking control method based on intelligent driving as described in the above embodiments.

[0047] This disclosure can achieve the following technical effects:

[0048] (1) This disclosure adopts a hierarchical control system. The upper layer is used to output speed planning decisions. It utilizes the autonomous learning capability of reinforcement learning algorithms to process complex geometric information such as road curvature and slope, and outputs a globally optimized speed sequence. The lower layer is used for tracking and safety control. An explicit safety controller is introduced to perform real-time constraint solving and safety verification on the upper layer output, thereby generating deterministic control actions that meet safety constraints.

[0049] (2) An analytical generation method for explicit safety control law based on the fusion of control Lyapunov-obstacle functions. By modeling the path tracking accuracy requirement as a control Lyapunov function and unifying obstacle avoidance and safety constraints as control obstacle functions, and fusing them in a weighted manner, a control Lyapunov-obstacle function is constructed. For the controlled system after discrete linearization, the theoretical range of weight selection is derived, avoiding the uncertainty of relying on empirical parameter adjustment, so that the control law has stronger stability and safety theoretical guarantees while ensuring millisecond-level real-time performance.

[0050] (3) In the upper-level reinforcement learning design, a multi-objective reward function based on road geometric features is constructed to train the speed planning strategy. The reward function comprehensively considers energy consumption, comfort, and traffic efficiency, and couples geometric features such as road curvature and slope into the reward term to achieve adaptive speed decision-making in complex environments. This design not only improves the adaptability of the strategy to unstructured scenarios, but also enables the learning process to balance the multi-objective requirements of energy saving and efficiency, thereby significantly enhancing the generalization ability and robustness of the speed planning strategy.

[0051] The above general description and the description below are exemplary and illustrative only and are not intended to limit this application. Attached Figure Description

[0052] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations and drawings do not constitute a limitation on the embodiments. Elements having the same reference numerals in the drawings are shown as similar elements. The drawings are not to be scaled. And wherein:

[0053] Figure 1 This is a schematic flowchart of the path tracking control method based on intelligent driving provided in the embodiments of this disclosure;

[0054] Figure 2 This is a schematic diagram of a process for training a reinforcement learning policy model according to an embodiment of this disclosure;

[0055] Figure 3 This is a schematic flowchart of a control Lyapunov-barrier function provided in an embodiment of the present disclosure;

[0056] Figure 4 This is a schematic diagram of the set relationship provided in the embodiments of this disclosure;

[0057] Figure 5 This is a schematic diagram illustrating the control effect of using existing technology to control the Lyapunov function (λ=0);

[0058] Figure 6 This is a schematic diagram illustrating the effect of the control law for controlling the Lyapunov-barrier function (λ=20) provided in the embodiments of this disclosure;

[0059] Figure 7 This is a schematic diagram illustrating the safety obstacle avoidance effect of the MPC tracking method and the tracking control method provided in this disclosure, as demonstrated in the embodiments of this disclosure.

[0060] Figure 8 This is a schematic diagram illustrating the real-time performance of the MPC tracking method and the tracking control method provided in this disclosure, as demonstrated in the embodiments of this disclosure.

[0061] Figure 9 This is a schematic diagram of a path tracking control device based on intelligent driving provided in an embodiment of this disclosure;

[0062] Figure 10 This is a schematic diagram of another path tracking control device based on intelligent driving provided in an embodiment of this disclosure. Detailed Implementation

[0063] To provide a more detailed understanding of the features and technical content of the embodiments of this disclosure, the implementation of the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for illustrative purposes only and are not intended to limit the embodiments of this disclosure. In the following technical description, for ease of explanation, several details are used to provide a full understanding of the disclosed embodiments. However, one or more embodiments may still be implemented without these details. In other cases, well-known structures and devices may be simplified in their depiction to simplify the drawings.

[0064] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0065] Unless otherwise stated, the term "multiple" means two or more.

[0066] In this embodiment of the disclosure, the character " / " indicates that the objects before and after it are in an "or" relationship. For example, A / B means: A or B.

[0067] The term "and / or" describes an association between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.

[0068] The term "correspondence" can refer to an association or binding relationship. The correspondence between A and B means that there is an association or binding relationship between A and B.

[0069] In some embodiments, a path tracking control device based on intelligent driving is provided, including a processor and a memory storing program instructions, the processor being configured to execute a path tracking control method based on intelligent driving as described in any of the following embodiments when running the program instructions.

[0070] In some embodiments, combined with Figure 1 As shown, a path tracking control method based on intelligent driving is provided, including:

[0071] S101, obtain vehicle operating status information and planned route information.

[0072] Optionally, the operating status information includes the vehicle's longitudinal position, lateral position, heading angle, longitudinal speed, lateral speed, and longitudinal acceleration; and / or, the planned path information includes the coordinates of path points, road curvature, and slope.

[0073] In this embodiment, the vehicle's operating status information can be acquired through an onboard sensor system. The onboard sensor system communicates with the vehicle network in real time. The onboard sensor system includes, but is not limited to, GPS (Global Positioning System), IMU (Inertial Measurement Unit), wheel speedometers, onboard radar, and onboard cameras. The vehicle's position coordinates, heading angle, and acceleration are acquired via GPS and IMU. Vehicle speed is acquired via wheel speedometers or onboard radar / camera.

[0074] Specifically, operational status information ,in, p x Indicates the longitudinal position of the vehicle; p y Indicates the lateral position of the vehicle; Indicates the vehicle's heading angle; v x Indicates the longitudinal speed of the vehicle; v y Indicates the lateral speed of the vehicle. a x This indicates the longitudinal acceleration of the vehicle.

[0075] The planned path information is generated by the vehicle's upper-level path planning module and includes a series of discrete path point sequences. Each path point's information includes, but is not limited to, its coordinates, the road curvature at that point, and the road slope angle. Thus, the planned path information not only includes geometric path details such as coordinates and curvature but also road slope information. By adding slope information, the upper-level reinforcement learning policy model can predict the power demand when going uphill and the braking demand when going downhill when planning vehicle speed, thereby generating a more energy-efficient and smoother speed sequence and improving the ability to model complex driving environments with greater precision.

[0076] In this system, the vehicle's position coordinates and planned path information are in the same coordinate system to achieve data synchronization and consistency.

[0077] S102, construct the state space based on the running status information and the planned path information.

[0078] Optionally, a state space is constructed based on operational status information and planned path information, including: the state space is defined as follows: ,in, p i,x ，p i,y For the first i The ordinate and abscissa of the location point K i For the first i Road curvature at the location point θ i The slope angle is defined as a; the action space of the reinforcement learning policy model is defined as a. i =[ v i ] ，v i For the first i Target vehicle speed at the location point.

[0079] In this embodiment, the input state space of the upper-layer reinforcement learning policy model is constructed. s i The task for reinforcement learning policy models is to learn a mapping from road features to optimal vehicle speed. Therefore, the state space... s i It can be defined as a feature strongly related to road geometry; that is, the state space is defined as the state space of the planned path. i A collection of environmental and vehicle status information at each location point. , p i,x ，p i,y These represent the vehicle's longitudinal and lateral positions relative to the origin or reference path point of the global coordinate system at the current moment. Ki For the first i The road curvature at each location point is used to describe the degree of road bending. θ i For the first i The road slope angle at each location point is used to describe the degree of road inclination. State space. s i Extract only the road geometry features most relevant to velocity planning, including the first... i The ordinates and abscissas of the location points, along with the road curvature and slope angle, enable the policy to directly perceive changes in road shape, reducing the learning dimensionality and improving the policy's generalization ability. By sequentially inputting the points along the planned path, the reinforcement learning policy model can preview the road conditions ahead.

[0080] The action space of a reinforcement learning model is a continuous one-dimensional space, and the output action is a. i =[ v i ] ，v i For the first i The target vehicle speed at the location point. The target vehicle speed is limited to a reasonable physical range, for example, the physical range is [ v min , v max = [0, 120] km / h, to ensure the feasibility and safety of the output speed.

[0081] S103, input the state space into the reinforcement learning policy model to obtain the vehicle speed sequence based on the planned path information.

[0082] In this embodiment, the constructed state space sequence s i The data is sequentially input into a pre-trained reinforcement learning policy model. The reinforcement learning policy model outputs an optimized target speed sequence that matches the road's geometric features. This speed sequence comprehensively considers factors such as road curvature and gradient, making it applicable to various complex scenarios.

[0083] Optionally, the reward function R of the reinforcement learning policy model i (s, a) is a multi-objective weighting function, expressed as:

[0084] R i (s, a)=λ1C energy + λ2C comfort + λ3C time Among them, C energy As an energy consumption bonus item, C comfort For comfort bonus items, C timeFor efficiency reward items, λ1, λ2, and λ3 are the weight coefficients of each reward item, where λ1+λ2+λ3=1.

[0085] In this embodiment, the reward function comprehensively considers energy consumption, comfort, and efficiency, guiding the reinforcement learning agent to learn a multi-objective optimal strategy through weighted summation. This scheme ensures that the final planned speed sequence not only tracks the path but also meets the actual driving needs for economy and comfort, thus improving the driving experience. This indicates the fuel consumption rate, where L is the total fuel consumption. This encourages higher cruising speeds to reduce the fuel consumption rate or energy consumption rate per unit distance. Indicating ride comfort, among which, This is the longitudinal acceleration weighting coefficient. This factor penalizes larger accelerations, such as rapid acceleration or sudden braking, thereby improving ride comfort. λ1 represents the expected travel time factor, and σ is the travel rate weighting coefficient. This term encourages higher average speeds to shorten travel time. λ1, λ2, and λ3 can be adjusted according to the priority of the three objectives of energy consumption, comfort, and travel time efficiency. For example, if energy consumption is prioritized, comfort is secondary, and travel efficiency is last, then the values ​​of λ1, λ2, and λ3 will decrease in that order.

[0086] In some embodiments, combined with Figure 2 As shown, the steps to train a reinforcement learning policy model include:

[0087] S1031, Build the simulation environment.

[0088] By constructing a simulation environment for autonomous driving, it is possible to simulate vehicle dynamics, various road geometry information, and traffic scenarios.

[0089] S1032, initialize the neural network parameters required for the Soft Actor-Critic algorithm and design the hyperparameters.

[0090] In this example, the SAC algorithm is used. The network structure includes: one Actor network (policy network), two Critic networks, Critic1 and Critic2 (Q-value networks), and two corresponding Critic networks, Critic1 and Critic2. The parameters of each neural network in the SAC algorithm, including the Actor network parameters, are initialized. Critic1 network parameters Critic2 network parameters Target Critic1 network parameters Target Critic2 network parameters And design hyperparameters, including: regularization parameters, soft update step size. Discount factor.

[0091] For example, set the regularization parameter j=-3, the soft update step size τ=0.005, and the discount factor. =0.99.

[0092] S1033, Create an experience playback buffer for storing training data.

[0093] In this example, an experience replay buffer is created to store training interaction information, continuously recording the state, action, reward, and subsequent state data at each time step. When the amount of data in the buffer reaches a set threshold, the system randomly selects several experience samples to construct a mini-batch training set for updating network parameters. This breaks down the correlation between data, improves sample utilization, and stabilizes the training process. For example, the threshold is set to 512. For instance, when the amount of data in the buffer reaches 512, 64 interaction records are randomly selected from the experience pool to form a training batch for iterative optimization of the policy network.

[0094] S1034: In the simulation environment, collect interaction data between the vehicle and the environment, and store the interaction data in the experience playback buffer.

[0095] S1035, randomly extract a preset number of data from the experience playback buffer.

[0096] S1036 updates the parameters of the Critic and Actor networks using the loss function of the Soft Actor-Critic algorithm.

[0097] In this context, Critic network updates refer to Critic1 and Critic2 networks being updated separately by minimizing the mean square Bellman error, with the target value calculated by the target network. Using two Critic networks and taking the minimum value helps reduce Q-value overestimation.

[0098] Actor network updates by maximizing expected reward and policy entropy, allowing the policy to maintain a certain degree of randomness while pursuing high returns, thus enhancing exploration capabilities.

[0099] The parameters of the two target Critic networks are slowly updated to track the parameters of their corresponding main networks, further stabilizing the training process.

[0100] S1037, iterate the above steps repeatedly until the policy performance of the Actor network converges.

[0101] In this embodiment, within a simulation environment, the Actor network controls the vehicle to interact with the environment. At each time step, the operational state information and planned path information generated by the interaction are acquired, and the state space s of the current state is constructed. t . Transform the state space s t The input is fed into the Actor network, and after global optimization, the output is the action space. The action space is located within the predetermined action space range, that is, it satisfies the condition that the physical interval is [ v min , v max ]. Execution space Then, return the reward value based on the current status. and new state values Then the state space Action space Reward Value and new status The experience is stored in the experience replay buffer. When the number of experiences in the experience replay buffer reaches 512, 64 experiences are randomly selected from the pool. These experiences are then input into the Critic1 network, Critic2 network, target Critic1 network, and target Critic2 network to calculate... , Target value and And the time difference target value is calculated using the following formula (1).

[0102] Formula (1)

[0103] Formula (2)

[0104] Formula (3)

[0105] By minimizing the loss functions in formulas (2) and (3), the Critic1 and Critic2 networks are updated. In the above formulas, r t The current state returns the reward value. t is the discount factor, j is the regularization parameter, t is the current time, and t+1 is the next time.

[0106] Sample actions using reparameterization. The Actor network is updated using the loss function in formula (4).

[0107] Formula (4)

[0108] The target Critic1 network and the target Critic2 network are updated using formulas (5) and (6) respectively, as follows:

[0109] Formula (5)

[0110] Formula (6)

[0111] The training process described above will continue to iterate multiple times until the algorithm converges. After convergence, the optimal network parameters are selected and loaded into the Actor network, thus completing the training process of the reinforcement learning policy model in this application. In the above formula, j is the regularization parameter, τ is the soft update step size, t is the current time step, and t+1 is the next time step.

[0112] S104. Based on the planned path information and vehicle speed sequence, the control Lyapunov-obstacle function is constructed and solved to obtain the explicit control law.

[0113] In some embodiments, combined with Figure 3 As shown, the steps for constructing the control Lyapunov-barrier function include:

[0114] S1041. Based on the running status information and target vehicle speed corresponding to the reference points in the planned path information, a Taylor series expansion is performed on the nonlinear vehicle model to obtain a locally linear affine model.

[0115] In this embodiment, the nonlinear vehicle dynamics model is linearized. Different degrees of freedom can be used as the vehicle dynamics model for autonomous driving control. In this example, a bicycle model is used, as shown in equation (7).

[0116] Formula (7)

[0117] in, X represents the vehicle's state vector; express The derivative with respect to time, U represents the control input vector; Indicates the longitudinal position of the vehicle; Indicates the lateral position of the vehicle; Indicates the vehicle's heading angle; Indicates the longitudinal speed of the vehicle; Indicates the lateral speed of the vehicle; Indicates the angular velocity of the vehicle; Indicates acceleration; Indicates the front wheel steering angle; Indicates the stiffness of the front wheels; Indicates the rear wheel stiffness; Iz This represents the moment of inertia of the vehicle relative to the Z-axis of its own coordinate system; l f Indicates the distance from the center of the vehicle to the front axle; This indicates the distance from the center of the vehicle to the rear axle; Indicates vehicle mass.

[0118] For the model At the reference trajectory point , Perform a Taylor series expansion at this point, ignoring higher-order terms (second order and above). This yields formula (8):

[0119] Formula (8)

[0120] Convert formula (8) to: Where A and B are the Jacobian matrices of the system at the reference point, respectively. This linearized model serves as the basis for designing explicit control laws. p x,ref Indicates the longitudinal position of the reference point; p y,ref Indicates the lateral position of the reference point; The heading angle representing the reference point; Indicates the longitudinal velocity at the reference point; Indicates the lateral velocity of the reference point; Indicates the angular velocity of the reference point; This represents the acceleration at the reference point; The front wheel steering angle represents the reference point; ΔX is the state increment, and ΔU is the input increment.

[0121] S1042, constructing a control Lyapunov function based on the tracking target.

[0122] Optionally, a control Lyapunov function is constructed based on the tracking target, including: defining the control Lyapunov function V(x), as shown in formula (9):

[0123] Formula (9)

[0124] Where P is represented by the algebraic equation PA + A T P - PBB T P + Q = 0 n×n The calculations show that Q is a positive definite diagonal matrix of dimension n×n, and X is the vehicle state vector. ref To represent the state vector of the reference trajectory point, A and B are the Jacobian matrices at the reference point, respectively. The superscript "T" indicates the transpose of the matrix. n×n It is a zero matrix of dimension n×n.

[0125] In this embodiment, the gradient of the control Lyapunov function is less than zero, and the system trajectory moves towards the equilibrium point, thereby ensuring the stability of the system trajectory tracking.

[0126] S1043, Construct a control barrier function based on safety constraints.

[0127] Optionally, a control barrier function is constructed based on safety constraints, including: constructing the control barrier function using the Softplus function, wherein the expression of the control barrier function B(x) is: Where c is a parameter that adjusts the magnitude of the function gradient, Hx-b represents the linearization safety constraint, H is a row vector used to define the normal direction of the safety constraint, and b is a scalar representing the boundary threshold of the constraint. When Hx-b=0, the system is on the safety boundary.

[0128] In constrained optimal control problems, constraints are typically established based on safety requirements. This application introduces a control obstacle function to ensure system safety. The constraints of autonomous vehicle control systems mainly fall into two categories: upper and lower bound constraints related to the physical characteristics of the system itself, and constraints related to surrounding obstacles. In this embodiment, a control obstacle function suitable for both types of constraints is constructed using a unified Softplus activation function. The original constraints are converted into exponents of exponential functions. The control obstacle function is defined as follows: As shown in formula (10):

[0129] Formula (10)

[0130] Where c is a parameter that adjusts the magnitude of the function gradient, Hx-b represents the linearization safety constraint, H is a row vector used to define the normal direction of the safety constraint, and b is a scalar representing the boundary threshold of the constraint. When Hx-b=0, the system is on the safety boundary.

[0131] B(x) is the affine transformation of the Softplus function, for all Both are strictly differentiable and continuously differentiable, avoiding the singularity problem at the boundary of the traditional logarithmic barrier function -ln(b-Hx). Since the Softplus function itself is convex, and Hx-b is an affine function, the composite function B(x) maintains its convexity. Therefore, B(x) can be used as a convex relaxation of the constraint. Its gradient is... ,Right now ,in It is the sigmoid function, and c is a parameter that adjusts the magnitude of the function's gradient. When Hx - b ≤ 0, The system is far from the constraint boundaries and has almost no additional control force. When Hx-b≥0, The system experiences a linearly increasing control force near the boundary. This reflects the adaptive constraint strength within the boundary region. This zero-linear asymptotic property ensures that B(x) hardly interferes with the master optimization in the safe region, while maintaining a linearly controllable penalty in the unsafe region.

[0132] In this way, by constructing a control barrier function using the Softplus function, the linear constraint Hx-b is smoothly handled, avoiding the non-differentiability and discontinuity problems caused by hard constraints. This results in a gentle gradient within the safe region, minimizing its impact on system performance; the gradient increases sharply near the constraint boundary, effectively pulling the system state back to the safe region. When the system approaches the state constraint boundary, the Softplus function value increases. Thus, the function gradient rapidly changes from a near-zero value to a large negative value, causing the Softplus function value to decrease, thereby returning the system to the safe region far from the constraint boundary. After the system moves far from the constraint boundary, the Softplus gradient approaches zero and does not affect the system's motion. This control barrier function ensures safe behavior of the system near the constraint boundary while having minimal impact on control performance within the safe region. Therefore, the control barrier function of this disclosure has a uniform form and a large gradient at the constraint boundary, resulting in fast system convergence and enhanced safety.

[0133] S1044, the control Lyapunov function and the control barrier function are weighted to obtain the control Lyapunov-barrier function.

[0134] Optionally, the control Lyapunov function and the control barrier function are weighted to obtain the control Lyapunov-barrier function, including: adding the control Lyapunov function and the control barrier function with corresponding weights to construct the control Lyapunov-barrier function W(x), with the expression: W(x) = V(x) + λB(x) = Where λ is the weighting coefficient, and P is derived from the algebraic equation PA + A T P - PBB T P + Q = 0 n×n The calculations show that c is a parameter that adjusts the magnitude of the gradient of the function, Hx-b represents the safety constraint for linearization, and A and B are the Jacobian matrices at the reference point. The algebraic equation is PA + A T P - PBB T P + Q = 0 n×n In the diagram, Q is a positive definite diagonal matrix of dimension n×n, and X is the vehicle state vector. ref To represent the state vector of the reference trajectory point, A and B are the Jacobian matrices at the reference point, respectively. The superscript "T" indicates the transpose of the matrix. n×n It is a zero matrix of dimension n×n.

[0135] Optionally, to improve the stability and safety of vehicle obstacle avoidance tracking, the conditions for determining the value of λ include: Let μ i Let m be the eigenvalues ​​of P, and let m = min{μ1, μ2, ..., μ...} n}, M=max{μ1,μ2,…,μ n}

[0136] (1) For any, There is m‖x‖ 2 ≤V(x)≤M‖x‖ 2 .in, It represents the space consisting of vectors composed of n real numbers, which is the n-dimensional real vector space.

[0137] (2) For a set X that contains the unsafe set D, i.e. There is a balance point And B(x) = -ε, where ε is a constant and ε > 0. .

[0138] (3) .

[0139] Condition (1) defines the boundary of the control Lyapunov function. When the system state x is far from the equilibrium point, i.e., when tracking the target, V(x) will be large; when it is close to the equilibrium point, V(x) will be small. Condition (2) defines the value of the control barrier function outside the safe boundary; X is a set slightly larger than the unsafe set D, which can be understood as a buffer or boundary region of the "danger zone". The safe zone is the portion of the entire state space excluding the dangerous zone X. B(x) = -ε means that within the safe zone, B(x) is a negative constant value of -ε. Thus, when the vehicle is within the safe zone, B(x) maintains a small negative value and will not have a dominant influence on the control law W(x). At this time, the controller mainly optimizes the tracking performance, i.e., V(x). Once the vehicle begins to approach the dangerous zone X, B(x) will increase sharply from -ε, thereby increasing W(x) and triggering the control law to pull the vehicle back to the safe zone. Condition (3) is used to calculate the critical value of the safety weight and gives the lower limit of λ to ensure stability and safety. M is the upper bound coefficient of the control Lyapunov function V(x), which comes from condition (1). The larger M is, the faster V(x) grows at a distance, and the lower the stability. c is the boundary of the dangerous zone. The maximum distance from all points to the equilibrium point is the square of c. The larger c is, the farther the danger zone may be from the target tracking point. ε comes from condition (2) and is the safe value within the safe zone. The smaller ε is, the smaller the base value set for B(x) within the safe zone is. Mc / ε measures how many times the stability requirement is greater than the basic safety requirement in the worst case, i.e., when the vehicle is on the edge of danger; the numerator Mc estimates what happens if the vehicle runs to the boundary of the danger zone. When considering the above, purely for stability, the maximum possible value of V(x) is taken as the order of magnitude; the denominator ε represents the value of the basic safety requirement B(x) within the safe area. That is, when λ > Mc / ε, the basic safety requirement will exceed the performance term V(x) at the boundary of the entire danger zone. This means that when the vehicle approaches danger, the behavior of the combined function W(x) will be dominated by B(x), and the controller will prioritize obstacle avoidance rather than tracking to ensure safety.

[0140] Specifically, the reasoning logic is as follows:

[0141] For all ,because Therefore, it can be concluded that

[0142]

[0143] For all ,because Therefore,

[0144]

[0145] Due to the continuity of W(x), when , exists as Figure 4 The set relationship shown.

[0146] Since V(x) and B(x) satisfy the control Lyapunov function and control barrier function conditions respectively and λ>0, the gradients are as follows:

[0147]

[0148] Among them, L f This represents the Lie derivative, i.e., the function W(x) along the system dynamics. f(x) directional derivative; f(x) The system dynamics function is the system evolution law without control input, as shown in the above formula (7).

[0149] The above parameter selection conditions ensure that the gradient of the control Lyapunov-barrier function is less than zero. Its function value is greater than zero within the unsafe set and less than zero within the safe set. Due to the continuity of the control Lyapunov-barrier function, when the system approaches the safe constraint boundary, the control barrier function takes effect, pulling the system trajectory back to the safe region. Under the influence of the control Lyapunov function, the system trajectory will converge to the equilibrium point.

[0150] For example, suppose the system is a simple linear system.

[0151]

[0152] Where x represents the state vector; denoted as x with respect to time; u represents the control input vector; st represents the constraint, and x1>1 represents a specific inequality constraint.

[0153] First, let's assume:

[0154] ;

[0155] ,but

[0156]

[0157] The control Lyapunov-barrier function is:

[0158] .

[0159] Combination Figure 5 The figure shows the control effect using only the Lyapunov function (λ=0), combined with... Figure 6 The effect of the control law using the control Lyapunov-barrier function (λ=20) provided in this disclosure is shown. It can be seen that the control Lyapunov-barrier function provided in this disclosure, by adding a control barrier function, can avoid entering the unsafe region and converge back to the system equilibrium point.

[0160] Optionally, the control Lyapunov-barrier function is solved to obtain the explicit control law, including: differentiating the control Lyapunov-barrier function to solve for the explicit control law.

[0161] The explicit control law expression is:

[0162] .

[0163] Where k represents the control input vector. α represents the Lie derivative of W(x) with respect to Ax. Where A is the Jacobian matrix A, x is the state vector; β represents the Lie derivative of W(x) with respect to B. B is the Jacobian matrix; γ represents the convergence parameter of the adjustment function W(x); b represents the boundary threshold of the constraint.

[0164] Based on the constructed control Lyapunov-obstacle function, the derivative of the control Lyapunov-obstacle function W(x) over time is kept less than zero, ensuring that W(x) monotonically decreases and approaches 0. This is illustrated by the following example: the stable region is described as a function whose lower bound is 0; while the dangerous region is described as a function greater than a certain positive value. The fact that the derivative of the control Lyapunov-obstacle function W(x) is less than 0 means that the system will avoid entering the dangerous region greater than the specific value, while tending towards the equilibrium point where the function value is 0. This ensures the stability and safety of the vehicle obstacle avoidance and tracking system. The control law k satisfies the requirement that the derivative of the control Lyapunov-obstacle function W(x) is less than 0, thus ensuring the tracking stability and safety of the system. Finally, the vehicle is controlled through its explicit control law k.

[0165] In this way, compared with online optimization methods such as MPC in related technologies, the solution process of the explicit control law directly obtains the analytical expression of the explicit control law through mathematical derivation. The calculation process of this control law does not require iterative optimization, and the control law k can be directly calculated based on the vehicle's operating status and planned path information, resulting in extremely low computational overhead, increased control frequency, effective response to rapidly changing dynamic environments, and greatly improved control real-time performance.

[0166] S105 controls vehicle operation based on explicit control laws.

[0167] Optionally, based on an explicit control law, the vehicle operation is controlled, including converting the explicit control law into control signals for the accelerator pedal opening, brake pedal opening, and steering wheel angle, and sending them to the vehicle actuators to control the vehicle's movement.

[0168] In this embodiment, the explicit control law k output by the lower layer includes the target longitudinal acceleration. And the target front wheel steering angle δ, that is Based on the target longitudinal acceleration, it is converted into accelerator pedal opening or brake pedal opening; based on the target front wheel angle, it is converted into steering wheel angle. The control signals for the vehicle's accelerator pedal opening or brake pedal opening and steering wheel angle are sent to the corresponding actuators through the vehicle network, thereby precisely controlling the vehicle's movement and ensuring that it travels safely and smoothly according to the speed planned by reinforcement learning and the trajectory calculated by the explicit safety controller.

[0169] Optionally, according to the acceleration command The sign and magnitude of the value are converted into an accelerator pedal opening signal (0-100%) or a brake pedal pressure signal or opening signal through a pre-calibrated mapping relationship. For example, >0 corresponds to the throttle opening. The braking force is less than 0.

[0170] Optionally, based on the front wheel steering angle command δ and parameters such as the vehicle's steering gear ratio, the target steering wheel angle is calculated and sent to the steering actuator, such as an EPS electric power steering system.

[0171] In this way, the output is converted into physical signals that can be directly applied to the vehicle's actuators: throttle opening, brake pressure or opening, and steering wheel angle. These signals are sent to the corresponding controllers via the vehicle's CAN bus or dedicated wiring harness, ultimately achieving coordinated longitudinal and lateral control of the vehicle.

[0172] In this way, after the vehicle performs an action, new operating state information and new path information, such as new curvature and slope, are collected again. This new state data is then input again into the upper-layer reinforcement learning strategy model and the lower-layer explicit control law solution steps, repeating this cycle to form a closed-loop feedback control logic, achieving continuous autonomous tracking and safety control of the vehicle. By adopting the path tracking control method based on intelligent driving provided in this disclosure, speed planning and safety tracking in the intelligent driving control logic are decoupled. A layered collaborative architecture of reinforcement learning planning layer and explicit control execution layer is adopted. The upper layer focuses on slow time-varying, forward-looking speed planning decisions. Utilizing the environmental perception and autonomous learning capabilities of RL, it processes complex geometric information such as road curvature and slope, outputting a globally optimized speed sequence to solve the adaptability problem. The lower layer focuses on fast time-varying, high real-time tracking and safety control. Utilizing the analytical and safety guarantees of explicit control theory, safety control commands are calculated within milliseconds, solving the real-time and safety issues. Thus, the tracking control method provided in this disclosure can solve the problem of high computational complexity caused by using MPC, and can overcome the problems of low security and stability of relying solely on reinforcement learning strategies.

[0173] Combination Figure 7 and Figure 8 As shown, a demonstration and verification example of the safety obstacle avoidance effect and real-time performance is given. Specifically, the interaction data corresponding to the MPC tracking method in related technologies and the tracking control method provided in this disclosure are shown respectively. Figure 7 It can be seen that the trajectories produced by the two are not significantly different. However, Figure 8 As can be seen, the single-step time of the control method of this disclosure is much lower than that of the MPC tracking method, which further illustrates that the real-time data processing efficiency of the path tracking method provided by this disclosure is better.

[0174] In some embodiments, combined with Figure 9As shown, a path tracking control device 90 based on intelligent driving is provided, comprising:

[0175] The information acquisition module 910 is configured to acquire vehicle operating status information and planned route information.

[0176] The state space construction module 920 is configured to construct the state space based on the running state information and the planned path information.

[0177] The reinforcement learning module 930 is configured to input the state space into the reinforcement learning policy model to obtain a vehicle speed sequence based on the planned path information.

[0178] The explicit control module 940 is configured to construct and solve the control Lyapunov-obstacle function based on the planned path information and vehicle speed sequence to obtain the explicit control law. The construction of the control Lyapunov-obstacle function includes: performing Taylor expansion on the vehicle nonlinear model based on the running state information corresponding to the reference point in the planned path information and the target vehicle speed to obtain a locally linear affine model; constructing the control Lyapunov function based on the tracking target; constructing the control obstacle function based on safety constraints; and weighting the control Lyapunov function and the control obstacle function to obtain the control Lyapunov-obstacle function.

[0179] The control barrier function is constructed based on safety constraints, including: using the Softplus function to construct the control barrier function B(x), the expression of which is: Where c is a parameter that adjusts the magnitude of the function gradient, Hx-b represents the linearization safety constraint, H is a row vector used to define the normal direction of the safety constraint, and b is a scalar representing the boundary threshold of the constraint. When Hx-b=0, the system is on the safety boundary;

[0180] The control Lyapunov function and the control barrier function are weighted to obtain the control Lyapunov-barrier function, which includes: adding the control Lyapunov function and the control barrier function with corresponding weights to construct the control Lyapunov-barrier function W(x), with the expression:

[0181] W(x) = V(x) + λB(x) = ,

[0182] Where λ is the weighting coefficient, and P is derived from the algebraic equation PA + A T P - PBB T P + Q = 0 n×n The calculation shows that c is the parameter for adjusting the gradient of the function, Hx-b represents the safety constraint for linearization, and A and B are the Jacobian matrices at the reference point, respectively.

[0183] The conditions for λ to take values ​​include: Let μ i Let m be the eigenvalues ​​of P, and let m = min{μ1, μ2, ..., μ...} n}, M=max{μ1,μ2,…,μ n}; for any, There is m‖x‖ 2 ≤V(x)≤M‖x‖ 2 For a set X that contains the unsafe set D, i.e. There is a balance point And B(x) = -ε, where ε is a constant and ε > 0. ; .

[0184] The execution module 950 is configured to control vehicle operation based on explicit control laws.

[0185] Combination Figure 10 As shown, this disclosure provides a path tracking control device 100 for intelligent driving, including a processor 1000 and a memory 1001. Optionally, the device 100 may further include a communication interface 1002 and a bus 1003. The processor 1000, communication interface 1002, and memory 1001 can communicate with each other via the bus 1003. The communication interface 1002 can be used for information transmission. The processor 1000 can call logical instructions in the memory 1001 to execute the path tracking control method for intelligent driving described in the above embodiment.

[0186] Furthermore, the logic instructions in the aforementioned memory 1001 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0187] The memory 1001, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as program instructions / modules corresponding to the methods in the embodiments of this disclosure. The processor 1000 executes functional applications and data processing by running the program instructions / modules stored in the memory 1001, thereby implementing the path tracking control method based on intelligent driving in the above embodiments.

[0188] The memory 1001 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function. The data storage area may store data created based on the use of the terminal device. Furthermore, the memory 1001 may include high-speed random access memory and may also include non-volatile memory.

[0189] This disclosure provides a vehicle, including a vehicle body and a path tracking control device based on intelligent driving as described in any of the above embodiments. The path tracking control device based on intelligent driving as described in any of the above embodiments is installed on the vehicle body. The installation relationship described herein is not limited to placement inside the vehicle body, but also includes installation and connection with other components of the vehicle, including but not limited to physical connections, electrical connections, or signal transmission connections. Those skilled in the art will understand that the path tracking control device based on intelligent driving as described in any of the above embodiments can be adapted to feasible vehicle bodies to achieve other feasible embodiments.

[0190] In some embodiments, a vehicle is provided, including a vehicle body. A path tracking control device based on intelligent driving, as described in any of the above embodiments, is installed on the vehicle body.

[0191] In some embodiments, a readable storage medium is provided storing program instructions that, when executed, cause a computer to perform the path tracking control method based on intelligent driving as described in the above embodiments.

[0192] The technical solutions of this disclosure can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in this disclosure. The aforementioned storage medium can be a non-transitory storage medium, such as a USB flash drive, external hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc., and other media capable of storing program code.

[0193] The foregoing description and accompanying drawings fully illustrate embodiments of this disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, procedural, and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. Moreover, the terminology used in this application is for describing embodiments only and is not intended to limit the claims. As used in the description of embodiments and claims, the singular forms “a,” “an,” and “the” are intended to equally include the plural forms unless the context clearly indicates otherwise. Similarly, the term “and / or” as used in this application means including one or more of the associated listed items and all possible combinations thereof. Additionally, when used in this application, the term "comprise" and its variations "comprises" and / or "comprising" refer to the presence of stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. Without further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, or apparatus that includes said element. In this document, each embodiment may focus on the differences from other embodiments, and similar or identical parts between embodiments can be referred to mutually. For methods, products, etc., disclosed in the embodiments, if they correspond to the method section disclosed in the embodiments, the relevant parts can be referred to the description of the method section.

[0194] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this disclosure. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0195] The methods and products disclosed in the embodiments herein (including but not limited to devices and equipment) can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units may be merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to implement this embodiment according to actual needs. In addition, the functional units in the embodiments of this disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0196] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. Each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

Claims

1. A path tracking control method based on intelligent driving, characterized in that, include: Obtain vehicle operating status information and planned route information; A state space is constructed based on the coordinates of path points, road curvature, and slope from the path planning information, including: the state space is defined as follows: ,in, p i,x , p i,y For the first i The ordinate and abscissa of the location point K i For road curvature, θ i Let be the slope angle; input the state space into the reinforcement learning policy model to obtain the vehicle speed sequence based on the planned path information; where the action space of the reinforcement learning policy model is defined as 'a'. i =[ v i ], v i For the first i Target vehicle speed at the location point; Based on the planned path information and vehicle speed sequence, a control Lyapunov-obstacle function is constructed and solved to obtain an explicit control law. The construction of the control Lyapunov-obstacle function includes: performing a Taylor expansion on the vehicle's nonlinear model based on the operating state information corresponding to the reference points in the planned path information and the target vehicle speed to obtain a locally linear affine model; constructing a control Lyapunov function based on the tracking target; constructing a control obstacle function based on safety constraints; and weighting the control Lyapunov function and the control obstacle function to obtain the control Lyapunov-obstacle function. The control barrier function is constructed based on safety constraints, including: using the Softplus function to construct the control barrier function B(x), the expression of which is: , where c is a parameter that adjusts the magnitude of the gradient of the function, and Hx-b represents the safety constraint for linearization; The control Lyapunov function and the control barrier function are weighted to obtain the control Lyapunov-barrier function, which includes: adding the control Lyapunov function and the control barrier function with corresponding weights to construct the control Lyapunov-barrier function W(x), with the expression: W(x)=V(x)+λB(x)= , Where λ is the weighting coefficient, and P is derived from the algebraic equation PA + A T P - PBB T P + Q = 0 n×n The calculation shows that c is the parameter for adjusting the gradient of the function, Hx-b represents the safety constraint for linearization, and A and B are the Jacobian matrices at the reference point, respectively. The conditions for λ to take values ​​include: Let μ i Let m be the eigenvalues ​​of P, and let m = min{μ1, μ2, ..., μ...} n }, M=max{μ1,μ2,…,μ n }; for any, There is m‖x‖ 2 ≤V(x)≤M‖x‖ 2 For a set X that contains the unsafe set D, i.e. There is a balance point And B(x) = -ε, where ε is a constant and ε >

0. ; ; The vehicle operation is controlled based on explicit control laws.

2. The method according to claim 1, characterized in that, Based on the target being tracked, a control Lyapunov function is constructed, including: The control Lyapunov function V(x) is defined as follows: ， Where P is represented by the algebraic equation PA + A T P - PBB T P + Q = 0 n×n The calculations show that Q is a positive definite diagonal matrix of dimension n×n, and X is the vehicle state vector. ref To represent the state vector of the reference trajectory point, A and B are the Jacobian matrices at the reference point, respectively. The superscript "T" indicates the matrix transpose operation. n×n It is a zero matrix of dimension n×n.

3. The method according to claim 1 or 2, characterized in that, Solving for the control Lyapunov-barrier function yields the explicit control law, including: Differentiate the control Lyapunov-barrier function to solve for the explicit control law; The explicit control law is expressed as: ; Where k represents the control input vector; α represents the Lie derivative of W(x) with respect to Ax; β represents the Lie derivative of W(x) with respect to B; and γ represents the parameter that adjusts the convergence of the function W(x).

4. The method according to claim 1 or 2, characterized in that, Based on explicit control laws, vehicle operation is controlled, including: The explicit control law is converted into control signals for the accelerator pedal opening, brake pedal opening, and steering wheel angle, and sent to the vehicle actuators to control the vehicle's movement.

5. The method according to claim 1 or 2, characterized in that, The operating status information includes the vehicle's longitudinal position, lateral position, heading angle, longitudinal velocity, lateral velocity, and longitudinal acceleration.

6. The method according to claim 1 or 2, characterized in that, The reward function R of the reinforcement learning policy model i (s, a) is a multi-objective weighting function, expressed as: R i (s,a)=λ1 C energy + λ2 C comfort + λ3 C time ; Among them, C energy As an energy consumption bonus item, C comfort For comfort bonus items, C time For efficiency reward items, λ1, λ2, and λ3 are the weight coefficients of each reward item.

7. The method according to claim 1 or 2, characterized in that, The reinforcement learning policy model is trained using the Soft Actor-Critic algorithm. The training steps include: Before proceeding with the training loop steps, the following steps are included: building the simulation environment; initializing the neural network parameters required by the Soft Actor-Critic algorithm and designing the hyperparameters; and creating an experience replay buffer to store the training data. The training loop steps include: collecting interaction data between the vehicle and the environment in the simulation environment, and storing the interaction data in the experience playback buffer; A preset number of data points are randomly selected from the experience playback buffer; The parameters of the Critic network and the Actor network are updated using the loss function of the Soft Actor-Critic algorithm; Repeat the above steps until the policy performance of the Actor network converges.

8. A path tracking control device based on intelligent driving, characterized in that, include: The information acquisition module is configured to acquire vehicle operating status information and planned route information; The state space construction module is configured to construct a state space based on the coordinates of path points, road curvature, and slope in the path planning information, including: the state space is defined as follows: ,in, p i,x , p i,y For the first i The ordinate and abscissa of the location point K i For road curvature, θ i The slope angle; The reinforcement learning module is configured to input the state space into the reinforcement learning policy model to obtain a vehicle speed sequence based on the planned path information; wherein, the action space of the reinforcement learning policy model is defined as a. i =[ v i ], v i For the first i Target vehicle speed at the location point; The explicit control module is configured to construct and solve the control Lyapunov-obstacle function based on the planned path information and vehicle speed sequence to obtain the explicit control law. The construction of the control Lyapunov-obstacle function includes: performing a Taylor expansion on the vehicle's nonlinear model based on the operating state information corresponding to the reference points in the planned path information and the target vehicle speed to obtain a locally linear affine model; constructing the control Lyapunov function based on the tracking target; constructing the control obstacle function based on safety constraints; and weighting the control Lyapunov function and the control obstacle function to obtain the control Lyapunov-obstacle function. The control barrier function is constructed based on safety constraints, including: using the Softplus function to construct the control barrier function B(x), the expression of which is: , where c is a parameter that adjusts the magnitude of the gradient of the function, and Hx-b represents the safety constraint for linearization; The control Lyapunov function and the control barrier function are weighted to obtain the control Lyapunov-barrier function, which includes: adding the control Lyapunov function and the control barrier function with corresponding weights to construct the control Lyapunov-barrier function W(x), with the expression: W(x)=V(x)+λB(x)= , Where λ is the weighting coefficient, and P is derived from the algebraic equation PA + A T P - PBB T P + Q = 0 n×n The calculation shows that c is the parameter for adjusting the gradient of the function, Hx-b represents the safety constraint for linearization, and A and B are the Jacobian matrices at the reference point, respectively. The conditions for λ to take values ​​include: Let μ i Let m be the eigenvalues ​​of P, and let m = min{μ1, μ2, ..., μ...} n }, M=max{μ1,μ2,…,μ n }; for any, There is m‖x‖ 2 ≤V(x)≤M‖x‖ 2 For a set X that contains the unsafe set D, i.e. There is a balance point And B(x) = -ε, where ε is a constant and ε >

0. ; ; The execution module is configured to control vehicle operation based on explicit control laws.

9. A path tracking control device based on intelligent driving, comprising a processor and a memory storing program instructions, characterized in that, The processor is configured to execute the path tracking control method based on intelligent driving as described in any one of claims 1 to 7 when running the program instructions.

10. A vehicle, characterized in that, include: Vehicle body; The path tracking control device based on intelligent driving as described in claim 8 or 9 is installed on the vehicle body.

11. A readable storage medium storing program instructions, characterized in that, When the program instructions are executed, they cause the computer to perform the path tracking control method based on intelligent driving as described in any one of claims 1 to 7.

Images