Vehicle anti-aiming control methods, devices, electronic equipment, storage media and programs
By establishing and discretizing a vehicle dynamics model, and combining it with an Actor-Critic network and an extended Kalman filter, vehicle preview control commands are generated. This solves the problems of poor applicability and insufficient control accuracy of path tracking control methods for autonomous vehicles, and achieves efficient vehicle preview control.
Patent Information
- Application Number
- CN202511442511.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-10
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2045-10-10
AI Technical Summary
The path tracking control method for autonomous vehicles has poor applicability, insufficient control accuracy, high engineering implementation difficulty, and requires retraining or online adaptation when system parameters change, resulting in high costs.
By acquiring vehicle driving state parameters, a vehicle dynamics model is established and discretized to construct an augmented dynamics model. An Actor-Critic network is used to approximate the optimal tracking control law online. Combined with the extended Kalman filter method, parameter identification is performed to generate vehicle aiming control commands.
This improves the control accuracy and applicability of the vehicle anti-aiming control method, enabling it to adapt quickly to changes in system parameters and reducing the difficulty of engineering implementation.
Smart Images

Figure CN121348740B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of vehicle technology, and in particular to a vehicle anti-aiming control method, device, electronic device, storage medium, and program. Background Technology
[0002] In the field of autonomous vehicle path tracking control, a well-designed vehicle path tracking controller can effectively utilize perception information to achieve accurate, smooth, and responsive tracking performance.
[0003] In related technologies, the pre-aiming path tracking control method is mainly adopted. By integrating future path information into the state vector, an enhanced linear quadratic optimization problem is constructed, thereby generating the optimal control law in analytical form. Furthermore, reinforcement learning methods are introduced to solve problems such as nonlinearity and parameter uncertainty in practical systems. Designed based on optimal control theory, it has the ability to handle high-dimensional and nonlinear complex systems and can directly approximate the optimal strategy through trial and error learning.
[0004] However, the relevant technologies rely on high-fidelity simulation platforms, carefully designed reward functions, and a large amount of trial and error interaction, making engineering implementation difficult. At the same time, once the vehicle's system parameters change, the trained strategies are prone to failure, requiring retraining or costly online adaptation. This results in poor applicability and insufficient control accuracy of the path tracking control method for autonomous vehicles, which urgently needs to be addressed. Summary of the Invention
[0005] This invention provides a vehicle preview control method, device, electronic device, storage medium, and program to solve the problems of poor applicability and insufficient control accuracy of path tracking control methods for autonomous vehicles in related technologies, thereby improving the control accuracy and applicability of vehicle preview control methods.
[0006] A first aspect of the present invention provides a vehicle anti-aiming control method, comprising the following steps: acquiring vehicle driving state parameters; establishing a vehicle dynamics model based on the vehicle driving state parameters; discretizing the vehicle dynamics model using a preset Euler discretization method to obtain a discretized vehicle dynamics model; obtaining an augmented dynamics model based on the vehicle driving state parameters and the discretized vehicle dynamics model; and obtaining an optimal tracking control law based on the augmented dynamics model; approximating the optimal tracking control law online using a pre-constructed Actor-Critic network to obtain optimized vehicle control parameters; establishing nonlinear state equations and observation equations based on a preset extended Kalman filter method; discretizing the nonlinear state equations and the observation equations respectively; obtaining parameter identification results based on the discretized nonlinear state equations and the discretized observation equations; obtaining a vehicle anti-aiming control command based on the pre-constructed Actor-Critic network and the parameter identification results; and performing anti-aiming control on the vehicle based on the control command.
[0007] Further, in some embodiments, the steps of acquiring vehicle driving state parameters, establishing a vehicle dynamics model based on the vehicle driving state parameters, discretizing the vehicle dynamics model using a preset Euler discretization method to obtain a discretized vehicle dynamics model, obtaining an augmented dynamics model based on the vehicle driving state parameters and the discretized vehicle dynamics model, and obtaining the optimal tracking control law of the augmented dynamics model include: acquiring vehicle lateral offset and heading angle error; establishing the vehicle dynamics model based on the vehicle lateral offset and heading angle error; discretizing the vehicle dynamics model according to a preset control period using a preset Euler discretization method to obtain a compact discrete form of the vehicle dynamics model; obtaining the augmented dynamics model based on the current road curvature and the road curvature at the next moment, wherein the augmented dynamics model includes a utility function and a value function; and obtaining the optimal tracking control law of the augmented dynamics model based on the value function.
[0008] Further, in some embodiments, the step of approximating the optimal tracking control law online based on a pre-built Actor-Critic network to obtain optimized vehicle control parameters includes: acquiring the current augmented state parameters, historical augmented state parameters, and control input parameters of the augmented dynamics model; updating the time difference error of the current augmented state parameters, the historical augmented state parameters, and the control input parameters based on the Critic network in the pre-built Actor-Critic network to obtain the learning law of the Critic network; acquiring the learning rate of the weights of each layer of the Critic network; and updating the Actor network parameters in the pre-built Actor-Critic network by minimizing the value function of the augmented dynamics model based on the learning rate of the weights of each layer of the Critic network, the current augmented state parameters, the historical augmented state parameters, and the control input parameters to obtain the learning law of the Actor network; and obtaining the optimized vehicle control parameters based on the learning law of the Critic network and the learning law of the Actor network.
[0009] Furthermore, in some embodiments, obtaining vehicle pre-aiming control commands based on the pre-constructed Actor-Critic network and the parameter identification results, and performing pre-aiming control on the vehicle based on the control commands, includes: obtaining a first dynamic term in the learning law of the Actor network and a second dynamic term in the learning law of the Critic network based on the extended Kalman filter method; obtaining vehicle pre-aiming control commands based on the first dynamic term, the second dynamic term, and the parameter identification results, and performing pre-aiming control on the vehicle based on the control commands.
[0010] Furthermore, in some embodiments, before establishing the nonlinear state equation and observation equation based on the preset extended Kalman filter method, the method further includes: establishing the full dynamic equation of the controlled object based on the vehicle driving state parameters, and determining the scaling factor according to the full dynamic equation; and obtaining the full dynamic equation of a new controlled object based on the scaling factor and the full dynamic equation of the controlled object.
[0011] The vehicle preview control method provided by this invention obtains vehicle driving state parameters, establishes a vehicle dynamics model, and obtains a discretized model through Euler discretization. An augmented dynamics model is constructed based on the parameters, and its optimal tracking control law is obtained. Then, an Actor-Critic network is used to approximate the control law online to obtain optimized vehicle control parameters. Finally, based on extended Kalman filtering, nonlinear state equations and observation equations are established and discretized to obtain parameter identification results. Combined with the Actor-Critic network, vehicle preview control commands are generated to achieve vehicle preview control. This solves the problems of poor applicability and insufficient control accuracy of path tracking control methods for autonomous vehicles in related technologies, and improves the control accuracy and applicability of the vehicle preview control method.
[0012] A second aspect of the present invention provides a vehicle pre-aiming control device, the device comprising: a mathematical modeling module, configured to acquire vehicle driving state parameters, establish a vehicle dynamics model based on the vehicle driving state parameters, discretize the vehicle dynamics model using a preset Euler discretization method to obtain a discretized vehicle dynamics model, and obtain an augmented dynamics model based on the vehicle driving state parameters and the discretized vehicle dynamics model, and obtain an optimal tracking control law based on the augmented dynamics model; an optimization module, configured to approximate the optimal tracking control law online according to a pre-constructed Actor-Critic network to obtain optimized vehicle control parameters; and a control module, configured to establish a nonlinear state equation and an observation equation based on a preset extended Kalman filter method, and discretize the nonlinear state equation and the observation equation respectively, and obtain parameter identification results based on the discretized nonlinear state equation and the discretized observation equation, obtain a vehicle pre-aiming control command based on the pre-constructed Actor-Critic network and the parameter identification results, and perform pre-aiming control on the vehicle based on the control command.
[0013] Further, in some embodiments, the mathematical modeling module is specifically used for: acquiring the vehicle's lateral offset and heading angle error; establishing the vehicle dynamics model based on the vehicle's lateral offset and heading angle error; discretizing the vehicle dynamics model according to a preset control period based on a preset Euler discretization method to obtain a compact discrete form of the vehicle dynamics model; obtaining an augmented dynamics model based on the current road curvature and the road curvature at the next moment, wherein the augmented dynamics model includes a utility function and a value function; and obtaining the optimal tracking control law of the augmented dynamics model based on the value function.
[0014] Further, in some embodiments, the optimization module is specifically used for: obtaining the current augmented state parameters, historical augmented state parameters, and control input parameters of the augmented dynamics model; updating the time difference error of the current augmented state parameters, the historical augmented state parameters, and the control input parameters based on the Critic network in the pre-built Actor-Critic network to obtain the learning law of the Critic network; obtaining the learning rate of the weights of each layer of the Critic network; and updating the Actor network parameters in the pre-built Actor-Critic network by minimizing the value function of the augmented dynamics model based on the learning rate of the weights of each layer of the Critic network, the current augmented state parameters, the historical augmented state parameters, and the control input parameters to obtain the learning law of the Actor network; and obtaining the optimized vehicle control parameters according to the learning law of the Critic network and the learning law of the Actor network.
[0015] Furthermore, in some embodiments, the control module is specifically used to: obtain a first dynamic term in the learning law of the Actor network and a second dynamic term in the learning law of the Critic network based on the extended Kalman filter method; obtain a vehicle pre-aiming control command based on the first dynamic term, the second dynamic term and the parameter identification result, and perform pre-aiming control on the vehicle based on the control command.
[0016] Furthermore, in some embodiments, before establishing the nonlinear state equation and observation equation based on the preset extended Kalman filter method, the control module is also used to: establish the full dynamic equation of the controlled object based on the vehicle driving state parameters, and determine the scaling factor according to the full dynamic equation; and obtain the full dynamic equation of a new controlled object based on the scaling factor and the full dynamic equation of the controlled object.
[0017] According to the vehicle pre-aiming control device provided in the embodiments of the present invention, vehicle driving state parameters are acquired, a vehicle dynamics model is established and a discretized model is obtained through Euler discretization, an augmented dynamics model is constructed in combination with the parameters and its optimal tracking control law is obtained; then, the control law is approximated online using an Actor-Critic network to obtain optimized vehicle control parameters; finally, nonlinear state equations and observation equations are established and discretized based on extended Kalman filtering to obtain parameter identification results, and vehicle pre-aiming control commands are generated in combination with the Actor-Critic network to realize vehicle pre-aiming control. This solves the problems of poor applicability and insufficient control accuracy of path tracking control methods for autonomous vehicles in related technologies, and improves the control accuracy and applicability of vehicle pre-aiming control methods.
[0018] A third aspect of the present invention provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the vehicle anti-aiming control method described in the above embodiments.
[0019] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the vehicle aiming control method as described in the above embodiments.
[0020] A fifth aspect of the present invention provides a computer program product, including a computer program that is executed to implement the vehicle anti-aiming control method as described in the above embodiments.
[0021] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description
[0022] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 A flowchart of a vehicle anti-aiming control method provided according to an embodiment of the present invention; Figure 2 This is a schematic diagram of an Actor network structure according to a specific embodiment of the present invention; Figure 3 A schematic diagram of a Critic network structure provided according to a specific embodiment of the present invention; Figure 4 This is a schematic diagram of the weight change curve of an Actor network according to a specific embodiment of the present invention; Figure 5 A schematic diagram of a self-learning control framework provided according to a specific embodiment of the present invention; Figure 6 This is a comparative curve diagram of a vehicle anti-aiming control method provided according to a specific embodiment of the present invention; Figure 7 This is a schematic diagram illustrating the online identification results of the scaling factor and the distance from the centroid of the new controlled object to the front axis, according to a specific embodiment of the present invention. Figure 8 This is a block diagram of a vehicle anti-aiming control device provided according to an embodiment of the present invention; Figure 9 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present invention. Detailed Implementation
[0023] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.
[0024] The vehicle preview control method, device, electronic device, storage medium, and program of the present invention are described below with reference to the accompanying drawings. Addressing the problems of poor applicability and insufficient control accuracy in related technologies for autonomous vehicles mentioned in the background section, the present invention provides a vehicle preview control method. This method acquires vehicle driving state parameters, establishes a vehicle dynamics model, and obtains a discretized model through Euler discretization. An augmented dynamics model is constructed based on the parameters, and its optimal tracking control law is obtained. Then, an Actor-Critic network is used to approximate the control law online, obtaining optimized vehicle control parameters. Finally, based on extended Kalman filtering, nonlinear state equations and observation equations are established and discretized to obtain parameter identification results. Combined with the Actor-Critic network, vehicle preview control commands are generated to achieve vehicle preview control. This solves the problems of poor applicability and insufficient control accuracy in related technologies for autonomous vehicles, improving the control accuracy and applicability of the vehicle preview control method.
[0025] Specifically, Figure 1 This is a flowchart of a vehicle aiming control method provided according to an embodiment of the present invention.
[0026] like Figure 1 As shown, the vehicle anti-aiming control method includes the following steps: In step S101, vehicle driving state parameters are obtained, a vehicle dynamics model is established based on the vehicle driving state parameters, the vehicle dynamics model is discretized using a preset Euler discretization method to obtain a discretized vehicle dynamics model, and an augmented dynamics model is obtained based on the vehicle driving state parameters and the discretized vehicle dynamics model, and the optimal tracking control law of the augmented dynamics model is obtained based on the augmented dynamics model.
[0027] Among them, the vehicle dynamics model refers to the mathematical model describing vehicle trajectory tracking control; the preset Euler discretization method refers to the method of decomposing continuous state changes into discrete time step state updates by using forward difference to approximate the derivative of continuous time; the augmented dynamics model refers to the augmented discretized vehicle dynamics model formed by extending future path information (pre-aimed road curvature) into the original state vector; and the optimal tracking control law refers to the calculation rule of the control command that optimizes the vehicle control index.
[0028] Further, in some embodiments, vehicle driving state parameters are obtained, a vehicle dynamics model is established based on the vehicle driving state parameters, and the vehicle dynamics model is discretized using a preset Euler discretization method to obtain a discretized vehicle dynamics model. An augmented dynamics model is obtained based on the vehicle driving state parameters and the discretized vehicle dynamics model, and the optimal tracking control law of the augmented dynamics model is obtained based on the augmented dynamics model. This includes: obtaining the vehicle's lateral offset and heading angle error; establishing a vehicle dynamics model based on the vehicle's lateral offset and heading angle error; discretizing the vehicle dynamics model according to a preset control period using a preset Euler discretization method to obtain a compact discrete form of the vehicle dynamics model; obtaining the augmented dynamics model based on the current road curvature and the road curvature at the next moment, wherein the augmented dynamics model includes a utility function and a value function; and obtaining the optimal tracking control law of the augmented dynamics model based on the value function.
[0029] Among them, vehicle lateral offset refers to the lateral position deviation of the vehicle's center of gravity relative to the target driving trajectory; vehicle heading angle refers to the angle between the vehicle's own driving direction and the tangent direction of the target road trajectory; preset control cycle refers to the fixed time interval between the calculation and output of control commands by the on-board ECU; and the value function is the optimal performance index function, which refers to the cumulative minimum value of the utility function at all future moments when following the optimal control strategy starting from the current augmented state.
[0030] Specifically, in this embodiment of the invention, the lateral offset and heading angle error of the vehicle are first obtained through a sensing system. Using these two parameters as the core state vector, and combined with the vehicle's inherent parameters and longitudinal velocity, a continuous-time vehicle dynamics model is constructed based on Newton's laws of motion and the theorem of angular momentum. Then, according to a preset control cycle, the Euler discretization method is used to approximate the state derivative with forward difference, transforming the continuous model into a compact discrete form. The system, control, and disturbance matrices are derived from the original continuous model matrix and the control cycle, describing the update relationship between the state, control command, and current road curvature in adjacent discrete time steps. Subsequently, the road curvature (i.e., pre-aiming information) of the current and next time steps is extended to the discrete model state vector, and the equations are reconstructed to form an augmented dynamics model. At the same time, a quadratic utility function containing augmented state weights, control input weights, and discount factors is defined, and a value function (i.e., the accumulated value of future optimal utility under the current augmented state) is defined based on the Bellman equation. Finally, the partial derivative of the value function with respect to the control command is taken and set to zero. The future value function term is eliminated by combining the augmented model state update equation, and the optimal tracking control law of the augmented dynamics model in analytical form is derived.
[0031] For example, in this embodiment of the invention, the lateral offset and heading angle error of the vehicle are used as state vectors to establish a single-vehicle dynamics model (Dynamic Bicycle Model) as the vehicle dynamics model. After derivation, it can be written in the form of error dynamics: ; ; ; ; in, For system status, This represents the lateral offset of the centroid relative to the target trajectory. This represents the vehicle's heading angle error relative to the road. For vehicle quality, Let yaw moment of inertia be the moment of inertia of the vehicle. This represents the longitudinal speed of the vehicle. This is the distance from the center of gravity to the front axle. This is the distance from the center of mass to the rear axle. For the front wheel lateral stiffness, This refers to the rear wheel lateral stiffness. This refers to the front wheel steering angle, which also corresponds to the control command. The curvature of the road at the current location.
[0032] Furthermore, based on the Euler discretization method, the system is divided according to the control period. Discretization transforms the system into a compact discrete form: ; in By introducing the concept of advance aiming control, the original system state is obtained by considering the road curvature augmentation at the current and next time moments. ,in, To determine the aiming length, the augmented dynamics system is reconstructed: ; Furthermore, we define a utility function in quadratic form. ,in It is a symmetric positive definite matrix. It is a positive constant; according to the Bellman equations for discrete systems, the utility function of the augmented system is... and value function for: ; ; in, As a discount factor, Defined as: ; Furthermore, according to optimal control theory, the value function is made... Control commands Since the partial derivative is zero, the optimal tracking control law for the augmented system can be obtained as follows: ; in, For function pairs of control instructions The partial derivatives of .
[0033] It should be noted that in practical applications, the value function Not easy to obtain, and Because the modeling error is not accurate enough, it is necessary to introduce the Actor-Critic network in reinforcement learning to approximate the optimal tracking control law online.
[0034] In step S102, the optimal tracking control law is approximated online based on the pre-constructed Actor-Critic network to obtain the optimized vehicle control parameters.
[0035] Among them, the Actor-Critic network refers to a reinforcement learning algorithm that combines a decision network (Actor network) and a value network (Critic network) with policy gradient methods and value function estimation.
[0036] It should be noted that, due to the high state dimension of augmented systems, directly using the Critic neural network to fit the system's value function or state behavior value function will not fully utilize the dynamic structural characteristics of the augmented system, resulting in the need for a large amount of data trial and error to approximate the actual value function of the system. Therefore, this invention is based on the fitting characteristics of the Critic in the Dual Heuristic Dynamic Programming (DHP) method, and uses the Critic to learn the mapping of the value function to the state partial derivatives, so as to introduce more system knowledge into the learning process and accelerate the efficiency of online learning.
[0037] Furthermore, in some embodiments, the optimal tracking control law is approximated online based on a pre-built Actor-Critic network to obtain optimized vehicle control parameters. Obtaining optimized vehicle control parameters includes: acquiring the current augmented state parameters, historical augmented state parameters, and control input parameters of the augmented dynamics model; updating the time difference error of the current augmented state parameters, historical augmented state parameters, and control input parameters based on the Critic network in the pre-built Actor-Critic network to obtain the learning law of the Critic network; acquiring the learning rate of the weights of each layer of the Critic network; and updating the Actor network parameters in the pre-built Actor-Critic network by minimizing the value function of the augmented dynamics model based on the learning rate of the weights of each layer of the Critic network, the current augmented state parameters, historical augmented state parameters, and control input parameters to obtain the learning law of the Actor network; and obtaining the optimized vehicle control parameters based on the learning law of the Critic network and the learning law of the Actor network.
[0038] Here, the current augmented state parameter refers to the state vector of the augmented dynamics model at the current discrete time step, the historical augmented state parameter refers to the state vector of the augmented dynamics model at past discrete time steps, the time difference error refers to the difference between the value estimate of the current augmented state and the value estimate of the historical augmented state plus the immediate reward generated by the control input, the learning law of the Critic network refers to the update rule of the Critic network parameters, and the learning law of the Actor network refers to the update rule of the Actor network parameters.
[0039] Specifically, this embodiment of the invention first obtains the augmented state parameters of the augmented dynamics model at the current discrete time step, the historical augmented state parameters of past time steps, and the control input parameters of the corresponding time steps. These parameters are then input into the Critic network within a pre-constructed Actor-Critic network. The Critic network calculates the time difference error (i.e., quantization value estimation bias) corresponding to the current and historical augmented states and control inputs. Based on this error, the update rule for the Critic network parameters is derived, resulting in the learning law of the Critic network. Subsequently, the learning rate of the weights of each layer of the Critic network is obtained. Combining the current augmented state parameters, historical augmented state parameters, and control input parameters, with the goal of minimizing the value function of the augmented dynamics model, the update rule for the Actor network parameters is derived by calculating the gradient of the value function with respect to the Actor network parameters, resulting in the learning law of the Actor network. Finally, the Critic network parameters are updated according to the learning law of the Critic network to optimize the value assessment, and the Actor network parameters are updated according to the learning law of the Actor network to optimize the control decision output. The two work together to obtain the optimized vehicle control parameters.
[0040] Figure 2 This is a schematic diagram of an Actor network structure according to a specific embodiment of the present invention. Figure 3 This is a schematic diagram of a Critic network structure provided according to a specific embodiment of the present invention, such as... Figure 2 As shown, These are the network weights from the input layer to the hidden layer of the Actor network. For example, the network weights from the hidden layer to the output layer of the Actor network, such as Figure 3 As shown, These are the network weights from the input layer to the hidden layer of the Critic network. These are the network weights from the hidden layer to the output layer of the Critic network.
[0041] It should be noted that, due to The system state at any given moment is difficult to estimate; therefore, this invention uses the current moment. And an approximate derivation of the learning law using historical moment data.
[0042] For example, the Critic network minimizes the augmented state acquired at the current time step. The augmented state of historical moments and control input The time difference error is updated accordingly: ; in: ; ; ; Will By inputting the Critic network and performing forward computation, we can obtain... The estimated value Similarly, we can obtain The estimated value ; With Actor Network Network parameters at time and and the augmented state of the input. The relevant information can be calculated using the chain rule.
[0043] As one possible approach, updating the Critic network parameters based on stochastic gradient descent improves the objective function. Minimizing this yields the learning law of the Critic network: ; ; Therefore, the learning law of Critic networks can be derived. and .
[0044] As another possible approach, the Actor network can be updated by minimizing the value function: ; ; Therefore, the learning law of Actor networks can be derived. and .
[0045] Figure 4 This is a schematic diagram of the weight change curve of an Actor network according to a specific embodiment of the present invention, as shown below. Figure 4 As shown, in the initial stage (i.e. when the time is short), some weights fluctuate significantly, and then gradually stabilize and remain in a relatively fixed range. During the learning process, the weight parameters from the hidden layer to the output layer of the Actor network gradually converge to a stable value from the stage of dynamically adjusting to fit the optimal control law. The later weights tend to stabilize, which means that the network's learning of the vehicle control strategy tends to converge and can provide stable decision support for vehicle control.
[0046] In step S103, a nonlinear state equation and an observation equation are established based on a preset extended Kalman filter method, and the nonlinear state equation and the observation equation are discretized respectively. The parameter identification result is obtained based on the discretized nonlinear state equation and the discretized observation equation. The vehicle pre-aiming control command is obtained based on the pre-constructed Actor-Critic network and the parameter identification result, and the vehicle is pre-aiming controlled based on the control command.
[0047] The pre-defined extended Kalman filter method refers to the method of Kalman filtering in nonlinear scenarios, which uses linearization to approximate nonlinear systems, thereby achieving state estimation and parameter identification of nonlinear systems.
[0048] Specifically, this embodiment of the invention constructs a nonlinear state equation describing the changes in vehicle state and an observation equation relating to sensor measurements based on a preset extended Kalman filter method. These two equations are discretized to adapt to discrete sampling characteristics. Using the discretized nonlinear state equation and observation equation, the optimal estimate of unknown vehicle parameters, i.e., the parameter identification result, is obtained through an iterative process of extended Kalman filtering. The parameter identification result is input into a pre-constructed Actor-Critic network, and the network's decision and evaluation mechanism is combined to generate a vehicle anti-aiming control command. Finally, anti-aiming control is implemented on the vehicle according to the control command. It should be noted that, considering the changes in system dynamic parameters caused by changes in the controlled object, the system key parameter identification method based on extended Kalman filtering in this invention only needs to measure the length of the new controlled object to sense changes in system key parameters through online data. For example, by augmenting the full dynamic system, we can obtain the nonlinear state equations as follows: ; The system is divided into sampling periods. Discretization From this, we can obtain the nonlinear state equation and the observation equation: ; ; Furthermore, the extended Kalman filter algorithm is used to... and Make corrections and... The identification process is performed, and the parameters obtained are: .
[0049] It should be noted that when performing covariance matrix calculations... During the update, The derivation method is as follows: ; ; ; ; ; ; ; ; Furthermore, in some embodiments, vehicle aiming control commands are obtained based on a pre-constructed Actor-Critic network and parameter identification results, and vehicle aiming control is performed based on the control commands, including: obtaining a first dynamic term in the learning law of the Actor network and a second dynamic term in the learning law of the Critic network based on the extended Kalman filter method; obtaining vehicle aiming control commands based on the first dynamic term, the second dynamic term, and parameter identification results, and performing vehicle aiming control based on the control commands.
[0050] Among them, the first dynamic term refers to the term in the Actor network learning law that reflects the coupling relationship between vehicle dynamic characteristics and control strategy adjustment, and the second dynamic term refers to the term in the Critic network learning law that reflects the correlation between vehicle dynamic characteristics and value assessment and vehicle state evolution and control effect quality inspection.
[0051] As one possible approach, based on the extended Kalman filter method, the first dynamic term in the learning law of the Actor network is obtained: ; Simultaneously, obtain the second dynamic term in the learning law of the Critic network: ; in, ; ; ; ; Therefore, efficient control strategy learning is performed based on online data. The network parameters of the Actor network and Critic network are updated online based on the learning rate. Finally, the Actor network outputs control commands to achieve vehicle aiming control.
[0052] Furthermore, in some embodiments, before establishing the nonlinear state equation and observation equation based on the preset extended Kalman filter method, the method further includes: establishing the full dynamic equation of the controlled object based on the vehicle driving state parameters, and determining the scaling factor based on the full dynamic equation; and obtaining the full dynamic equation of a new controlled object based on the scaling factor and the full dynamic equation of the controlled object.
[0053] For example, the total dynamic equation of the original controlled object is: ; in, Let be the lateral speed of the vehicle. The yaw rate of the vehicle; make After the controlled object is replaced, the dynamic matrix parameters change. Assume the actual measurable front-to-rear axis distance of the new controlled object is... ; using the scaling factor Describe the system parameters of the new controlled object. The change is determined, and the unknown distance from the center of mass to the front axle is set as... The full dynamic equation of the new controlled object is established based on the comparison relationship in Table 1, wherein Table 1 is a system parameter comparison table provided according to a specific embodiment of the present invention.
[0054] Table 1
[0055] The total dynamic equation of the new controlled object is then: ; Therefore, only identification is required. It can then be sensitive to all changes in the dynamic parameters of the new controlled object.
[0056] To enable those skilled in the art to better understand the vehicle anti-aiming control method of the present invention, the following explanation will be provided in conjunction with specific embodiments.
[0057] Figure 5 This is a schematic diagram of a self-learning control framework provided according to a specific embodiment of the present invention, such as... Figure 5 As shown, the self-learning control framework includes a vehicle dynamics system and the continuous / discrete states output by the vehicle dynamics system. Maintain and transmit the current state. State at historical moments The system includes a zero-order hold (ZOH) that provides state input for subsequent modules, an Actor network, a Critic network, a time difference error, and an extended Kalman filter (EKF) identification algorithm. The Actor-Critic network enables interactive learning between control strategy optimization (Actor) and value evaluation feedback (Critic). Simultaneously, the extended Kalman filter identifies vehicle parameters online and corrects deviations in the dynamics model. Ultimately, the vehicle generates anticipatory control commands based on the learned strategy and model, enabling early response and accurate tracking of future road conditions.
[0058] Figure 6 This is a comparative curve diagram of a vehicle anti-aiming control method according to a specific embodiment of the present invention, such as... Figure 6 As shown, the road curvature change curve exhibits periodic square wave fluctuations, reflecting the time-varying characteristics of road curvature, which is sometimes straight and sometimes curved, and the degree of curvature varies positively and negatively. In the lateral offset error curve, the dashed line represents the error curve of the pre-aiming control method, while the solid line represents the vehicle pre-aiming control method combining Actor-Critic and Extended Kalman filtering in this embodiment of the invention. In the initial stage (around 0 seconds), the lateral offset error of the pre-aiming control method fluctuates significantly, and the error change of the vehicle pre-aiming control method in this embodiment of the invention is obvious, reflecting the online optimization process of the control strategy. In the subsequent stage (after 20 seconds), the lateral offset error of the vehicle pre-aiming control method in this embodiment of the invention is basically stable near 0, with minimal fluctuations. The error of the pre-aiming control method still exhibits significant oscillations and poor stability. Therefore, the vehicle pre-aiming control method in this embodiment of the invention can more effectively suppress the vehicle lateral offset error and improve the accuracy and stability of trajectory tracking.
[0059] further, Figure 7 This is a schematic diagram illustrating the online identification results of the scaling factor and the distance from the centroid of the new controlled object to the front axis, according to a specific embodiment of the present invention. Figure 7 As shown, in the initial stage, the identified values of each parameter fluctuate drastically due to the dynamic adjustment of the system; as time goes by, the identification results gradually converge and stabilize at a specific value, indicating that the embodiments of the present invention can gradually and accurately estimate vehicle parameters in a dynamic process.
[0060] The vehicle preview control method provided by this invention obtains vehicle driving state parameters, establishes a vehicle dynamics model, and obtains a discretized model through Euler discretization. An augmented dynamics model is constructed based on the parameters, and its optimal tracking control law is obtained. Then, an Actor-Critic network is used to approximate the control law online to obtain optimized vehicle control parameters. Finally, based on extended Kalman filtering, nonlinear state equations and observation equations are established and discretized to obtain parameter identification results. Combined with the Actor-Critic network, vehicle preview control commands are generated to achieve vehicle preview control. This solves the problems of poor applicability and insufficient control accuracy of path tracking control methods for autonomous vehicles in related technologies, and improves the control accuracy and applicability of the vehicle preview control method.
[0061] Next, the vehicle aiming control device according to an embodiment of the present invention will be described with reference to the accompanying drawings.
[0062] Figure 8 This is a block diagram of a vehicle anti-aiming control device provided according to an embodiment of the present invention.
[0063] like Figure 8 As shown, the vehicle aiming control device 10 includes: a mathematical modeling module 100, an optimization module 200, and a control module 300.
[0064] The mathematical modeling module 100 is used to acquire vehicle driving state parameters, establish a vehicle dynamics model based on the vehicle driving state parameters, discretize the vehicle dynamics model using a preset Euler discretization method to obtain a discretized vehicle dynamics model, and obtain an augmented dynamics model based on the vehicle driving state parameters and the discretized vehicle dynamics model, and obtain the optimal tracking control law based on the augmented dynamics model; the optimization module 200 is used to approximate the optimal tracking control law online according to a pre-constructed Actor-Critic network to obtain optimized vehicle control parameters; the control module 300 is used to establish nonlinear state equations and observation equations based on a preset extended Kalman filter method, discretize the nonlinear state equations and observation equations respectively, obtain parameter identification results based on the discretized nonlinear state equations and discretized observation equations, obtain vehicle pre-aiming control commands based on the pre-constructed Actor-Critic network and parameter identification results, and perform pre-aiming control on the vehicle based on the control commands.
[0065] Furthermore, in some embodiments, the mathematical modeling module 100 is specifically used for: acquiring the vehicle's lateral offset and heading angle error; establishing a vehicle dynamics model based on the vehicle's lateral offset and heading angle error; discretizing the vehicle dynamics model according to a preset control cycle based on a preset Euler discretization method to obtain a compact discrete form of the vehicle dynamics model; obtaining an augmented dynamics model based on the current road curvature and the road curvature at the next moment, wherein the augmented dynamics model includes a utility function and a value function; and obtaining the optimal tracking control law of the augmented dynamics model based on the value function.
[0066] Further, in some embodiments, the optimization module 200 is specifically used for: obtaining the current augmented state parameters, historical augmented state parameters, and control input parameters of the augmented dynamics model; updating the time difference error of the current augmented state parameters, historical augmented state parameters, and control input parameters based on the Critic network in the pre-built Actor-Critic network to obtain the learning law of the Critic network; obtaining the learning rate of the weights of each layer of the Critic network; and updating the Actor network parameters in the pre-built Actor-Critic network by minimizing the value function of the augmented dynamics model based on the learning rate of the weights of each layer of the Critic network, the current augmented state parameters, historical augmented state parameters, and control input parameters to obtain the learning law of the Actor network; and obtaining the optimized vehicle control parameters according to the learning law of the Critic network and the learning law of the Actor network.
[0067] Furthermore, in some embodiments, the control module 300 is specifically used to: obtain the first dynamic term in the learning law of the Actor network and the second dynamic term in the learning law of the Critic network based on the extended Kalman filter method; obtain vehicle pre-aiming control commands based on the first dynamic term, the second dynamic term and the parameter identification results, and perform pre-aiming control on the vehicle based on the control commands.
[0068] Furthermore, in some embodiments, before establishing the nonlinear state equation and observation equation based on the preset extended Kalman filter method, the control module 300 is also used to: establish the full dynamic equation of the controlled object based on the vehicle driving state parameters, and determine the scaling factor according to the full dynamic equation; based on the scaling factor, obtain the full dynamic equation of the new controlled object according to the full dynamic equation of the controlled object.
[0069] It should be noted that the above explanation of the vehicle anti-aiming control method embodiment also applies to the vehicle anti-aiming control device of this embodiment, and will not be repeated here.
[0070] According to the vehicle pre-aiming control device provided in the embodiments of the present invention, vehicle driving state parameters are acquired, a vehicle dynamics model is established and a discretized model is obtained through Euler discretization, an augmented dynamics model is constructed in combination with the parameters and its optimal tracking control law is obtained; then, the control law is approximated online using an Actor-Critic network to obtain optimized vehicle control parameters; finally, nonlinear state equations and observation equations are established and discretized based on extended Kalman filtering to obtain parameter identification results, and vehicle pre-aiming control commands are generated in combination with the Actor-Critic network to realize vehicle pre-aiming control. This solves the problems of poor applicability and insufficient control accuracy of path tracking control methods for autonomous vehicles in related technologies, and improves the control accuracy and applicability of vehicle pre-aiming control methods.
[0071] Figure 9 This is a schematic diagram of an electronic device provided according to an embodiment of the present invention. The electronic device may include: The memory 901, the processor 902, and the computer program stored on the memory 901 and capable of running on the processor 902.
[0072] When processor 902 executes the program, it implements the vehicle aiming control method provided in the above embodiments.
[0073] Furthermore, the electronic device also includes: Communication interface 903 is used for communication between memory 901 and processor 902.
[0074] The memory 901 is used to store computer programs that can run on the processor 902.
[0075] The memory 901 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.
[0076] If the memory 901, processor 902, and communication interface 903 are implemented independently, then the communication interface 903, memory 901, and processor 902 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 9 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.
[0077] Optionally, in a specific implementation, if the memory 901, processor 902, and communication interface 903 are integrated on a single chip, then the memory 901, processor 902, and communication interface 903 can communicate with each other through an internal interface.
[0078] The processor 902 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.
[0079] In addition, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the vehicle aiming control method described above.
[0080] In addition, embodiments of the present invention also provide a computer program product, including a computer program, which is executed to implement the vehicle anti-aiming control method described above.
[0081] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.
[0082] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.
[0083] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more N executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of the invention pertain.
[0084] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any of the following techniques known in the art, or a combination thereof: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.
[0085] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.
Claims
1. A vehicle anti-aiming control method, characterized in that, Includes the following steps: The vehicle driving state parameters are obtained, and a vehicle dynamics model is established based on the vehicle driving state parameters. The vehicle dynamics model is discretized using a preset Euler discretization method to obtain a discretized vehicle dynamics model. An augmented dynamics model is obtained based on the vehicle driving state parameters and the discretized vehicle dynamics model. The optimal tracking control law of the augmented dynamics model is obtained based on the augmented dynamics model. Based on the pre-constructed Actor-Critic network, the optimal tracking control law is approximated online to obtain the optimized vehicle control parameters; A nonlinear state equation and an observation equation are established based on a pre-defined extended Kalman filter method. These equations are then discretized. Parameter identification results are obtained based on the discretized equations. Vehicle aiming control commands are then derived using the pre-constructed Actor-Critic network and the parameter identification results. Finally, aiming control of the vehicle is performed based on these commands. The step of obtaining vehicle pre-aiming control commands based on the pre-constructed Actor-Critic network and the parameter identification results, and performing pre-aiming control on the vehicle based on the control commands, includes: obtaining a first dynamic term in the learning law of the Actor network and a second dynamic term in the learning law of the Critic network based on the extended Kalman filter method; obtaining vehicle pre-aiming control commands based on the first dynamic term, the second dynamic term, and the parameter identification results; and performing pre-aiming control on the vehicle based on the control commands. Specifically, based on the extended Kalman filter method, the first dynamic term in the learning law of the Actor network is obtained: Based on the extended Kalman filter method, the second dynamic term in the learning law of the Critic network is obtained: in, For the augmented dynamics model, For the augmented state parameters at historical moments, For control input parameters at historical moments, For aiming length, for A zero matrix of dimension 1 , , , Let be the block matrix of the augmented dynamics model. The first dynamics term is a term in the Actor network learning law that reflects the coupling relationship between vehicle dynamics characteristics and control strategy adjustment. The second dynamics term is a term in the Critic network learning law that reflects the correlation between vehicle dynamics characteristics and value assessment and vehicle state evolution and control effect.
2. The method according to claim 1, characterized in that, The process of acquiring vehicle driving state parameters, establishing a vehicle dynamics model based on the vehicle driving state parameters, discretizing the vehicle dynamics model using a preset Euler discretization method to obtain a discretized vehicle dynamics model, obtaining an augmented dynamics model based on the vehicle driving state parameters and the discretized vehicle dynamics model, and obtaining the optimal tracking control law based on the augmented dynamics model includes: Obtain the vehicle's lateral offset and heading angle error, and establish the vehicle dynamics model based on the vehicle's lateral offset and heading angle error; Based on the preset Euler discretization method, the vehicle dynamics model is discretized according to the preset control period to obtain a compact discrete form of the vehicle dynamics model. Based on the current road curvature and the road curvature at the next moment, an augmented dynamics model is obtained according to the compact discrete form of the vehicle dynamics model, wherein the augmented dynamics model includes a utility function and a value function; The optimal tracking control law of the augmented dynamics model is obtained based on the value function.
3. The method according to claim 1, characterized in that, The process of approximating the optimal tracking control law online based on a pre-constructed Actor-Critic network to obtain optimized vehicle control parameters includes: The current augmented state parameters, historical augmented state parameters, and control input parameters of the augmented dynamics model are obtained. Based on the Critic network in the pre-constructed Actor-Critic network, the time difference error of the current augmented state parameters, the historical augmented state parameters, and the control input parameters is updated to obtain the learning law of the Critic network. The learning rate of each layer weight of the Critic network is obtained, and based on the learning rate of each layer weight of the Critic network, the current augmented state parameter, the historical augmented state parameter and the control input parameter, the Actor network parameters in the pre-built Actor-Critic network are updated by minimizing the value function of the augmented dynamics model to obtain the learning law of the Actor network. The optimized vehicle control parameters are obtained based on the learning laws of the Critic network and the Actor network.
4. The method according to claim 1, characterized in that, Before establishing the nonlinear state equation and observation equation based on the preset extended Kalman filter method, the following steps are also included: The full dynamic equation of the controlled object is established based on the vehicle driving state parameters, and the scaling factor is determined according to the full dynamic equation. Based on the scaling factor, the full dynamic equation of the new controlled object is obtained according to the full dynamic equation of the controlled object.
5. A vehicle anti-aiming control device, characterized in that, The device includes: The mathematical modeling module is used to acquire vehicle driving state parameters, establish a vehicle dynamics model based on the vehicle driving state parameters, discretize the vehicle dynamics model using a preset Euler discretization method to obtain a discretized vehicle dynamics model, and obtain an augmented dynamics model based on the vehicle driving state parameters and the discretized vehicle dynamics model, and obtain the optimal tracking control law of the augmented dynamics model based on the augmented dynamics model. The optimization module is used to approximate the optimal tracking control law online based on the pre-built Actor-Critic network to obtain the optimized vehicle control parameters; The control module is used to establish nonlinear state equations and observation equations based on a preset extended Kalman filter method, discretize the nonlinear state equations and observation equations respectively, obtain parameter identification results based on the discretized nonlinear state equations and observation equations, obtain vehicle pre-aiming control commands based on the pre-constructed Actor-Critic network and the parameter identification results, and perform pre-aiming control on the vehicle based on the control commands. The step of obtaining vehicle pre-aiming control commands based on the pre-constructed Actor-Critic network and the parameter identification results, and performing pre-aiming control on the vehicle based on the control commands, includes: obtaining a first dynamic term in the learning law of the Actor network and a second dynamic term in the learning law of the Critic network based on the extended Kalman filter method; obtaining vehicle pre-aiming control commands based on the first dynamic term, the second dynamic term, and the parameter identification results; and performing pre-aiming control on the vehicle based on the control commands. Specifically, based on the extended Kalman filter method, the first dynamic term in the learning law of the Actor network is obtained: Based on the extended Kalman filter method, the second dynamic term in the learning law of the Critic network is obtained: in, For the augmented dynamics model, For the augmented state parameters at historical moments, For control input parameters at historical moments, For aiming length, for A zero matrix of dimension 1 , , , Let be the block matrix of the augmented dynamics model. The first dynamics term is a term in the Actor network learning law that reflects the coupling relationship between vehicle dynamics characteristics and control strategy adjustment. The second dynamics term is a term in the Critic network learning law that reflects the correlation between vehicle dynamics characteristics and value assessment and vehicle state evolution and control effect.
6. The apparatus according to claim 5, characterized in that, The mathematical modeling module is specifically used for: Obtain the vehicle's lateral offset and heading angle error, and establish the vehicle dynamics model based on the vehicle's lateral offset and heading angle error; Based on the preset Euler discretization method, the vehicle dynamics model is discretized according to the preset control period to obtain a compact discrete form of the vehicle dynamics model. Based on the current road curvature and the road curvature at the next moment, an augmented dynamics model is obtained according to the compact discrete form of the vehicle dynamics model, wherein the augmented dynamics model includes a utility function and a value function; The optimal tracking control law of the augmented dynamics model is obtained based on the value function.
7. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the vehicle anti-aiming control method as described in any one of claims 1-4.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the vehicle aiming control method as described in any one of claims 1-4.
9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the vehicle aiming control method as described in any one of claims 1-4.
Citation Information
Patent Citations
Path tracking method and device, electronic equipment and storage medium
CN115179935A
Intelligent network connection vehicle distributed optimization control method based on reinforcement learning strategy
CN119087808A
