A model-free robust optimal control method, device, and medium for connected vehicles

By constructing an affine nonlinear hybrid traffic system model and the H-infinity robust optimal control problem, and combining evaluation neural networks and the Bellman optimality principle, robust optimal control commands are generated. This solves the robustness, optimality, and real-time issues of connected autonomous vehicles in hybrid traffic environments, and improves the control performance.

CN123104701APending Publication Date: 2026-11-17SHANDONG UNIV OF SCI & TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611602047.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-10-16
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

In mixed traffic environments, existing technologies for the control of connected autonomous vehicles struggle to achieve robustness, optimality, and real-time performance without relying on accurate system dynamics models and initial stable control strategies. In particular, the control performance is poor when faced with the randomness of manually driven vehicles and external disturbances.

Method used

By constructing an affine nonlinear hybrid transportation system model, and taking the longitudinal acceleration of the connected autonomous vehicle and the unknown external disturbance as two action quantities in a zero-sum game, an H-infinite robust optimal control problem is generated. The problem is then parameterized using an evaluation neural network, and robust optimal control commands are generated by combining the Bellman optimality principle and integral Q-learning regression relationship.

Benefits of technology

It achieves improved stability and safety in mixed traffic flow, suppresses unknown external interference, alleviates speed oscillations and traffic wave phenomena, and ensures the car-following safety and real-time control of connected autonomous vehicles in complex environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123104701A_ABST
    Figure CN123104701A_ABST
Patent Text Reader

Abstract

This application discloses a model-free robust optimal control method, device, and medium for connected vehicles, relating to the field of model-free robust optimal control technology for connected vehicles. The method includes: constructing single-vehicle error states and global error states based on vehicle speed and spacing errors in a mixed traffic scenario, and establishing an affine nonlinear mixed traffic system model; generating an H-infinite robust optimal control problem by using the control input of the connected autonomous vehicle and unknown external disturbances as two agents in a zero-sum game; constructing a Q-function and parameterizing it to transform it into a learning problem for evaluating neural network weights; establishing an integral Q-learning regression relationship based on the Bellman optimality principle, and constructing auxiliary filter variables and auxiliary error feedback terms to generate convergent evaluation neural network weights; calculating the robust optimal control input of the connected autonomous vehicle to generate the robust optimal control command for the connected autonomous vehicle.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of intelligent transportation and autonomous driving control technology, and in particular to a model-free robust optimal control method, device and medium for connected vehicles. Background Technology

[0002] In recent years, with the development of intelligent transportation and connected autonomous driving technologies, connected autonomous vehicles have received widespread attention for improving road traffic efficiency, alleviating traffic congestion, and enhancing traffic safety. Since manually driven vehicles will remain prevalent for a considerable period, mixed traffic flows comprised of both manually driven and connected autonomous vehicles will become an important form of road traffic system. In mixed traffic environments, the following behavior of manually driven vehicles is random and uncertain, and interactions between vehicles can easily cause speed fluctuations, traffic congestion, and stop-and-go traffic waves. Simultaneously, factors such as driver distraction, uneven road surfaces, pedal operation errors, and sudden acceleration or deceleration can create unknown external disturbances, further affecting the stability and robustness of the mixed traffic system.

[0003] In existing technologies, model-linearized control methods typically linearize the car-following model of manually driven vehicles and the dynamic model of connected autonomous vehicles near the equilibrium point, and then design state feedback, cruise control, or robust controllers. While the modeling process is clear, the linearized approximation cannot accurately describe the nonlinear dynamics of the mixed traffic system, and the control effect is highly sensitive to model errors and external disturbances. Cooperative control methods based on specific traffic structures typically construct control strategies by dividing sub-vehicle groups, cooperating with connected autonomous vehicles, or using safety constraints to improve the stability of mixed traffic flows or the safety of car-following. These methods are highly scenario-specific, but rely on pre-defined traffic structures, vehicle interaction relationships, or constraints, and have limited adaptability to unknown human-driven vehicle behavior and complex disturbance environments. Reinforcement learning-based control methods learn control strategies for connected autonomous vehicles through data, reducing reliance on accurate models and making them suitable for handling control problems in complex traffic environments. However, existing methods often require a large amount of interactive data, and some methods still rely on initial stable control strategies, execution-evaluation network structures, or complete system dynamic information, resulting in high online computational complexity and difficulty in simultaneously achieving robustness, optimality, and real-time performance.

[0004] Therefore, how to achieve online control of connected autonomous vehicles without relying on accurate system dynamics models and initial stable control strategies, thereby improving the robustness, optimality, and real-time performance of hybrid traffic flow control, has become an urgent technical problem to be solved. Summary of the Invention

[0005] This application provides a model-free, robust, and optimal control method, device, and medium for connected vehicles to address the following technical problem: how to achieve online control of connected autonomous vehicles without relying on an accurate system dynamics model and an initial stable control strategy, thereby improving the robustness, optimality, and real-time performance of hybrid traffic flow control.

[0006] In a first aspect, embodiments of this application provide a model-free robust optimal control method for connected vehicles. The method includes: constructing a single-vehicle error state and a global error state based on the speed error between vehicle speed and desired speed and the spacing error between vehicle spacing and desired spacing in a mixed traffic scenario; inputting the longitudinal acceleration of the connected autonomous vehicle and unknown external disturbances into a preset optimal speed model to establish an affine nonlinear mixed traffic system model; generating an H-infinite robust optimal control problem based on the affine nonlinear mixed traffic dynamic model, using the control input of the connected autonomous vehicle and unknown external disturbances as two action quantities in a zero-sum game; constructing a Q-function based on the H-infinite robust optimal control problem, and using an evaluation neural network to parameterize the Q-function, transforming the infinite robust optimal control problem into a learning problem of evaluation neural network weights; establishing an integral Q-learning regression relationship based on the Bellman optimality principle, and constructing auxiliary filter variables and auxiliary error feedback terms to generate convergent evaluation neural network weights; calculating the robust optimal control input of the connected autonomous vehicle based on the evaluation neural network weights, and outputting it to the longitudinal control end of the connected autonomous vehicle to generate the robust optimal control command of the connected autonomous vehicle.

[0007] In one implementation of this application, based on the speed error between the vehicle speed and the desired speed and the distance error between the vehicle spacing and the desired distance in a mixed traffic scenario, a single-vehicle error state and a global error state are constructed. Specifically, this includes: collecting the position and speed of the target vehicle and the vehicle in front of it in the mixed traffic scenario, and determining the vehicle spacing of the target vehicle based on the difference between the position of the target vehicle and the position of the vehicle in front; subtracting the vehicle spacing from the desired distance to obtain the spacing error of the target vehicle, and subtracting the target vehicle speed from the desired speed to obtain the speed error of the target vehicle; using the spacing error as a first state component and the speed error as a second state component to form the single-vehicle error state of the target vehicle; and combining the single-vehicle error states of each vehicle in the mixed traffic scenario in the order of vehicles from front to back to generate the global error state.

[0008] In one implementation of this application, the longitudinal acceleration of the connected autonomous vehicle and unknown external disturbances are input into a preset optimal speed model to establish an affine nonlinear hybrid traffic system model. Specifically, the affine nonlinear hybrid traffic system model is represented by the following formula:

[0009] in, The rate of change of the state vector over time. This is the global error state vector. Let be the system drift state function, describing the nonlinear evolution of the system state itself in the absence of control input and external disturbances. These are control input channel functions or matrices used to describe how the control inputs of a connected autonomous vehicle affect changes in the system state. For control input, namely the longitudinal acceleration command of the connected autonomous vehicle, The disturbance input channel function or matrix is ​​used to describe how an unknown external disturbance affects the system state change. Unknown external disturbances include random following behavior of manually driven vehicles, driver operation errors, and external traffic influences.

[0010] In one implementation of this application, based on an affine nonlinear hybrid traffic dynamics model, the control input of the connected autonomous vehicle and the unknown external disturbance are used as two agents in a zero-sum game to generate an H-infinite robust optimal control problem. Specifically, this includes: defining an instantaneous reward function containing a state error term, a control input term, and a disturbance attenuation term based on the global error state, control input, and unknown external disturbance; and integrating the instantaneous reward function over the time interval to construct a performance index function, expressed by the following formula:

[0011]

[0012] in, For performance index functions, For integration time variable, For the current moment, For instantaneous reward function, This is a penalty term for state error. This is the transpose of the state vector. Here, is the state weight matrix, a symmetric positive definite matrix used to weigh the importance of each state error component. This is the global error state vector. To control input penalty items, To control the transpose of the input vector. To control the weight matrix, This is the control input vector for connected autonomous vehicles. For disturbance penalty items, The level of disturbance attenuation. This is the transpose of the perturbation vector. Let the unknown external disturbance vector be used; the control input is used as the action that minimizes the performance index function, and the unknown external disturbance is used as the action that maximizes the performance index function to construct a zero-sum game relationship; based on the zero-sum game relationship, the H-infinite robust optimal control problem that minimizes the performance index function in the worst case of unknown external disturbance is generated.

[0013] In one implementation of this application, a Q function is constructed based on the H-infinite robust optimal control problem, and an evaluation neural network is used to parameterize the Q function, thereby transforming the infinite robust optimal control problem into a learning problem of the evaluation neural network weights. Specifically, this includes: adding the optimal value function and Hamiltonian function of the H-infinite robust optimal control problem to construct the Q function, and using an evaluation neural network to parameterize the Q function. The Q function is represented by a combination of the ideal weights and activation function of the evaluation neural network, as shown by the following formula:

[0014] in, The Q function in parameterized representation, To evaluate the ideal weight vector of a neural network, This is the transpose of the weight vector. For activation function, The approximation error of the neural network is used to map the global error state, the control input of the connected autonomous vehicle, and the unknown external disturbances into a unified learning feature through the activation function. Based on the parameterized Q function, the solution of the optimal value function of the H-infinite robust optimal control problem is transformed into the learning problem of evaluating the weights of the neural network, so that the learning of the evaluation neural network weights is equivalent to solving the H-infinite robust optimal control problem.

[0015] In one implementation of this application, based on the Bellman optimality principle, an integral Q-learning regression relationship is established, and auxiliary filter variables and auxiliary error feedback terms are constructed to generate convergent evaluation neural network weights. Specifically, this includes: within the sampling period, based on the Bellman optimality principle, the state-value function before the current time step is expressed as the sum of the integral of the state-value function at the current time step and the instantaneous reward function within the sampling period; based on the integral relationship between the state-value function and the instantaneous reward function, an integral reinforcement term is defined, and combined with the parameterized Q-function, an integral Q-learning regression relationship for evaluating neural network weight learning is established; based on the difference in activation functions at adjacent sampling times, a first auxiliary filter variable is constructed, and a second auxiliary filter variable is constructed based on the integral reinforcement term; based on the first auxiliary filter variable, the second auxiliary filter variable, and the estimated evaluation neural network weights, an auxiliary error feedback term is constructed; a finite excitation condition is introduced, and when the finite excitation condition is satisfied, the filter data corresponding to the time step that maximizes the minimum eigenvalue of the filter variable matrix is ​​selected, and an online adaptive law for evaluating neural network weights is designed based on the auxiliary error feedback term and the filter data to update the evaluation neural network weights online until convergence.

[0016] In one implementation of this application, the robust optimal control input for a connected autonomous vehicle is calculated based on the weights of the evaluation neural network. Specifically, this includes: extracting the weight estimates of the evaluation neural network weights after the online adaptive law converges; and based on the converged weight estimates, calculating the partial derivatives of the parameterized Q-function with respect to the control input and the unknown external disturbance, respectively; setting the partial derivatives of the Q-function with respect to the control input and the unknown external disturbance to zero, and solving the equations simultaneously to obtain the estimated control input and the estimated unknown external disturbance; and using the estimated control input as the robust optimal control input for the connected autonomous vehicle.

[0017] In one implementation of this application, the input is output to the longitudinal control terminal of the connected autonomous vehicle to generate robust optimal control commands for the connected autonomous vehicle. Specifically, this includes: converting the calculated robust optimal control input of the connected autonomous vehicle into longitudinal acceleration commands and sending the longitudinal acceleration commands to the longitudinal control terminal of the connected autonomous vehicle; the longitudinal control terminal generates drive actuator control signals and brake actuator control signals based on the longitudinal acceleration commands; and using the drive actuator control signals and brake actuator control signals to drive the vehicle's longitudinal actuators, adjusting the speed and spacing of the connected autonomous vehicle to suppress unknown external disturbances and stabilize mixed traffic flow.

[0018] Secondly, embodiments of this application also provide a model-free robust optimal control device for connected vehicles. The device includes: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores instructions executable by the at least one processor, which, when executed, enable the at least one processor to: construct a single-vehicle error state and a global error state based on the speed error between vehicle speed and desired speed and the spacing error between vehicle spacing and desired spacing in a mixed traffic scenario; and input the longitudinal acceleration of the connected autonomous vehicle and unknown external disturbances into a preset optimal speed model to establish an affine nonlinear mixed traffic system model; and based on the affine nonlinear mixed traffic dynamic model, to... The control input and unknown external disturbances of a connected autonomous vehicle are treated as two agents in a zero-sum game, generating an H-infinite robust optimal control problem. Based on this problem, a Q-function is constructed, and an evaluation neural network is used to parameterize it, transforming the infinite robust optimal control problem into a learning problem for the evaluation neural network weights. Based on the Bellman optimality principle, an integral Q-learning regression relationship is established, and auxiliary filter variables and auxiliary error feedback terms are constructed to generate convergent evaluation neural network weights. Based on these weights, the robust optimal control input for the connected autonomous vehicle is calculated and output to the longitudinal control terminal to generate the robust optimal control command for the vehicle.

[0019] Thirdly, this application also provides a non-volatile computer storage medium for model-free robust optimal control of connected vehicles, storing computer-executable instructions. These instructions are configured to: construct a single-vehicle error state and a global error state based on the speed error between vehicle speed and desired speed and the spacing error between vehicle spacing and desired spacing in a mixed traffic scenario; and input the longitudinal acceleration of the connected autonomous vehicle and unknown external disturbances into a preset optimal speed model to establish an affine nonlinear mixed traffic system model; based on the affine nonlinear mixed traffic dynamic model, using the control input of the connected autonomous vehicle and unknown external disturbances as zero... The two action quantities in the game are used to generate the H-infinite robust optimal control problem. Based on the H-infinite robust optimal control problem, a Q-function is constructed, and an evaluation neural network is used to parameterize the Q-function, transforming the infinite robust optimal control problem into a learning problem of the evaluation neural network weights. Based on the Bellman optimality principle, an integral Q-learning regression relationship is established, and auxiliary filter variables and auxiliary error feedback terms are constructed to generate convergent evaluation neural network weights. Based on the evaluation neural network weights, the robust optimal control input of the connected autonomous vehicle is calculated and output to the longitudinal control end of the connected autonomous vehicle to generate the robust optimal control command of the connected autonomous vehicle.

[0020] This application provides a model-free robust optimal control method, device, and medium for connected vehicles, which has the following advantages: By introducing the longitudinal acceleration of the connected autonomous vehicle and unknown external disturbances, an affine nonlinear hybrid traffic system model is established. This eliminates the need for linear approximations of the hybrid traffic system at equilibrium points, and can fully characterize the randomness of human-driven vehicle behavior, driver error, and the nonlinear dynamics of the system caused by external traffic disturbances. Furthermore, by treating the control input of the connected autonomous vehicle and unknown external disturbances as the interaction quantities of two parties in a zero-sum game, a robust optimal control problem with H infinity is constructed, effectively addressing fluctuations in human-driven behavior and various external disturbances. The system incorporates game theory into its processing, pursuing optimal performance in vehicle car-following control while suppressing unknown disturbances, improving the overall stability of mixed traffic flow, mitigating speed oscillations and stop-and-go traffic waves, and ensuring the car-following safety of connected autonomous vehicles in mixed traffic environments with unknown disturbances. It utilizes the Q-function to evaluate the parameterization transformation problem of the neural network, eliminating the need for pre-setting an initial stable control strategy. Integral Q-learning, auxiliary filter variables, and auxiliary error feedback terms ensure online convergence of weights, improving the controller's real-time performance. It outputs robust optimal control commands end-to-end, achieving stable management of mixed traffic flow with strong engineering feasibility. Attached Figure Description

[0021] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A flowchart of a model-free robust optimal control method for connected vehicles provided in this application embodiment; Figure 2 This is a schematic diagram of the internal structure of a model-free robust optimal control device for connected vehicles, provided as an embodiment of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] This application provides a model-free, robust, and optimal control method, device, and medium for connected vehicles to address the following technical problem: how to achieve online control of connected autonomous vehicles without relying on an accurate system dynamics model and an initial stable control strategy, thereby improving the robustness, optimality, and real-time performance of hybrid traffic flow control.

[0024] The technical solutions proposed in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0025] Figure 1 A flowchart illustrating a model-free robust optimal control method for connected vehicles provided in this application embodiment. Figure 1 As shown in the embodiment of this application, a model-free robust optimal control method for connected vehicles includes the following steps: Step 10: Based on the speed error between vehicle speed and desired speed and the spacing error between vehicle spacing and desired spacing in the mixed traffic scenario, construct the single-vehicle error state and global error state, and input the longitudinal acceleration of the connected autonomous vehicle and unknown external disturbances into the preset optimal speed model to establish an affine nonlinear mixed traffic system model.

[0026] As an optional embodiment, based on the speed error between the vehicle speed and the desired speed and the spacing error between the vehicle spacing and the desired spacing in the mixed traffic scenario, a single-vehicle error state and a global error state are constructed. The longitudinal acceleration of the connected autonomous vehicle and unknown external disturbances are input into the preset optimal speed model to establish an affine nonlinear mixed traffic system model. Specifically, it may include: Step 101: Collect the position and speed of the target vehicle and the vehicle in front of it in the mixed traffic scenario, and determine the vehicle spacing of the target vehicle based on the difference between the position of the target vehicle and the position of the vehicle in front.

[0027] In this step, in a mixed traffic scenario, it is necessary to collect the position and speed information of the target vehicle and the vehicle in front of it. The aforementioned motion state data can be obtained through vehicle perception and network information interaction. The difference between the collected target vehicle position and the position of the vehicle in front is processed to determine the vehicle distance of the target vehicle. The acquired vehicle speed and the calculated vehicle distance are the basis for subsequent state error calculation. Based on the actual vehicle speed and the preset expected speed, the speed error can be obtained. Combined with the calculated actual vehicle distance and expected distance, the distance error can be obtained. Through the speed error and distance error, the single-vehicle error state and the global error state can be further constructed, providing the original state input for the subsequent construction of a mixed traffic system model and the solution of robust optimal control. The entire information collection and distance determination process is the preliminary data link for realizing the control of connected autonomous vehicles.

[0028] Step 102: Subtract the vehicle spacing from the desired spacing to obtain the target vehicle spacing error, and subtract the target vehicle speed from the desired speed to obtain the target vehicle speed error.

[0029] In this step, under a mixed traffic environment, after calculating the actual vehicle spacing using vehicle location information and simultaneously acquiring the actual operating speed of the target vehicle, error processing is performed separately. The difference between the obtained actual vehicle spacing and the preset expected spacing is calculated to obtain the spacing error corresponding to the target vehicle; the difference between the actual operating speed of the target vehicle and the expected speed is calculated to obtain the speed error corresponding to the target vehicle. The spacing error and speed error together characterize the degree to which the vehicle's car-following state deviates from the ideal car-following state. Based on these two types of errors, a single-vehicle error state can be further constructed, and then the state information of each vehicle can be integrated to form a global error state. These two types of error states are used as core state variables input to the modeling stage. Combined with the optimal speed model describing the car-following behavior of manually driven vehicles, the longitudinal acceleration of the connected autonomous vehicle is used as the control input. At the same time, unknown external disturbances characterizing various uncertain driving behaviors and external factors are introduced to establish an affine nonlinear mixed traffic system model. This model, based on the error states, can reflect the inherent nonlinear characteristics and uncertainties of the mixed traffic system.

[0030] Step 103: Use the spacing error as the first state component and the speed error as the second state component to form the single-vehicle error state of the target vehicle.

[0031] Step 104: Combine the individual vehicle error states of each vehicle in the mixed traffic scenario according to the order of vehicles from front to back to generate the global error state.

[0032] In this step, after calculating the spacing error and speed error, the spacing error is used as the first state component and the speed error as the second state component, combined to form the single-vehicle error state of the target vehicle. This single-vehicle error state can centrally reflect the overall deviation of a single vehicle from the ideal car-following condition in the two key dimensions of spacing and speed during the car-following process. After obtaining the single-vehicle error states of each vehicle, the single-vehicle error states of all vehicles are integrated to further construct the global error state, which fully describes the overall deviation of the entire hybrid traffic system. The single-vehicle error state is the basic state variable for building the hybrid traffic dynamics model. Combined with the optimal speed model used to characterize the car-following behavior of manually driven vehicles, and using the longitudinal acceleration of the connected autonomous vehicle as the control input, unknown external disturbances from the driver's random behavior and the external environment are introduced to establish an affine nonlinear hybrid traffic system model. This model fully reflects the nonlinearity and disturbance uncertainty of the hybrid traffic system, laying the model foundation for the subsequent construction of the H-infinity robust optimal control problem. It is also the core state input source for the subsequent construction of the Q function, the online learning of the evaluation neural network, and the solution of the robust optimal control input of the connected autonomous vehicle.

[0033] Step 105: Affine nonlinear hybrid traffic system model, expressed by the following formula:

[0034] in, The rate of change of the state vector over time. This is the global error state vector. Let be the system drift state function, describing the nonlinear evolution of the system state itself in the absence of control input and external disturbances. These are control input channel functions or matrices used to describe how the control inputs of a connected autonomous vehicle affect changes in the system state. For control input, namely the longitudinal acceleration command of the connected autonomous vehicle, The disturbance input channel function or matrix is ​​used to describe how an unknown external disturbance affects the system state change. Unknown external disturbances include random following behavior of manually driven vehicles, driver operation errors, and external traffic influences.

[0035] In this step, an affine nonlinear hybrid traffic system model is used to characterize the evolution of the state over time under hybrid traffic flow. The model uses the global error state vector as the core state variable. In the model expression, the rate of change of the state vector over time is determined by three parts. The first part is the drift state function, which represents the inherent nonlinear evolution law of the hybrid traffic system under the condition of no control input and external disturbance, reflecting the inherent dynamic characteristics of the traffic flow itself. The second part is composed of the control input channel function and the control input in the form of longitudinal acceleration command from the connected autonomous vehicle. The control input channel function reflects the way the control command of the connected autonomous vehicle affects the state change. The third part is composed of the disturbance input channel function and the unknown external disturbance. The disturbance input channel function represents the path of various external disturbances affecting the state. The unknown external disturbances include the random following behavior of the manually driven vehicle, driver operation errors, and various external traffic influencing factors. This model fully takes into account the nonlinear characteristics of the hybrid traffic system and various uncertain disturbances in the real-world scenario, providing an accurate description for the subsequent construction of the H-infinity robust optimal control problem. It is also an important model foundation for carrying out Q-function design, evaluating online learning of neural networks, and then solving for the robust optimal control input of the connected autonomous vehicle.

[0036] Step 20: Based on the affine nonlinear hybrid traffic dynamics model, the control input of the connected autonomous vehicle and the unknown external disturbance are used as two action quantities in a zero-sum game to generate the H-infinite robust optimal control problem.

[0037] As an optional implementation, based on an affine nonlinear hybrid traffic dynamics model, the control input of the connected autonomous vehicle and the unknown external disturbance are used as two agents in a zero-sum game to generate an H-infinite robust optimal control problem. Specifically, it may include: Step 201: Based on the global error state, control input, and unknown external disturbance, define an instantaneous reward function that includes a state error term, a control input term, and a disturbance attenuation term, and integrate the instantaneous reward function over the time interval to construct a performance index function, expressed by the following formula:

[0038]

[0039] in, For performance index functions, For integration time variable, For the current moment, For instantaneous reward function, This is a penalty term for state error. This is the transpose of the state vector. Here, is the state weight matrix, a symmetric positive definite matrix used to weigh the importance of each state error component. This is the global error state vector. To control input penalty items, To control the transpose of the input vector. To control the weight matrix, This is the control input vector for connected autonomous vehicles. For disturbance penalty items, The level of disturbance attenuation. This is the transpose of the perturbation vector. The external disturbance vector is unknown.

[0040] In this step, using the established affine nonlinear hybrid traffic system model, an instantaneous reward function is defined with global error state, control input, and unknown external disturbances as basic elements. This instantaneous reward function simultaneously includes a state error term, a control input term, and a disturbance attenuation term. The state error term constrains the deviation of the global error state, the control input term constrains the control behavior of the connected autonomous vehicle, and the disturbance attenuation term suppresses unknown external disturbances. The instantaneous reward function is integrated over the corresponding time interval to construct a performance index function. This performance index function comprehensively considers car-following deviation, control behavior cost, and disturbance suppression capability, and can fully evaluate the comprehensive control effect of the connected autonomous vehicle in a hybrid traffic environment. Based on this performance index function, the control input and unknown external disturbances of the connected autonomous vehicle are regarded as two agents in a zero-sum game, thus establishing an H-infinite robust optimal control problem. The goal of solving this control problem is to find the optimal control strategy that makes the above performance index function optimal in the corresponding game sense.

[0041] Step 202: Use the control input as the action that minimizes the performance index function and the unknown external disturbance as the action that maximizes the performance index function to construct a zero-sum game relationship.

[0042] In this step, a zero-sum game relationship is constructed based on the already established performance index function. The control input of the connected autonomous vehicle is set as the action that strives to minimize the performance index function, representing the controller's desire to reduce state deviations and control costs as much as possible to achieve ideal vehicle following control performance. Conversely, unknown external disturbances are set as the action that strives to maximize the performance index function, simulating the adverse effects of various uncertain disturbances on the control performance of the hybrid traffic system. By setting these mutually opposing optimization objectives for the two types of actions, a zero-sum game relationship is formed.

[0043] Step 203: Based on the zero-sum game relationship, generate the H-infinite robust optimal control problem that minimizes the performance index function under the worst case of unknown external disturbances.

[0044] In this step, based on the established zero-sum game relationship, the control input objective of the connected autonomous vehicle is to minimize the performance index function, while the unknown external disturbance aims to maximize the performance index function, forming an antagonistic game relationship between the two. Based on this game framework, an H-infinite robust optimal control problem is generated. The goal of solving this control problem is to obtain a control strategy that minimizes the performance index function under the worst-case scenario of unknown external disturbance. The optimal solution to this problem satisfies the Hamiltonian equation, which can simultaneously take into account control optimality and the ability to suppress external disturbances.

[0045] Step 30: Construct the Q function based on the infinite robust optimal control problem, and use the evaluation neural network to parameterize the Q function, thus transforming the infinite robust optimal control problem into a learning problem of the weights of the evaluation neural network.

[0046] As an optional embodiment, a Q function is constructed based on the H-infinite robust optimal control problem, and an evaluation neural network is used to parameterize the Q function, thereby transforming the infinite robust optimal control problem into a learning problem of the evaluation neural network weights. Specifically, this may include: Step 301: Adding the optimal value function and Hamiltonian function of the H-infinite robust optimal control problem to construct the Q function, and using an evaluation neural network to parameterize the Q function. The Q function is represented by a combination of the ideal weights and activation function of the evaluation neural network, as shown by the following formula:

[0047] in, The Q function in parameterized representation, To evaluate the ideal weight vector of a neural network, This is the transpose of the weight vector. For activation function, This represents the approximation error of the neural network.

[0048] In this step, for the already generated H-infinity robust optimal control problem, the optimal value function corresponding to the control problem is added to the Hamiltonian function to complete the construction of the Q function. This Q function can fully represent the entire H-infinity robust optimal control problem, realizing a unified description of the correlation between state, control input, unknown external disturbance, and performance indicators. To eliminate the dependence on precise dynamic information and realize online solution of the problem, an evaluation neural network is used to perform parameterization processing on the Q function. The Q function is expressed by using a combination of the ideal weights and activation function of the evaluation neural network, and this parameterized expression also includes the approximation error of the neural network. Through the above parameterization processing, the original complex problem of solving H-infinity robust optimal control is transformed into a learning problem of the ideal weights of the evaluation neural network. The original complex optimal control solution task is transformed into a neural network parameter learning task, laying the foundation for subsequent acquisition of control strategies using a data-driven approach, and providing a feasible implementation path for solving control strategies for connected autonomous vehicles under conditions without precise models.

[0049] Step 302: The global error state, the control input of the connected autonomous vehicle, and the unknown external disturbances are mapped into unified learning features through the activation function. Based on the parameterized Q function, the optimal value function of the H-infinite robust optimal control problem is transformed into the learning problem of the evaluation neural network weights, so that the learning of the evaluation neural network weights is equivalent to solving the H-infinite robust optimal control problem.

[0050] In this step, activation functions are used to map the global error state, the control input of the connected autonomous vehicle, and unknown external disturbances, transforming these different types of variables into a unified form of learning features. This achieves feature fusion of state, control behavior, and external disturbance information. Based on the Q-function parameterized by the evaluation neural network, the problem form is transformed, converting the original optimal value function solution task in the H-infinite robust optimal control problem into a learning problem of evaluation neural network weights. After this transformation, the learning process of the evaluation neural network weights is equivalent to solving the H-infinite robust optimal control problem. This approach no longer requires prior knowledge of complete dynamic information. All information related to the control problem is encapsulated in the neural network weights, and the process of the weights continuously approaching the ideal value is the process of approaching the theoretical solution of the original robust optimal control problem.

[0051] Step 40: Based on the Bellman optimality principle, establish an integral Q-learning regression relationship, and construct auxiliary filter variables and auxiliary error feedback terms to generate convergent evaluation neural network weights.

[0052] As an optional implementation, based on the Bellman optimality principle, an integral Q-learning regression relationship is established, and auxiliary filter variables and auxiliary error feedback terms are constructed to generate convergent evaluation neural network weights. Specifically, it may include: Step 401: Within the sampling period, based on the Bellman optimality principle, the state value function before the current time is expressed as the sum of the state value function at the current time and the integral of the instantaneous reward function within the sampling period.

[0053] In this step, within the set sampling period, the Bellman optimality principle is used to derive the relationship. The Bellman optimality principle embodies the recursive nature of optimal control problems, establishing an intrinsic correlation between the state-value function and reward information at different times. According to this principle, the state-value function before the current time is expressed as the sum of the state-value function at the current time and the integral of the instantaneous reward function within the sampling period. This expression establishes a recursive relationship between sampling times, incorporating the reward integral obtained over a period into the calculation logic of the state-value function. This recursive relationship theoretically characterizes the evolution of the state-value function with the sampling time sequence, organically combining the temporal state changes and the reward information obtained within the sampling period. Using this relationship, an integral Q-learning update relationship can be further constructed without relying on explicit analytical expressions of dynamics, fully utilizing the data information obtained from temporal sampling. This function expression based on the Bellman principle completes the mathematical abstraction of the temporal control process, providing a theoretical basis for subsequent learning and solving using sampled data, and transforming the robust optimal control problem in continuous time into a form that can utilize sampled data.

[0054] Step 402: Based on the integral relationship between the state value function and the instantaneous reward function, define an integral reinforcement term and, in conjunction with the parameterized Q function, establish an integral Q-learning regression relationship for evaluating the learning of neural network weights.

[0055] In this step, the integral relationship between the state-value function and the instantaneous reward function obtained through the Bellman optimality principle is used to define a corresponding integral reinforcement term. This integral reinforcement term integrates the instantaneous reward integral information within the sampling period and is used to characterize the comprehensive reward obtained by the control process within a sampling period. Combined with the parameterized Q-function, the integral reinforcement term is correlated with the temporal changes of the Q-function, thereby establishing an integral Q-learning regression relationship specifically for evaluating the learning of neural network weights. This regression relationship establishes a connection between the actual data acquired through sampling and the parameters of the evaluation neural network, transforming the task of solving robust optimal control in continuous time into a regression solution task. This integral construction method is adaptable to the characteristics of continuous sampling processing and can directly utilize the time-series data collected during operation without needing to know the internal dynamic analytical form. Through this regression relationship, the state, control input, and disturbance-related information generated during actual operation can all participate in the learning process, providing constraints for the iterative update of the evaluation neural network weights, achieving data-driven approximation of ideal weights, and providing a core data regression foundation for solving robust optimal control problems under model-free conditions.

[0056] Step 403: Based on the difference of the activation function at adjacent sampling times, construct the first auxiliary filter variable and construct the second auxiliary filter variable based on the integral enhancement term.

[0057] In this step, after obtaining the integral Q-learning regression relationship, two sets of auxiliary filter variables need to be constructed to solve the data processing problem in the evaluation of neural network weight learning. The first auxiliary filter variable is constructed based on the difference in activation functions at adjacent sampling times. This variable filters the change information of activation functions at different sampling times, extracting the change information of features in the temporal dimension and processing the temporal differences of neural network input features. The second auxiliary filter variable is then constructed based on the already defined integral reinforcement term. This variable filters the accumulated reward information within the sampling period, converting the control reward information contained in the integral reinforcement term into a variable form suitable for regression calculation. The two sets of auxiliary filter variables respectively process the temporal changes of neural network features and the reinforcement reward information within the sampling period, preprocessing the original sampled data and eliminating some factors unfavorable to online learning. The filtered data can adapt to the input requirements of subsequent adaptive learning, building a bridge between the original sampled data and the weight update logic. By constructing the above auxiliary filter variables, the complex integral regression relationship can be transformed into a variable form that is easy to implement online, creating the preconditions for the subsequent generation of auxiliary error feedback terms and supporting the relevant computational processing of the evaluation neural network in a model-free framework.

[0058] Step 404: Construct an auxiliary error feedback term based on the first auxiliary filter variable, the second auxiliary filter variable, and the estimated weights of the evaluation neural network.

[0059] In this step, after constructing the first and second auxiliary filter variables, the auxiliary error feedback term is constructed based on the current estimated weights of the evaluation neural network. The first auxiliary filter variable carries the temporal change information of the activation function between adjacent sampling times, the second auxiliary filter variable retains the reward information after filtering transformation of the integral reinforcement term, and the estimated weights of the evaluation neural network represent the approximate result of the ideal weights at the current stage. The three types of elements are combined to obtain the auxiliary error feedback term. This auxiliary error feedback term reflects the deviation between the current approximate result of the neural network and the regression relationship under ideal conditions, quantifying the fitting error caused by the current weight estimation. It integrates the temporal change information of features, the periodic reward information, and the estimation deviation of network parameters into a unified error quantity, which can serve as a feedback signal driving weight adjustment. With this feedback term, the deviation information of the regression relationship can be extracted without relying on an accurate dynamic model, transforming the control problem-solving information contained in the data into an error signal that can be used for adaptive adjustment. This provides an error basis for the iterative correction of the evaluation neural network weights, realizing the effective utilization of fitting deviations within the online learning framework.

[0060] Step 405: Introduce finite excitation conditions. When the finite excitation conditions are met, select the filtering data corresponding to the moment when the minimum eigenvalue of the filtering variable matrix is ​​maximized. Based on the auxiliary error feedback term and the filtering data, design an online adaptive law to evaluate the weights of the neural network, so as to update the weights of the neural network online until convergence.

[0061] In this step, to ensure the successful convergence of the evaluation neural network weights, a finite excitation condition is introduced. This condition guarantees that the data used for learning has sufficient information richness, avoiding the problem of information matrix degradation during the learning process. When the finite excitation condition is met during the operation, the filtered data corresponding to the moment when the smallest eigenvalue of the filter variable matrix is ​​the largest is selected, thereby filtering out effective sample data with sufficient information content. Combining the filtered data and the constructed auxiliary error feedback term, an online adaptive law for the evaluation neural network weights is designed. This online adaptive law uses the auxiliary error feedback term as the driving signal and utilizes the filtered effective data to achieve continuous online updates of the evaluation neural network weights. The entire update process does not rely on precise dynamic information, nor does it require a pre-provided initial stable control strategy. Under the continuous action of the adaptive law, the weight estimates are continuously corrected until the evaluation neural network weights reach a convergent state. By constraining and filtering the learning data through the finite excitation condition, and in conjunction with the corresponding adaptive law, the stability of the weight iteration process is ensured, achieving an approximation of the ideal weights of the evaluation neural network under online conditions, and completing the data-driven solution to the robust optimal control problem of H infinity.

[0062] Step 50: Based on the evaluation neural network weights, calculate the robust optimal control input for the connected autonomous vehicle and output it to the longitudinal control end of the connected autonomous vehicle to generate the robust optimal control command for the connected autonomous vehicle.

[0063] As an optional embodiment, the robust optimal control input of the connected autonomous vehicle is calculated based on the evaluation neural network weights and output to the longitudinal control end of the connected autonomous vehicle to generate the robust optimal control command of the connected autonomous vehicle. Specifically, it may include: Step 501: Extract the weight estimate value of the evaluation neural network weights after the online adaptive law converges, and based on the converged weight estimate value, calculate the partial derivatives of the parameterized Q function with respect to the control input and the unknown external disturbance.

[0064] In this step, after the evaluation neural network weights reach convergence through online adaptive law iterative updates, the converged weight estimates are extracted. These weight estimates are data-driven approximations of the ideal weights of the evaluation neural network, carrying the solution information for the H-infinity robust optimal control problem in a mixed traffic scenario. Based on these converged weight estimates, the parameterized Q-function is substituted to calculate the partial derivatives of the Q-function with respect to the control input of the connected autonomous vehicle and the partial derivatives of the Q-function with respect to unknown external disturbances. By calculating the partial derivatives, the extreme value conditions inherent in the parameterized Q-function can be discovered, corresponding to the saddle point solution logic under the zero-sum game framework. This operation does not require knowledge of the original dynamic analytical model; all calculations are based on the learned network weights. By calculating the partial derivatives for the two types of variables, the extreme value relationship between the control input, the unknown external disturbance, and the learned Q-function can be established. This provides a mathematical basis for deriving practically usable control inputs and worst-case disturbance strategy estimates, transforming the learning results of the neural network into the solution basis for the optimal control problem.

[0065] Step 502: Set the partial derivative of the Q function with respect to the control input to zero, and set the partial derivative of the Q function with respect to the unknown external disturbance to zero. Solve the system of equations simultaneously to obtain the estimated values ​​of the control input and the unknown external disturbance.

[0066] In this step, after obtaining the partial derivatives of the parameterized Q-function with respect to the control input and the unknown external disturbance, the partial derivatives of the Q-function with respect to the control input of the connected autonomous vehicle are set to zero, and the partial derivatives of the Q-function with respect to the unknown external disturbance are also set to zero. The two equations are then solved simultaneously. This approach corresponds to the extreme value condition of the saddle point in a zero-sum game, representing the simultaneous satisfaction of the minimization condition on the control side and the maximization condition on the disturbance side. The solution process of the simultaneous equations is entirely based on the converged weight estimates of the evaluation neural network, without relying on the dynamic analytical expressions. By solving the simultaneous equations, the estimated values ​​of the control input and the unknown external disturbance can be obtained, respectively. The estimated value of the control input corresponds to the robust optimal control output of the connected autonomous vehicle, while the estimated value of the unknown external disturbance is an approximation of the worst-case disturbance strategy in the scenario. This step transforms the Q-function information learned from the data into variable estimation results that can be actually output, and transforms the parameter learning results of the neural network into candidate solutions to the robust optimal control problem, providing theoretical solution results for the generation of control commands.

[0067] Step 503: Use the estimated control input as the robust optimal control input for the connected autonomous vehicle.

[0068] Step 504: Convert the calculated robust optimal control input of the connected autonomous vehicle into a longitudinal acceleration command, and send the longitudinal acceleration command to the longitudinal control terminal of the connected autonomous vehicle.

[0069] In this step, the robust optimal control input of the connected autonomous vehicle obtained from the simultaneous solution is transformed to generate a longitudinal acceleration command that the vehicle can execute. This completes the conversion of the control solution results into a vehicle-recognizable and executable command. This longitudinal acceleration command corresponds to the control behavior that meets the robust optimal requirement of H in a mixed traffic scenario, and can balance optimal performance and the ability to suppress unknown external disturbances. After the command conversion is completed, the longitudinal acceleration command is sent to the longitudinal control terminal of the connected autonomous vehicle. After receiving the command, the longitudinal control terminal can adjust the longitudinal motion state of the connected autonomous vehicle according to the command. The entire process is based on the results obtained from the previous model-free online learning, without the need for prior knowledge of the precise dynamic model. It directly converts the control quantity obtained from the learning solution into the actual execution signal of the vehicle. This step establishes a link between the control algorithm solution and the vehicle's actual execution, enabling the robust optimal solution obtained at the algorithm level to be applied to the actual vehicle, thereby leveraging the control strategy and realizing the complete transmission from the algorithm calculation results to the vehicle motion adjustment command.

[0070] Step 505: The longitudinal control terminal generates drive actuator control signals and brake actuator control signals based on the longitudinal acceleration command.

[0071] In this step, after receiving the longitudinal acceleration command, the longitudinal control terminal of the connected autonomous vehicle parses and allocates the acceleration command based on the characteristics of the vehicle's own longitudinal actuators, generating drive actuator control signals and brake actuator control signals respectively. The longitudinal acceleration command contains the control intention of the vehicle to accelerate or decelerate. The longitudinal control terminal completes the reasonable allocation of execution signals according to the different control intentions. When positive acceleration is required, drive actuator control signals are mainly generated; when negative acceleration is required, brake actuator control signals are output. The above two types of actuator control signals are control commands that can directly act on the vehicle's underlying hardware, transforming the abstract acceleration command output by the upper-level algorithm into electrical control signals that the vehicle hardware can recognize. This process realizes the further conversion of control commands from the algorithm logic level to the vehicle hardware execution level, connecting the upper-level robust optimal control solution results with the vehicle's physical execution components, providing a direct signal source for subsequent changes in the vehicle's motion state through drive and brake actuators, and completing the signal conversion link to the underlying hardware in the control command chain.

[0072] Step 506: Use the drive actuator control signal and the brake actuator control signal to drive the vehicle's longitudinal actuators to adjust the speed and spacing of the connected autonomous vehicle in order to suppress unknown external disturbances and stabilize the mixed traffic flow.

[0073] In this step, the generated drive actuator control signal and brake actuator control signal are input to the corresponding longitudinal actuator of the vehicle to drive the longitudinal actuator and adjust the motion state of the connected autonomous vehicle. Through the actuator's action response, the vehicle's speed and following distance are dynamically adjusted so that the vehicle's actual motion state continuously approaches the ideal following condition. This adjustment process can suppress unknown external disturbances in mixed traffic scenarios and reduce the adverse effects of random behavior of manually driven vehicles and various traffic disturbances. Based on the stable control of speed and distance of a single connected autonomous vehicle, the changes in the vehicle's following state will be transmitted in the traffic flow, thus having a positive effect on the driving state of surrounding vehicles and ultimately achieving the stable maintenance of the overall mixed traffic flow. This step completes the closed loop from the underlying execution signal to the actual change in vehicle motion, implementing the control strategy obtained from the previous data-driven robust optimal control algorithm. With the longitudinal regulation capability of the connected autonomous vehicle, it offsets various uncertain disturbances, reduces the risk of traffic loss and instability, and ensures the smooth and orderly operation of traffic flow in mixed traffic environments.

[0074] The above are embodiments of the method proposed in this application. Based on the same inventive concept, embodiments of this application also provide a model-free robust optimal control device for connected vehicles, the structure of which is as follows: Figure 2 As shown.

[0075] Figure 2 This is a schematic diagram of the internal structure of a model-free robust optimal control device for connected vehicles, provided as an embodiment of this application. Figure 2 As shown, the device includes: At least one processor 201; And a memory 202 that is communicatively connected to at least one processor; The memory 202 stores instructions executable by at least one processor. These instructions are executed by at least one processor 201 to enable the processor 201 to: construct a single-vehicle error state and a global error state based on the speed error between vehicle speed and desired speed, and the spacing error between vehicle spacing and desired spacing in a mixed traffic scenario; and input the longitudinal acceleration of the connected autonomous vehicle and unknown external disturbances into a preset optimal speed model to establish an affine nonlinear mixed traffic system model; and based on the affine nonlinear mixed traffic dynamic model, use the control input of the connected autonomous vehicle and unknown external disturbances as two elements in a zero-sum game. A robust optimal control problem (H-infinite) is generated using a given action. Based on this problem, a Q-function is constructed, and an evaluation neural network is used to parameterize it, transforming the infinite robust optimal control problem into a learning problem for the evaluation neural network weights. Based on the Bellman optimality principle, an integral Q-learning regression relationship is established, and auxiliary filter variables and auxiliary error feedback terms are constructed to generate convergent evaluation neural network weights. Based on these weights, the robust optimal control input for the connected autonomous vehicle is calculated and output to the longitudinal control terminal of the connected autonomous vehicle to generate the robust optimal control command for the connected autonomous vehicle.

[0076] Some embodiments of this application provide corresponding to Figure 1 A non-volatile computer storage medium for model-free robust optimal control of connected vehicles stores computer-executable instructions. These instructions are configured to: construct single-vehicle error states and global error states based on the speed error between vehicle speed and desired speed, and the spacing error between vehicle spacing and desired spacing in a mixed traffic scenario; input the longitudinal acceleration of the connected autonomous vehicle and unknown external disturbances into a preset optimal speed model to establish an affine nonlinear mixed traffic system model; and based on the affine nonlinear mixed traffic dynamic model, use the control input of the connected autonomous vehicle and unknown external disturbances as two parties in a zero-sum game. The system generates an infinite robust optimal control problem (H-infinite). Based on this problem, a Q-function is constructed, and an evaluation neural network is used to parameterize it, transforming the infinite robust optimal control problem into a learning problem for the evaluation neural network weights. Based on the Bellman optimality principle, an integral Q-learning regression relationship is established, and auxiliary filter variables and auxiliary error feedback terms are constructed to generate convergent evaluation neural network weights. Based on these weights, the robust optimal control input for the connected autonomous vehicle is calculated and output to the longitudinal control terminal of the connected autonomous vehicle to generate the robust optimal control command for the connected autonomous vehicle.

[0077] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments for IoT devices and media are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0078] The systems, media, and methods provided in this application are one-to-one correspondences. Therefore, the systems and media also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the systems and media will not be repeated here.

[0079] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0080] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0081] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0082] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0083] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0084] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0085] Computer-readable media include both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0086] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0087] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A model-free robust optimal control method for connected vehicles, characterized in that, The method includes: Based on the speed error between vehicle speed and desired speed and the spacing error between vehicle spacing and desired spacing in a mixed traffic scenario, a single-vehicle error state and a global error state are constructed. The longitudinal acceleration of the connected autonomous vehicle and unknown external disturbances are input into a preset optimal speed model to establish an affine nonlinear mixed traffic system model. The unknown external disturbances include random following behavior of vehicles, driver operation errors, and external traffic influences. Based on the aforementioned affine nonlinear hybrid traffic dynamics model, the control input of the connected autonomous vehicle and the unknown external disturbance are used as two action quantities in a zero-sum game to generate an H-infinite robust optimal control problem. Based on the aforementioned infinite robust optimal control problem, a Q function is constructed, and an evaluation neural network is used to parameterize the Q function, thereby transforming the infinite robust optimal control problem into a learning problem of the weights of the evaluation neural network. Based on the Bellman optimality principle, an integral Q-learning regression relationship is established, and auxiliary filter variables and auxiliary error feedback terms are constructed to generate convergent evaluation neural network weights. Based on the weights of the evaluation neural network, the robust optimal control input for the connected autonomous vehicle is calculated and output to the longitudinal control terminal of the connected autonomous vehicle to generate the robust optimal control command for the connected autonomous vehicle.

2. The model-free robust optimal control method for connected vehicles according to claim 1, characterized in that, Based on the speed error between vehicle speed and desired speed, and the spacing error between vehicle distance and desired distance in a mixed traffic scenario, a single-vehicle error state and a global error state are constructed, specifically including: The system collects the position and speed of a target vehicle and the vehicle in front of it in a mixed traffic scenario, and determines the vehicle spacing of the target vehicle based on the difference between the position of the target vehicle and the position of the vehicle in front of it. Subtract the desired distance from the vehicle distance to obtain the distance error of the target vehicle, and subtract the desired speed from the speed of the target vehicle to obtain the speed error of the target vehicle; The spacing error is used as the first state component and the speed error is used as the second state component to form the single-vehicle error state of the target vehicle. The individual vehicle error states of each vehicle in the mixed traffic scenario are combined in the order of vehicles from front to back to generate a global error state.

3. The model-free robust optimal control method for connected vehicles according to claim 1, characterized in that, The longitudinal acceleration of the connected autonomous vehicle and unknown external disturbances are input into a preset optimal speed model to establish an affine nonlinear hybrid traffic system model, specifically including: The affine nonlinear hybrid traffic system model is represented by the following formula: in, The rate of change of the state vector over time. This is the global error state vector. Let be the system drift state function, describing the nonlinear evolution of the system state itself in the absence of control input and external disturbances. These are control input channel functions or matrices used to describe how the control inputs of a connected autonomous vehicle affect changes in the system state. For control input, namely the longitudinal acceleration command of the connected autonomous vehicle, The disturbance input channel function or matrix is ​​used to describe how an unknown external disturbance affects the system state change. Unknown external disturbances include random following behavior of manually driven vehicles, driver operation errors, and external traffic influences.

4. The model-free robust optimal control method for connected vehicles according to claim 1, characterized in that, Based on the aforementioned affine nonlinear hybrid traffic dynamics model, and using the control input of the connected autonomous vehicle and the unknown external disturbance as two agents in a zero-sum game, an H-infinite robust optimal control problem is generated, specifically including: Based on the global error state, the control input, and the unknown external disturbance, an instantaneous reward function is defined, comprising a state error term, a control input term, and a disturbance attenuation term. The instantaneous reward function is then integrated over a time interval to construct a performance index function, expressed by the following formula: in, For performance index functions, For integration time variable, For the current moment, For instantaneous reward function, This is a penalty term for state error. This is the transpose of the state vector. Here, is the state weight matrix, a symmetric positive definite matrix used to weigh the importance of each state error component. This is the global error state vector. To control input penalty items, To control the transpose of the input vector. To control the weight matrix, This is the control input vector for connected autonomous vehicles. For disturbance penalty items, The level of disturbance attenuation. This is the transpose of the perturbation vector. The external disturbance vector is unknown. The control input is used as the force that minimizes the performance index function, and the unknown external disturbance is used as the force that maximizes the performance index function, so as to construct a zero-sum game relationship; Based on the zero-sum game relationship, an H-infinite robust optimal control problem is generated that minimizes the performance index function under the worst-case scenario of the unknown external disturbance.

5. The model-free robust optimal control method for connected vehicles according to claim 1, characterized in that, Based on the aforementioned infinite robust optimal control problem, a Q function is constructed, and an evaluation neural network is used to parameterize the Q function, thereby transforming the infinite robust optimal control problem into a learning problem of the evaluation neural network weights. Specifically, this includes: The optimal value function of the H-infinite robust optimal control problem is added to the Hamiltonian function to construct the Q-function. An evaluation neural network is then used to parameterize the Q-function, which is represented by the combination of the ideal weights and activation function of the evaluation neural network, as shown by the following formula: in, The Q function in parameterized representation, To evaluate the ideal weight vector of a neural network, This is the transpose of the weight vector. For activation function, This represents the approximation error of the neural network. The activation function maps the global error state, the connected autonomous vehicle control input, and the unknown external disturbance into a unified learning feature. Based on the parameterized Q function, the solution of the optimal value function of the H-infinite robust optimal control problem is transformed into the learning problem of the evaluation neural network weights, so that the learning of the evaluation neural network weights is equivalent to solving the H-infinite robust optimal control problem.

6. The model-free robust optimal control method for connected vehicles according to claim 1, characterized in that, Based on the Bellman optimality principle, an integral Q-learning regression relationship is established, and auxiliary filter variables and auxiliary error feedback terms are constructed to generate convergent evaluation neural network weights, specifically including: Within the sampling period, based on the Bellman optimality principle, the state value function before the current moment is represented as the sum of the state value function at the current moment and the integral of the instantaneous reward function within the sampling period; Based on the integral relationship between the state value function and the instantaneous reward function, an integral reinforcement term is defined, and combined with the parameterized Q function, an integral Q-learning regression relationship for evaluating the learning of neural network weights is established. Based on the difference in activation functions at adjacent sampling times, a first auxiliary filter variable is constructed, and based on the integral enhancement term, a second auxiliary filter variable is constructed. Based on the first auxiliary filter variable, the second auxiliary filter variable, and the weight estimate of the evaluation neural network, an auxiliary error feedback term is constructed; A finite excitation condition is introduced. When the finite excitation condition is satisfied, the filter data corresponding to the moment when the minimum eigenvalue of the filter variable matrix is ​​maximized is selected. Based on the auxiliary error feedback term and the filter data, an online adaptive law for evaluating the weights of the neural network is designed to update the weights of the neural network online until convergence.

7. The model-free robust optimal control method for connected vehicles according to claim 1, characterized in that, Based on the weights of the evaluation neural network, the robust optimal control input for the connected autonomous vehicle is calculated, specifically including: Extract the weight estimates of the evaluation neural network weights after the online adaptive law converges, and based on the converged weight estimates, calculate the partial derivatives of the parameterized Q function with respect to the control input and the unknown external disturbance. Set the partial derivative of the Q function with respect to the control input to zero, and set the partial derivative of the Q function with respect to the unknown external disturbance to zero, then solve the system of equations simultaneously to obtain the estimated values ​​of the control input and the unknown external disturbance. The estimated control input is used as the robust optimal control input for connected autonomous vehicles.

8. The model-free robust optimal control method for connected vehicles according to claim 1, characterized in that, Output to the longitudinal control terminal of the connected autonomous vehicle to generate robust optimal control commands for the connected autonomous vehicle, specifically including: The calculated robust optimal control input of the connected autonomous vehicle is converted into a longitudinal acceleration command, and the longitudinal acceleration command is sent to the longitudinal control terminal of the connected autonomous vehicle. The longitudinal control terminal generates drive actuator control signals and brake actuator control signals based on the longitudinal acceleration command; The drive actuator control signal and the brake actuator control signal are used to drive the vehicle's longitudinal actuator to adjust the speed and spacing of the connected autonomous vehicle in order to suppress unknown external interference and stabilize mixed traffic flow.

9. A model-free, robust, and optimal control device for connected vehicles, characterized in that, The device includes: At least one processor; And, a memory communicatively connected to the at least one processor; The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform a method as described in any one of claims 1-8.

10. A non-volatile computer storage medium for model-free robust optimal control of connected vehicles, storing computer-executable instructions, characterized in that, When the computer-executable instructions are executed, they implement a method as described in any one of claims 1-8.