Cooperative communication and energy consumption optimization heterogeneous vehicle queue control method

Abstract

The invention discloses a heterogeneous vehicle queue control method for cooperative communication and energy consumption optimization. A three-order heterogeneous multi-vehicle queue model, a dynamic event trigger mechanism module, a network update law evaluation module and an optimal queue controller module are included. Defining a local consistency error through heterogeneous vehicle formation information, and constructing a dynamic event triggering mechanism according to the local consistency error; according to the local consistency error and an error value output by the optimal queue controller module, designing an evaluation network weight updating law; and deriving the local consistency error to obtain the state of the local consistency error, and learning an optimal cost function and an optimization controller online in combination with an integral reinforcement learning algorithm and the evaluation network weight update law, so as to construct an optimal queue controller module. According to the method, the number of communication times between vehicles is reduced through a dynamic event triggering mechanism, and communication resources are saved. And meanwhile, the queue stability is realized in the optimal queue controller module, and the problem of minimization of energy consumption is solved.

Description

Technical Field

[0001] The present invention relates to the field of vehicle platoon control, and particularly to a heterogeneous vehicle platoon control method for collaborative communication and energy consumption optimization. Background Art

[0002] As an important carrier of intelligent transportation systems, automobiles, while providing convenient travel, also bring about many problems such as traffic congestion and environmental pollution. At the same time, the rapid development of fields such as information and communication technology, sensor technology, artificial intelligence, and autonomous driving technology has provided unprecedented opportunities to solve this problem. Therefore, the research on intelligent transportation systems has emerged. It not only aims to use advanced technologies to solve existing traffic problems, but also focuses on building a more efficient, safe, environmentally friendly, and sustainable transportation system. However, in the face of increasing traffic flow, complex road networks, and changing traffic patterns, optimizing traffic resource allocation, reducing traffic congestion, and improving traffic efficiency and safety have attracted wide attention.

[0003] During the process of multi-vehicle platoon driving, multiple vehicles need to maintain a constant inter-vehicle distance on a horizontal road surface. At present, many achievements have been made in the control research of multi-vehicle platoon systems, but there are still the following problems in the existing control methods for multi-vehicle platoon systems:

[0004] First, in the existing multi-vehicle platoon control systems, different types of vehicles have different dynamic characteristics. The heterogeneity of these vehicles (such as differences in acceleration and braking performance) will cause the model to deviate from the actual situation, making it difficult to accurately predict the platoon operation state and unable to design effective control strategies.

[0005] Second, during actual driving, factors such as wind resistance and emergencies will cause the vehicle platoon to have highly nonlinear and switching topological structures, which cannot be effectively handled by existing platoon control methods.

[0006] Third, in the existing control methods for heterogeneous multi-vehicle platoon systems, periodic sampling and control updates will generate a large amount of redundant calculations, especially when the traffic conditions are relatively stable. Frequent calculations and communications will not only increase the computational burden but also increase the communication pressure on the network. At the same time, it is impossible to ensure the minimization of the energy consumption of the vehicle platoon. Summary of the Invention

[0007] The present invention provides a heterogeneous vehicle platoon control method for collaborative communication and energy consumption optimization to overcome the above technical problems.

[0008] To achieve the above object, the technical solution of the present invention is:

[0009] A heterogeneous vehicle queue control method for collaborative communication and energy consumption optimization, characterized by including: a third-order heterogeneous multi-vehicle queue system model, a dynamic event triggering mechanism module, a judgment network update law module, and an optimal queue controller module;

[0010] The third-order heterogeneous multi-vehicle queue system module realizes heterogeneous vehicle formation by a leader vehicle model and a follower vehicle model, and outputs the corresponding heterogeneous vehicle formation information to the dynamic event triggering mechanism module;

[0011] And the heterogeneous vehicle formation information includes the displacement, speed, and acceleration of the leader vehicle, and the displacement, speed, and acceleration of the follower vehicle;

[0012] The dynamic event triggering mechanism module defines a local consistency error according to the heterogeneous vehicle formation information. When the local consistency error satisfies the conditions of the dynamic event triggering mechanism, the triggered local consistency error is output to the optimal queue controller module and the judgment network update law module;

[0013] The judgment network update law module is used to design a judgment network weight update law according to the local consistency error and the error output in the optimal queue controller module, and output the judgment network weight update law to the optimal queue controller module;

[0014] The optimal queue controller module uses the local consistency error state obtained by differentiating the local consistency error, combines the integral reinforcement learning algorithm and the judgment network weight update law, online learns the optimal cost function and the optimization controller, and at the same time inputs the error generated during the learning process into the judgment network update law module.

[0015] Further, the expression of the follower vehicle model is:

[0016]

[0017] In the formula: p i 、v i and a i respectively represent the displacement, speed, and acceleration of the i-th vehicle, Q i represents the mass of the i-th vehicle, represents a continuous unknown function, C i represents the air resistance coefficient, S i represents the cross-sectional area, and ρ is the air density; represents the actual driving force or braking force of the engine, e i represents the vehicle mechanical efficiency, h i represents an unknown external disturbance; s i =F ri (t)+F fi(t) represents the road surface slope coefficient, F ri (t) = Q i Gr i cos(θ i ) is the rolling resistance, θ i is the road surface slope, G is the acceleration due to gravity, r i is the rolling resistance coefficient; F fi (t) = Q i Gsin(θ i ) is the gravity; R i represents the radius of the vehicle tire; u i represents the throttle or brake control torque input, ι i represents the dynamic characteristic time constant;

[0018] The expression of the leading vehicle model is:

[0019]

[0020] Where: p0, v0, and a0 respectively represent the displacement, speed, and acceleration of the leading vehicle, and w0(t) is a bounded function.

[0021] Furthermore, the specific implementation steps of the dynamic event-triggering mechanism module for defining the local consistency error according to the heterogeneous vehicle formation information and constructing the dynamic event-triggering mechanism based on the local consistency error include:

[0022] First, define the local consistency error according to the heterogeneous vehicle formation as:

[0023]

[0024] Where: represents the switching signal, represents the connectivity between the following vehicle j and i under the switching signal ; that is, if the following vehicle i can receive the information of the following vehicle j, then otherwise, denote

[0025] Second, for the dynamic event-triggering mechanism, define the internal dynamic variable σ i , and take the derivative of the internal dynamic variable, and the corresponding expression is:

[0026]

[0027] Where: σ i (0) ≥ 0, γ i ∈(0, 1), θ i > 0 is a constant and σ i is non-negative; λmin (Q ii ) is the minimum eigenvalue of Q ii , where Q ii is the weight matrix in the cost function, and L i is a positive constant satisfying . is the triggering error at the l-th time, is an element in the diagonal matrix satisfying is an element in the diagonal matrix satisfying

[0028] Again, since the dynamic event-triggering mechanism determines whether the heterogeneous multi-vehicle queue system is triggered based on the changes in the internal dynamic variables, then according to the internal dynamic variable σ i , the (l + 1)-th triggering instant of the follower vehicle i is defined as:

[0029]

[0030] where:

[0031] Finally, combining the dynamic event-triggering mechanism (5), it is determined that when the local consistency error satisfies the following expression, the heterogeneous multi-vehicle queue system is triggered, and the corresponding expression is the following formula (6):

[0032]

[0033] At the same time, combining the triggering condition of the heterogeneous multi-vehicle queue topology, the triggering rule of the heterogeneous multi-vehicle queue system is:

[0034]

[0035] where the triggering condition of the heterogeneous multi-vehicle queue topology is that when the topology of the heterogeneous multi-vehicle queue switches, the heterogeneous multi-vehicle queue system will be triggered.

[0036] Further, the evaluation network update law module is used to design the specific implementation steps of the evaluation network weight update law according to the local consistency error and the error output in the optimal queue controller module, including:

[0037] First, define the cost function related to the local consistency error, and its expression is:

[0038]

[0039] where: Q ii T = Qii and R ii T = R ii denotes a positive definite matrix, R ij T = R ij is a positive semi - definite matrix; u j is the control input of the following vehicle j;

[0040] Secondly, the evaluation network is used to approximate the cost function, and its expression is:

[0041]

[0042] where: W i is the weight of the evaluation network, is the activation function, is the approximation error; and it satisfies and and are positive constants;

[0043] Thirdly, since the ideal evaluation network weight W i is unknown, we use to estimate the ideal evaluation network weight W i , then the estimated value of the cost function can be obtained as:

[0044]

[0045] where: the estimated value of the cost function is used in the policy evaluation step of the integral reinforcement learning algorithm, combined with the policy improvement step, to iteratively solve the optimal cost function;

[0046] Finally, for the third - order heterogeneous multi - vehicle queue, the following evaluation network weight update law is designed as the output of the evaluation network update law module, and its expression is:

[0047]

[0048] where: α i is the learning rate, g ∈ {1,..., M i} is the index of the g - th sample data stored in the sample stack ; and it satisfies rank(Η i ) = B < M i , B is the dimension of the evaluation network weight; the sample stack is the historical data of the activation function; is the residual caused by the evaluation network approximation error under the dynamic event - triggered mechanism; Λ i > 0 is the adjustment parameter; is the error generated during the iterative learning process in the optimal queue controller module.

[0049] Furthermore, the specific implementation steps of the optimal queue controller module for online learning of the optimal cost function and the optimal controller by using the local consensus error state obtained by differentiating the local consensus error, in combination with the integral reinforcement learning algorithm and the critic network weight update law, and inputting the error generated during the learning process into the critic network update law module are as follows:

[0050] First, by differentiating expression (3), the following local consensus error dynamics are obtained:

[0051]

[0052] where: ξ j :=[0,0,q j T , i ∈ {0} ∪ V;

[0053] Second, for the local consensus error dynamics (12), find the admissible control strategy that minimizes the cost function to obtain the optimal cost function, whose expression is:

[0054]

[0055] where: A(Ω) is the admissible control set and Ω is a compact set;

[0056] Third, by differentiating expression (8) and combining it with expression (12), the following Bellman equation is obtained:

[0057]

[0058] Based on the Bellman optimization criterion, the following Hamiltonian function is designed:

[0059]

[0060] By solving it can be obtained:

[0061]

[0062] Taking expression (16) as u i and substituting it into the Hamiltonian function, the Hamilton-Jacobi-Bellman equation is obtained:

[0063]

[0064] ​Fourthly, combining Expression (8) and the additivity of the integration interval, the following integral reinforcement learning Bellman equation is introduced to iteratively solve the Hamilton-Jacobi-Bellman equation (17):

[0065]

[0066] Fifthly, combining the integral reinforcement learning algorithm and the weight update law of the evaluation network, the following policy evaluation and policy improvement expressions are obtained:

[0067] Policy evaluation: Combining the weight update law of the evaluation network (11) and solving the integral reinforcement learning Bellman equation (18), we can obtain Its expression is:

[0068]

[0069] Policy improvement: Define the update iteration controller as:

[0070]

[0071] where: k is the number of iterations, T is the given time interval; σ i > 0 is a given threshold; the error generated during the iterative learning process and are input into the evaluation network update law module to update the weights of the evaluation network;

[0072] And loop to judge whether If so, it is determined that the weights of the evaluation network converge and the iteration stops; the cost function at this time is the optimal cost function, and the iteration controller is the optimized controller Optimized controller is the output value of the optimal queue controller module.

[0073] Compared with the prior art, the present invention has the following beneficial effects:

[0074] First, the present invention designs a dynamic event triggering mechanism by defining the local consistency error. Based on the internal dynamic variables, this mechanism dynamically judges the triggering timing of the third-order heterogeneous multi-vehicle queue system, and further designs the triggering rules of the third-order heterogeneous multi-vehicle queue system in combination with the topological triggering conditions of the heterogeneous multi-vehicle queue. The dynamic event triggering mechanism significantly improves the adaptability and flexibility of the system.

[0075] Second, the present invention designs a dynamic event trigger mechanism module for real-time judgment of the trigger conditions of dynamic events. When an event is triggered, this module synchronously transmits the trigger information to the evaluation network update law module and the optimal queue controller module. This mechanism effectively reduces the number of triggers, reduces the computational burden, and saves communication resources.

[0076] Third, by applying integral reinforcement learning combined with the evaluation network update law, the present invention online learns the optimal cost function and the optimal controller. After iterative cycles, the optimal cost function and the optimal controller are finally obtained. Therefore, the present invention fully considers the influence of unknown nonlinear dynamics on the heterogeneous multi-vehicle queue system, and there is no modeling error for unknown dynamics, thus ensuring the queue stability while achieving the minimization of energy consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0078] Figure 1 It is a schematic diagram of the results of the design method for the third-order heterogeneous multi-vehicle queue control;

[0079] Figure 2 It is a displacement trajectory diagram of the heterogeneous multi-vehicle queue system;

[0080] Figure 3 It is a speed trajectory diagram of the heterogeneous multi-vehicle queue system;

[0081] Figure 4 It is an acceleration trajectory diagram of the heterogeneous multi-vehicle queue system;

[0082] Figure 5 is the evaluation network parameters of the heterogeneous multi-vehicle queue system trajectory diagram;

[0083] Figure 5a It is the trajectory diagram when i = 1;

[0084] Figure 5b It is the trajectory diagram when i = 2;

[0085] Figure 5c It is the trajectory diagram when i = 3;

[0086] Figure 5d It is the trajectory diagram when i = 4;

[0087] Figure 5e It is the trajectory diagram when i = 5;

[0088] Figure 6 is a heterogeneous multi-vehicle queue system for judging dynamic event trigger diagrams;

[0089] Figure 7 is the internal dynamic variable σ of the heterogeneous multi-vehicle queue system i (i = 1, 2, 3, 4, 5) trajectory diagram;

[0090] Figure 8 is the norm trajectory diagram of the trigger error of the heterogeneous multi-vehicle queue system;

[0091] Figure 9 is the controller trajectory diagram of the heterogeneous multi-vehicle queue system. Specific implementation manner

[0092] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0093] This embodiment provides a control method for a heterogeneous vehicle queue for collaborative communication and energy consumption optimization, as Figure 1 shown, including a third-order heterogeneous multi-vehicle queue system model, a dynamic event trigger mechanism module, a judgment network update law module, and an optimal queue controller module; the connection is that the output end of the third-order heterogeneous multi-vehicle queue system model is connected to the input end of the dynamic event trigger mechanism module, and the input end is connected to the output end of the optimal queue controller module; the input end of the dynamic event trigger mechanism module is connected to the third-order heterogeneous multi-vehicle queue system model, and the output end is connected to the input ends of the judgment network update law module and the optimal queue controller module; the input end of the judgment network update law module is connected to the output end of the dynamic event trigger mechanism and the output end of the optimal queue controller module, and the output end is connected to the input end of the optimal queue controller module; the input end of the optimal queue controller module is connected to the output end of the dynamic event trigger mechanism module and the output end of the judgment network update law module, and the output end is connected to the input ends of the third-order heterogeneous multi-vehicle queue system model and the judgment network update law module.

[0094] The third-order heterogeneous multi-vehicle queue system module realizes heterogeneous vehicle formation by a leader vehicle model and a follower vehicle model, and outputs the corresponding heterogeneous vehicle formation information to the dynamic event trigger mechanism module; and the heterogeneous vehicle formation information includes the displacement, speed, and acceleration of the leader vehicle, and the displacement, speed, and acceleration of the follower vehicle;

[0095] Specifically, the expression of the follower vehicle model is:

[0096]

[0097] Where: p i , v i and a i represent the displacement, velocity, and acceleration of the i-th vehicle respectively, Q i represents the mass of the i-th vehicle, represents a continuous unknown function, C i represents the air resistance coefficient, S i represents the cross-sectional area, and ρ is the air density; represents the actual driving force or braking force of the engine, e i represents the mechanical efficiency of the vehicle, h i represents an unknown external disturbance; s i = F ri (t) + F fi (t) represents the road surface gradient coefficient, F ri (t) = Q i Gr i cos(θ i ) is the rolling resistance, θ i is the road surface gradient, G is the acceleration due to gravity, r i is the rolling resistance coefficient; F fi (t) = Q i G sin(θ i ) is the gravity; R i represents the radius of the vehicle tire; u i represents the throttle or brake control torque input, ι i represents the time constant of the dynamic characteristics;

[0098] The expression of the leading vehicle model is:

[0099]

[0100] Where: p0, v0, and a0 represent the displacement, velocity, and acceleration of the leading vehicle respectively, and w0(t) is a bounded function.

[0101] The specific implementation steps of the dynamic event-triggering mechanism module for defining the local consistency error according to the heterogeneous vehicle formation information and constructing the dynamic event-triggering mechanism based on the local consistency error include:

[0102] First, define the local consistency error according to the heterogeneous vehicle formation as:

[0103]

[0104] Where: Represents a handover signal, indicating the connectivity between following vehicle j and i under the handover signal; that is, if following vehicle i can receive information from following vehicle j, then otherwise, otherwise, denote

[0105] Secondly, for the dynamic event-triggering mechanism, define the internal dynamic variable σ i , and take the derivative of the internal dynamic variable. The corresponding expression is:

[0106]

[0107] In the formula: σ i (0)≥0, γ i ∈(0,1), θ i >0 is a constant and σ i is non-negative; λ min (Q ii ) is the minimum eigenvalue of Q ii , Q ii is the weight matrix in the cost function, L i is a positive constant that satisfies , is the triggering error at the l-th time, is an element in the diagonal matrix that satisfies is an element of the diagonal matrix that satisfies

[0108] Thirdly, since the dynamic event-triggering mechanism determines whether the heterogeneous multi-vehicle queue system is triggered based on the change of the internal dynamic variable, then according to the internal dynamic variable σ i , define the (l + 1)-th triggering instant of following vehicle i:

[0109]

[0110] In the formula:

[0111] Finally, combining the dynamic event-triggering mechanism (5), determine that when the local consistency error satisfies the following expression, the heterogeneous multi-vehicle queue system is triggered. The corresponding expression is the following formula (6):

[0112]

[0113] At the same time, combining the triggering conditions of the heterogeneous multi-vehicle queue topology, the triggering rule of the heterogeneous multi-vehicle queue system is:

[0114]

[0115] Among them, the triggering condition of the heterogeneous multi-vehicle queue topology is that when the topology of the heterogeneous multi-vehicle queue switches, the heterogeneous multi-vehicle queue system will be triggered.

[0116] In this embodiment, the dynamic event triggering mechanism module defines the local consistency error by using the information of the heterogeneous vehicle formation. When the local consistency error satisfies the dynamic event triggering mechanism, the local consistency error is input into the evaluation network update law module and the optimal queue controller module for subsequent work.

[0117] The evaluation network update law module is used to design the specific implementation steps of the evaluation network weight update law according to the local consistency error and the error output by the optimal queue controller module, including:

[0118] First, define the cost function related to the local consistency error, and its expression is:

[0119]

[0120] In the formula: Q ii T = Q ii and R ii T = R ii represent positive definite matrices, and R ij T = R ij is a semi-positive definite matrix; u j is the control input of the follower vehicle j;

[0121] Second, use the evaluation network to approximate the cost function, and its expression is:

[0122]

[0123] In the formula: W i evaluation network weight, is the activation function, is the approximation error; and it satisfies and and are positive constants;

[0124] Third, since the ideal evaluation network weight W i is unknown, use to estimate the ideal evaluation network weight W i , then the estimated value of the cost function can be obtained as:

[0125]

[0126] In the formula: the estimated value of the cost function will be used in the policy evaluation step of the integrated reinforcement learning algorithm, combined with the policy improvement step, to iteratively solve the optimal cost function;

[0127] Finally, for the third-order heterogeneous multi-vehicle queue, design the following judgment network weight update law as the output of the judgment network update law module, and its expression is:

[0128]

[0129] In the formula: α i is the learning rate, g ∈ {1,..., M i} is the index of the g-th sample data stored in the sample stack ; and it satisfies rank(Η i ) = B < M i , where B is the dimension of the judgment network weight; the sample stack is the historical data of the activation function; is the residual caused by the approximation error of the judgment network under the dynamic event-triggering mechanism ; Λ i > 0 is the adjustment parameter; is the error generated during the iterative learning process in the optimal queue controller module.

[0130] The output of the optimal queue controller module acts on the heterogeneous multi-vehicle queue system, and the data generated by the heterogeneous multi-vehicle queue system participates in the update of the judgment network weight in formula (11) to accurately evaluate the performance of the heterogeneous multi-vehicle queue system; forming a closed-loop learning process of "policy evaluation → policy improvement → weight update".

[0131] In this embodiment, the input of the judgment network update law module is the local consistency error and the error value of the output of the optimal queue controller module, which participates in the update of the judgment network weight in the network update law module to accurately evaluate the performance of the heterogeneous multi-vehicle queue system, and finally outputs the judgment network weight law and the estimated value of the cost function to the optimal queue controller module.

[0132] The specific implementation steps of the optimal queue controller module, which uses the local consistency error state obtained by differentiating the local consistency error, combines the integrated reinforcement learning algorithm and the judgment network weight update law, and online learns the optimal cost function and the optimal controller, and at the same time inputs the error generated during the learning process into the judgment network update law module, include:

[0133] First, by taking the derivative of expression (3), the following local consistency error dynamics are obtained:

[0134]

[0135] where: ξ j :=[0,0,q j T , i ∈ {0} ∪ V;

[0136] Second, for the local consistency error dynamics (12), find the admissible control strategy that minimizes the cost function to obtain the optimal cost function, whose expression is:

[0137]

[0138] where: A(Ω) is the admissible control set and Ω is a compact set;

[0139] Third, by taking the derivative of expression (8) and combining it with expression (12), the following Bellman equation is obtained:

[0140]

[0141] Based on the Bellman optimization criterion, design the following Hamiltonian function:

[0142]

[0143] By solving it can be obtained that:

[0144]

[0145] Substitute expression (16) as u i into the Hamiltonian function to obtain the Hamilton-Jacobi-Bellman equation:

[0146]

[0147] This equation transforms the optimal control problem of the system into a problem of solving a partial differential equation, establishing the connection between the local consistency error optimal control and the optimal performance index . The subsequent policy evaluation and policy improvement steps are all carried out around solving this equation. Through continuous iteration, the control input that optimizes the system performance is obtained.

[0148] However, since equation (17) contains the unknown function w j (v j , a j ​For the non - linear partial differential equation in (), it is very difficult or even impossible to directly obtain its analytical solution.

[0149] Fourthly, in order to eliminate the influence brought by unknown dynamics, combining Expression (8) and the additivity of the integration interval, the following integral reinforcement learning Bellman equation is introduced to iteratively solve the Hamilton - Jacobi - Bellman equation (17):

[0150]

[0151] Fifthly, combining the integral reinforcement learning algorithm and the weight update law of the evaluation network, the following policy evaluation and policy improvement expressions are obtained:

[0152] Policy evaluation: Combining the weight update law of the evaluation network (11) and solving the integral reinforcement learning Bellman equation (18), we can obtain Its expression is:

[0153]

[0154] Policy improvement: Define the update and iteration controller as:

[0155]

[0156] where: k is the number of iterations, T is the given time interval; σ i > 0 is a given threshold; the error generated in the iterative learning process and are input into the evaluation network update law module to update the weights of the evaluation network;

[0157] And it is loop - judged whether When, if so, it is determined that the weights of the evaluation network converge and the iteration stops; at this time, the cost function is the optimal cost function, and the iteration controller is the optimized controller The optimized controller is the output value of the optimal queue controller module.

[0158] In this embodiment of heterogeneous multi - vehicle queue control, the displacement p i (t), speed v i (t) and acceleration a i (t) of the follower vehicle and the p0(t), speed v0(t) and acceleration a0(t) information of the leader vehicle are output to the dynamic event - triggering mechanism module; the output of the dynamic event - triggering mechanism module (is input into the evaluation network update law module and the optimal queue controller module; the output of the evaluation network update law module Input to the optimal queue controller module; the optimal queue controller module transmits the output signal to the evaluation network update law module and the third-order heterogeneous multi-vehicle queue system. The design objective of the present invention is to achieve the dynamic processing of emergencies during the driving process of the heterogeneous multi-vehicle queue, ensure that the speed and acceleration of the following vehicle can accurately track the speed and acceleration of the leading vehicle, while maintaining the spacing error bounded, and ensure collision-free operation of the queue.

[0159] Compared with the prior art, the present invention has the following beneficial effects:

[0160] First, the present invention designs a dynamic event triggering mechanism by defining the local consistency error. The dynamic event triggering mechanism determines whether the third-order heterogeneous multi-vehicle queue system is triggered based on the internal dynamic variables. At the same time, combined with the topological triggering conditions of the heterogeneous multi-vehicle queue, the triggering rules of the third-order heterogeneous multi-vehicle queue system are designed, enhancing the adaptability and flexibility of the system.

[0161] Second, the present invention designs a dynamic event triggering mechanism module to judge whether a dynamic event is triggered. When it is judged that a dynamic event is triggered, communication is enabled to output the triggered information to the evaluation network update law module and the optimal queue controller module. Thereby reducing the triggering times, reducing the computational burden, and saving communication resources.

[0162] Third, the present invention applies integral reinforcement learning combined with the evaluation network update law to online learn the optimal cost function and the optimal controller. After cyclic iteration, the optimal cost function and the optimal controller are finally obtained. Therefore, the present invention fully considers the influence of unknown non-linear dynamics on the heterogeneous multi-vehicle queue system, and there is no modeling error for unknown dynamics, thereby ensuring the stability of the queue while achieving the minimization of energy consumption.

[0163] The simulation results of this embodiment are as Figures 2-9 shown, indicating that all signals in the queue are bounded. Figure 2 represents the displacement curves of all vehicles. From Figure 2 it can be seen that the distance between each vehicle and the vehicle in front is greater than the minimum safety distance, that is, no collision will occur between vehicles; Figures 3-4 represents the speed and acceleration trajectories of all vehicles. From Figures 3-4 it can be seen that the stability of a single vehicle can be ensured; Figure 5 reflects the dynamic adjustment and convergence process of the evaluation network parameters. From Figure 5, it can be seen that the evaluation network parameters can gradually stabilize over time, indicating that the evaluation network has good self-adaptability and stability; Figure 6 represents the dynamic event triggering graph, reducing the communication frequency and the communication burden by setting triggering rules; Figure 7 represents the trajectory of the internal dynamic variable σ i defined additionally in the dynamic event triggering, verifying its non-negativity property; Figure 8It represents the locus of the modulus of the dynamic event-triggered error. It can be observed from the figure that all errors eventually stabilize near zero, indicating that the system has good stability. Figure 9 The control strategies under the dynamic event-triggered mechanism are compared. with the continuous-time control strategy of the locus, indicating that the dynamic event-triggered mechanism only updates the control strategy when the system meets the trigger condition, saving communication costs. At the same time, like the continuous-time control strategy, it can ensure the stability of the system.

[0164] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A heterogeneous vehicle queue control method for collaborative communication and energy consumption optimization, characterized in that Including: A third-order heterogeneous multi-vehicle queue system model, a dynamic event-triggering mechanism module, a judgment network update law module, and an optimal queue controller module; The third-order heterogeneous multi-vehicle queue system module realizes heterogeneous vehicle formation by a leader vehicle model and a follower vehicle model, and outputs the corresponding heterogeneous vehicle formation information to the dynamic event-triggering mechanism module; And the heterogeneous vehicle formation information includes the displacement, speed, and acceleration of the leader vehicle, and the displacement, speed, and acceleration of the follower vehicle; The dynamic event-triggering mechanism module defines a local consistency error according to the heterogeneous vehicle formation information. When the local consistency error satisfies the conditions of the dynamic event-triggering mechanism, the triggered local consistency error is output to the optimal queue controller module and the judgment network update law module; The judgment network update law module is used to design a judgment network weight update law according to the local consistency error and the error output in the optimal queue controller module, and output the judgment network weight update law to the optimal queue controller module; The optimal queue controller module uses the local consistency error state obtained by differentiating the local consistency error, combines the integral reinforcement learning algorithm and the judgment network weight update law, online learns the optimal cost function and the optimal controller, and at the same time inputs the error generated in the learning process into the judgment network update law module.

2. The heterogeneous vehicle queue control method for collaborative communication and energy consumption optimization according to claim 1, characterized in that, The expression of the follower vehicle model is: where: p i , v i and a i represent the displacement, velocity and acceleration of the i-th vehicle respectively, Q i represents the mass of the i-th vehicle, represents a continuous unknown function, C i represents the air resistance coefficient, S i represents the cross-sectional area, and ρ is the air density; represents the actual driving force or braking force of the engine, e i represents the mechanical efficiency of the vehicle, h i represents an unknown external disturbance; s i = F ri (t) + F fi (t) represents the road surface gradient coefficient, F ri (t) = Q i Gr i cos(θ i ) is the rolling resistance, θ i is the road surface gradient, G is the acceleration due to gravity, r i is the rolling resistance coefficient; F fi (t) = Q i G sin(θ i ) is the gravity; R i represents the radius of the vehicle tire; u i represents the throttle or brake control torque input, ι i represents the time constant of the dynamic characteristics; The expression of the leader vehicle model is: Where: p0, v0, and a0 respectively represent the displacement, speed, and acceleration of the leader vehicle, and w0(t) is a bounded function.

3. A heterogeneous vehicle queue control method for collaborative communication and energy consumption optimization according to claim 1, characterized in that The specific implementation steps for the dynamic event-triggering mechanism module to define the local consistency error according to the heterogeneous vehicle formation information and construct the dynamic event-triggering mechanism based on the local consistency error include: First, define the local consistency error according to the heterogeneous vehicle formation as follows: Wherein: represents a switching signal, indicating the connectivity between following vehicle j and i under the switching signal l(t); that is, if following vehicle i can receive the information of following vehicle j, then otherwise, denote Secondly, for the dynamic event triggering mechanism, an internal dynamic variable σ is defined i , and the derivative of the internal dynamic variable is calculated. The corresponding expression is: where: σ i (0) ≥ 0, γ i ∈(0, 1), θ i >0 is a constant and σ i is non - negative; λ min (Q ii ) is the minimum eigenvalue of Q ii , Q ii is the weight matrix in the cost function, L i is a positive constant satisfying , is the triggering error at the l - th time, is an element in the diagonal matrix satisfying is an element of the diagonal matrix satisfying Secondly, since the dynamic event triggering mechanism determines whether the heterogeneous multi-vehicle queue system is triggered based on the changes of internal dynamic variables, then according to the internal dynamic variable σ i , define the (l + 1)-th triggering instant of the follower vehicle i: In the formula: Finally, combined with the dynamic event-triggering mechanism (5), it is determined that the heterogeneous multi-vehicle queue system is triggered when the local consistency error satisfies the following expression, and the corresponding expression is the following formula (6): At the same time, combined with the heterogeneous multi-vehicle queue topology trigger condition, the trigger rule of the heterogeneous multi-vehicle queue system is: Among them, The triggering condition of the heterogeneous multi-vehicle queue topology is that when the topology of the heterogeneous multi-vehicle queue switches, the heterogeneous multi-vehicle queue system will be triggered.

4. A heterogeneous vehicle queue control method for collaborative communication and energy consumption optimization according to claim 1, characterized in that, The specific implementation steps for the judgment network update law module to design a judgment network weight update law according to the local consistency error and the error output in the optimal queue controller module include: First, define the cost function related to the local consistency error, and its expression is: where: Q ii T = Q ii and R ii T = R ii denotes a positive definite matrix, and R ij T = R ij is a positive semi - definite matrix; u j is the control input of the following vehicle j; Secondly, use the judgment network to approximate the cost function, and its expression is: Where: W i The weight of the evaluation network, is the activation function, is the approximation error; and it satisfies and and are positive constants; Secondly, since the weights W of the ideal evaluation network i are unknown, the weights W of the ideal evaluation network are estimated i , and the estimated value of the cost function can be obtained as follows: where: estimated cost function It is used for the policy evaluation step in the integral reinforcement learning algorithm, combined with the policy improvement step, to iteratively solve the optimal cost function; Finally, for the third-order heterogeneous multi-vehicle queue, design the following judgment network weight update law as the output of the judgment network update law module, and its expression is: where: α i is the learning rate, g ∈ {1,..., M i} is the index of the g-th sample data stored in the sample stack ; and it satisfies rank(Η i ) = B < M i , B is the dimension of the evaluation network weights, and the sample stack is the historical data of the activation function; is the residual caused by the approximation error of the evaluation network under the dynamic event-triggered mechanism; Λ i > 0 is the adjustment parameter; is the error generated during the iterative learning process in the optimal queue controller module.

5. A heterogeneous vehicle queue control method for collaborative communication and energy consumption optimization according to claim 1, characterized in that, The specific implementation steps for the optimal queue controller module to use the local consistency error state obtained by differentiating the local consistency error, combine the integral reinforcement learning algorithm and the judgment network weight update law, online learn the optimal cost function and the optimal controller, and at the same time input the error generated in the learning process into the judgment network update law module include: First, by differentiating the expression (3), the following local consistency error dynamics is obtained: In the formula: ξ j :=[0,0,q j T , i ∈ {0} ∪ V;​ Second, for the local consistency error dynamics (12), find the admissible control strategy that minimizes the cost function The expression of the optimal cost function is obtained as follows: Where: A(Ω) is the admissible control set, and Ω is a compact set; Thirdly, by taking the derivative of expression (8) and combining it with expression (12), the following Bellman equation is obtained as follows: Based on the Bellman optimization criterion, the following Hamiltonian function is designed as follows: By solving it can be obtained that: Take the expression (16) as u i Substitute it into the Hamiltonian function to obtain the Hamilton-Jacobi-Bellman equation: Fourthly, combining expression (8) and the additivity of the integration interval, the following integral reinforcement learning Bellman equation is introduced to iteratively solve the Hamilton-Jacobi-Bellman equation (17): Fifthly, by combining the integral reinforcement learning algorithm and the weight update law of the critic network, the following policy evaluation and policy improvement expressions are obtained: Policy evaluation: By combining the weight update law of the critic network (11) and solving the integral reinforcement learning Bellman equation (18), we can obtain Its expression is as follows: Policy improvement: Define the update iteration controller as: where: k is the number of iterations, T is the given time interval; σ i > 0 is a given threshold; the error generated during the iterative learning process and are input into the evaluation network update law module for updating the weights of the evaluation network; And loop to judge whether When it is, it is determined that the evaluation network weights converge and the iteration stops; the cost function at this time Is the optimal cost function, and the iteration controller Is the optimized controller Optimized controller Is the output value of the optimal queue controller module.

Images