A vehicle-road-cloud collaborative decision method and system for heterogeneous mixed traffic flow

Abstract

This invention discloses a vehicle-road-cloud collaborative decision-making method and system for heterogeneous mixed traffic flow. It comprises a three-layer collaborative architecture consisting of an intelligent vehicle system, an intelligent roadside system, and a connected cloud platform, and achieves real-time data interaction through V2X communication. The vehicle end collects real-time vehicle status and road condition data, while the roadside end collects real-time intersection traffic flow and signal status data, interacting with the cloud. The cloud is responsible for global state perception and collaborative scheduling, implementing longitudinal speed optimization and lateral lane-changing decisions based on the ID3QN algorithm and LLM, respectively; and implementing adaptive signal phase control based on D3QN. In longitudinal control, this invention enhances the robustness and generalization of the model in dynamic traffic environments by introducing a parameterized noise mechanism. Lateral decision-making relies on the semantic understanding and reasoning capabilities of a large language model to handle complex lane-changing scenarios, and combines this with signal control. Through shared state space and collaborative optimization objectives, multi-dimensional fusion is achieved, effectively improving traffic efficiency.

Description

Technical Field

[0001] This invention belongs to the field of intelligent traffic control technology, specifically relating to a vehicle-road-cloud collaborative decision-making method and system for heterogeneous mixed traffic flows. Background Technology

[0002] In recent years, urban transportation has developed rapidly, and road traffic volume has increased year by year. As a result, urban traffic congestion has become increasingly serious, resulting in low urban traffic efficiency. The problem is mainly concentrated on the capacity of intersections. Comprehensive control of road traffic flow in both longitudinal and lateral directions based on the traffic conditions at intersections can effectively improve intersection efficiency.

[0003] For traffic efficiency issues arising from the coexistence of connected autonomous vehicles (CAVs) and human-driven vehicles (HDVs) at intersections in heterogeneous mixed traffic environments, traditional longitudinal vehicle control methods are based on reinforcement learning-based signal control, typically using Q-tables. However, Q-tables are limited to solving dynamic programming problems with finite state dimensions. They cannot effectively control longitudinal vehicles in complex traffic signal intelligent control (TSC) problems, resulting in insufficient adaptability and robustness of longitudinal vehicle control under different traffic flow scenarios. Traditional lateral lane-changing strategies often rely on rule-based models, such as the LC2013 model in SUMO. This model uses a decision tree mechanism to comprehensively consider factors such as the vehicle's current speed, relative distance to the vehicle in front, and speed in the target lane to generate lane-changing intentions and execute corresponding controls. However, these methods have limited adaptability and generalization ability in dynamic environments. Summary of the Invention

[0004] To address the problems existing in the above-mentioned background technology, the purpose of this invention is to provide a vehicle-road-cloud collaborative decision-making method and system for heterogeneous mixed traffic flow, which can effectively improve the traffic efficiency of intersections and reduce energy consumption and emissions.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: A vehicle-road-cloud collaborative decision-making method for heterogeneous mixed traffic flow includes the following steps: S1. The vehicle terminal collects real-time vehicle status and road condition data, while the roadside terminal collects real-time traffic flow and signal status data at intersections. The vehicle terminal, roadside terminal, and cloud terminal exchange data via V2X communication. S2. Based on the interactive data, the cloud optimizes the vehicle's acceleration and deceleration behavior, as well as lane changes and path adjustments, to complete the vehicle's longitudinal and lateral control decisions and form a vehicle trajectory optimization plan. The longitudinal control decision for the vehicle is as follows: The cloud-based system optimizes vehicle acceleration and deceleration behavior using the ID3QN algorithm within the built-in vehicle longitudinal decision-making agent, generating optimal action signals. The ID3QN algorithm incorporates a parameterized noise mechanism into the D3QN algorithm. This parameterized noise mechanism is as follows: The original network parameters of the D3QN algorithm Reparameterized to ,in and These are the mean and standard deviation parameters learned by the neural network from the samples, respectively. Let N(0,1) be the random noise independently sampled from the standard normal distribution N(0,1), with the symbol . This represents element-wise multiplication; Update parameters using the Adam optimization algorithm and Bias corrections are applied to the first-order and second-order moment estimates of the Adam optimization algorithm. The vehicle's lateral control decision is as follows: The cloud-based system outputs lateral control actions for vehicles through the Large Model (LLM) in the built-in global decision-making and collaborative scheduling module, enabling lane changes and path adjustments. S3, based on the interactive data, optimizes the signal phase through the D3QN algorithm in the built-in traffic signal control intelligent agent to form a signal control strategy; S4. The vehicle trajectory optimization scheme and signal control strategy generated in the cloud are sent to the vehicle end and the roadside end in real time via V2X communication. The vehicle end performs longitudinal and lateral control of the vehicle according to the vehicle trajectory optimization scheme, and the roadside end controls the traffic signal according to the signal control strategy, thereby building a closed-loop collaborative mechanism between vehicles, roads and cloud.

[0006] Furthermore, in S2, the formulas for bias correction of the first-order moment estimates and second-order moment estimates of the Adam optimization algorithm are as follows: The bias-corrected first-order moment estimate is: ; The bias-corrected second-order moment estimate is: ; in, and These are the first-order moment estimates and second-order moment estimates before correction, respectively. and These are the exponential decay rates for the first-order moment estimate and the second-order moment estimate, respectively.

[0007] Furthermore, parameters and Update using the following formula: ; in, and They are respectively and The parameters at time, For learning rate, is the numerical stability constant.

[0008] Furthermore, in S2, the state, actions, and rewards of the vehicle longitudinal decision-making agent are defined as follows: State space: The state includes the speed of the vehicle and the vehicle in front, the distance to the vehicle in front and the intersection, the current traffic light status and remaining time, and the queue length of the intersection entrance lane. The multi-source heterogeneous data included in the state together constitute the perception basis for the vehicle longitudinal decision-making agent to make multi-objective decisions. Action space: The action is set as possible driving actions such as acceleration, deceleration, and maintaining speed, and the acceleration is adjusted under the constraints of safety mechanisms, with the final speed determined by... Decision, among which , These represent safe speed and road speed limit, respectively. This indicates the recommended speed of the vehicle after receiving the suggested acceleration value from the neural network output; Reward space: The reward function is divided into three sub-rewards: speed reward, signal passing reward, and following safety reward. The overall reward function is composed of a weighted combination of the above three sub-rewards.

[0009] Furthermore, in the state space, the vehicle longitudinal decision-making agent obtains the status of vehicles on the same path in real time through V2V (Vehicle-to-Vehicle) communication, and receives the phase and timing information of the intersection traffic light controller through V2I (Vehicle-to-Infrastructure) communication.

[0010] Furthermore, in the reward space, the speed reward, signal passing reward, and following safety reward are defined as follows: ; ; ; ; ; In the formula, For speed rewards, and These are the vehicle's current speed and the desired cruising speed, respectively. The standard deviation parameter represents the sensitivity of the control reward function to the target velocity. and These represent the signal clearance rewards for green light and non-green light conditions, respectively. Indicates the distance between the vehicle and the signalized intersection. The remaining time of the current green light. This represents the remaining time of the current red light. For buffer time, and These represent the signal period and the green light duration, respectively. The length of the queue of vehicles at the intersection. For parking spacing, For driver reaction time, This is the braking gap time. For the deceleration to increase linearly over time, The speed of the vehicle in front. To accelerate the vehicle, Minimum parking distance, As a reward for safe following, Distance to the vehicle in front; Overall reward function The form is expressed as: ; in, , , These represent the weight parameters for speed reward, signal compliance reward, and following safety reward, respectively, used to balance the relative importance of different sub-rewards in the overall objective function. This applies when the traffic light is green in the traffic scenario. = When the traffic light is not green in a traffic scenario, = .

[0011] Furthermore, in S2, the formula for calculating the Q-value of the main network output in the improved ID3QN algorithm is as follows: ; in, This represents the optimal action value function, used to determine the optimal action signal, and the target network is used to evaluate the target Q value of the action signal. Represents the state value function. Represents the dominance function. This represents the total number of selectable actions in the action space. express Traffic signal status at any given time. express Momentary action signals; loss function for: ; in, This indicates the number of samples drawn from the experience pool. The ordinal number of the sample extracted from the experience pool; The target action value function is used to calculate the Q-value of the target network output. The calculation formula is as follows: ; in, The instantaneous reward at time t. As a discount factor, and They are Traffic signal status and action signals at any given time. and These represent the mean and standard deviation parameters of the noisy linear layer in the target network, respectively. Update the main network parameters using gradient descent algorithm and : , ; Update the target network parameters using a weighted average. and : , ; in, , is a hyperparameter that needs to be adjusted manually.

[0012] This noise injection method maintains a robust estimate of the optimal action value function even under local parameter perturbations, thereby enhancing the robustness and generalization ability of vehicle longitudinal speed control in dynamic traffic environments. This approach not only suppresses overfitting but also improves the algorithm's adaptability to unseen traffic scenarios, providing reliable control performance assurance for reinforcement learning-driven intelligent driving systems.

[0013] Furthermore, in S2, the vehicle lateral control decision-making process includes the following steps: (1) The cloud compiles and analyzes the traffic scene, and integrates multi-source information from vehicle sensors and V2X communication, including vehicle status, surrounding traffic flow characteristics and traffic light status, to form a comprehensive observation vector; (2) Input the integrated observation vector into the large model (LLM) and output the lateral control action, including three types of decisions: maintain the current lane, change lanes to the left and change lanes to the right.

[0014] The overall architecture of vehicle lateral control decision-making follows a closed-loop process of "observation-decision-output," achieving synergistic optimization of traffic efficiency and ecological driving goals while ensuring driving safety.

[0015] This invention further provides a system for implementing the above-mentioned vehicle-road-cloud collaborative decision-making method for heterogeneous mixed traffic flow, comprising: a three-layer collaborative architecture constructed from an intelligent vehicle system, an intelligent roadside system, and a connected cloud platform, wherein the three-layer collaborative architecture achieves real-time data interaction through V2X communication, wherein: Intelligent vehicle systems are used for longitudinal and lateral control of vehicles based on vehicle trajectory optimization schemes, including: The data acquisition module is used to collect data such as vehicle status and road conditions; Wireless communication module, used to realize vehicle-to-cloud and vehicle-to-infrastructure communication; The in-vehicle display module is used to show the driver information about the vehicle trajectory optimization scheme; Intelligent roadside systems are used for traffic environment monitoring and signal optimization, and for traffic signal control based on signal control strategies; including: The signal acquisition module is used to collect data such as traffic flow and signal status at intersections; The road test communication module is used to realize road-cloud and road-vehicle communication; The data processing module is used to process the collected traffic data; Roadside display panels are used to show drivers guidance information, speed limits, etc. The connected cloud platform is responsible for global state perception and collaborative scheduling. It receives dynamic data from the interaction between the intelligent vehicle system and the intelligent roadside system. It has built-in global decision-making and collaborative scheduling modules, vehicle longitudinal decision-making agents, and traffic signal control agents. The vehicle longitudinal decision-making agent is equipped with the ID3QN algorithm to optimize acceleration and deceleration behavior. The ID3QN algorithm is an improved version of the D3QN algorithm, which introduces a parameterized noise mechanism to complete longitudinal control decisions. The global decision-making and collaborative scheduling module is equipped with a large linear model (LLM) to realize lane changes and path adjustments, and complete lateral control decisions. Based on the longitudinal and lateral control decisions, a vehicle trajectory optimization scheme is formed. The traffic signal control agent is equipped with the D3QN algorithm to generate signal control strategies. The vehicle trajectory optimization scheme and signal control strategy are sent to the intelligent vehicle system and the intelligent roadside system in real time via V2X communication.

[0016] Compared with the shortcomings and deficiencies of existing technologies, the present invention has the following beneficial effects: 1. This invention proposes a hybrid decision-making method based on deep reinforcement learning and a large language model. At the vertical control level, an improved D3QN algorithm with a parameterized noise mechanism is introduced to enhance the algorithm model's adaptability to dynamic traffic environments. At the horizontal decision-making level, the semantic understanding capability of the large language model is utilized to handle complex lane-changing scenarios. The two decisions achieve information fusion through sharing a state space and co-optimizing objectives, constructing a technical path that simultaneously handles vertical trajectory optimization and horizontal behavioral decision-making. 2. In the context of noisy networks, this invention utilizes the Adam optimization algorithm to update parameters. and Furthermore, bias corrections are applied to the first and second moment estimates of the Adam optimization algorithm. On one hand, the coupling of gradient noise with the algorithm's adaptive normalization mechanism effectively mitigates gradient variance fluctuations caused by noise sampling, thereby improving training stability. On the other hand, the momentum term retains some historical gradient directions, suppressing parameter update oscillations caused by noise. and It can systematically converge to a robust optimal solution. Furthermore, from a generalization perspective, this optimization process implicitly approximates the posterior distribution of parameters and establishes a dynamic balance between exploration and exploitation by adaptively adjusting the learning rate, thereby enhancing the model's generalization ability. 3. The vehicle-road-cloud collaborative decision-making system for heterogeneous mixed traffic flow proposed in this invention uses V2X communication as a link to organically integrate the fine-grained control of the vehicle end, the signal optimization of the roadside end, and the global coordination of the cloud end. While ensuring traffic safety, it provides a systematic solution for improving traffic efficiency and ecological benefits. Attached Figure Description

[0017] Figure 1 This is an architecture diagram of a vehicle-road-cloud collaborative decision-making system for heterogeneous mixed traffic flow provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of the traffic signal control intelligent agent provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of the vehicle longitudinal decision-making intelligent agent provided in an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0019] This invention proposes a vehicle-road-cloud collaborative decision-making method and system for heterogeneous mixed traffic flows, such as... Figure 1 As shown, the overall decision-making system is a three-layer collaborative architecture built by the intelligent vehicle system, the intelligent roadside system, and the connected cloud platform. The three layers of collaborative architecture achieve real-time data interaction through V2X communication.

[0020] The intelligent vehicle system is used for longitudinal and lateral control of the vehicle based on a vehicle trajectory optimization scheme. The vehicle-side system integrates a data acquisition module, a wireless communication module, and an in-vehicle display module, used for real-time status perception, external information interaction, and human-machine interface feedback, respectively. Among these: The data acquisition module is used to collect data such as vehicle status and road conditions; Wireless communication module, used to realize vehicle-to-cloud and vehicle-to-infrastructure communication; The in-vehicle display module is used to show the driver information about the vehicle trajectory optimization scheme.

[0021] The intelligent roadside system is used for traffic environment monitoring and signal optimization, controlling traffic signals according to signal control strategies. The roadside unit integrates a signal acquisition module, a roadside communication module, a data processing module, and a roadside display panel. The signal acquisition and data processing modules enable real-time monitoring of traffic conditions, the roadside communication module ensures information exchange between vehicles, the road, and the cloud, and the roadside display panel provides intuitive traffic guidance for road users. Specifically: The signal acquisition module is used to collect data such as traffic flow and signal status at intersections; The road test communication module is used to realize road-cloud and road-vehicle communication; The data processing module is used to process the collected traffic data; Roadside display panels are used to show drivers guidance information, speed limits, etc.

[0022] The connected cloud platform is responsible for global state perception and collaborative scheduling. It receives dynamic data from the interaction between the intelligent vehicle system and the intelligent roadside system. It has built-in global decision-making and collaborative scheduling modules, a vehicle longitudinal decision-making agent, and a traffic signal control agent. The vehicle longitudinal decision-making agent uses a pre-set improved D3QN algorithm (ID3QN algorithm) to optimize acceleration and deceleration behavior and complete longitudinal control decisions. The global decision-making and collaborative scheduling module uses a pre-set large model (LLM) to realize lane changing and path adjustment and complete lateral control decisions. Based on the longitudinal and lateral control decisions, a vehicle trajectory optimization scheme is formed. The traffic signal control agent uses a pre-set D3QN algorithm to generate signal control strategies. The vehicle trajectory optimization scheme and signal control strategy are sent to the intelligent vehicle system and the intelligent roadside system in real time via V2X communication, respectively, to build a closed-loop collaborative mechanism between vehicles, roads, and the cloud. The decision-making method is explained in detail below.

[0023] 1. Three-tier architecture for data interaction The vehicle terminal collects real-time vehicle status and road condition data, while the roadside terminal collects real-time traffic flow and signal status data at intersections. The status data collected by the vehicle terminal and the roadside terminal are exchanged with the cloud via V2X communication.

[0024] 2. Generate comprehensive scheduling results in the cloud. After receiving dynamic data from the vehicle and roadside terminals, a comprehensive scheduling result is generated based on the set decision principles and optimization algorithms. The resulting vehicle trajectory optimization scheme and signal control strategy are then sent to the vehicle and roadside terminals in real time via V2X communication, thus establishing a closed-loop collaborative mechanism between vehicles, roads, and the cloud.

[0025] After receiving real-time dynamic data from vehicles and roadside terminals, the connected cloud platform utilizes the high-order reasoning capabilities of Large Language Models (LLM) for macro-level situational analysis and generates comprehensive dispatch instructions based on global optimization principles. The LLM, deployed in the global decision-making and collaborative dispatch module of the connected cloud platform, is responsible for understanding multi-source traffic information, reasoning about operational situations, and coordinating cross-terminal control strategies. Under this mechanism, the traffic signal control agent and the vehicle longitudinal decision-making agent within the connected cloud platform serve as the execution cores for the roadside and vehicle terminals, respectively: the former achieves adaptive optimization of signal timing through the D3QN algorithm deployed in the traffic signal control agent, while the latter utilizes the ID3QN algorithm embedded in the vehicle longitudinal decision-making agent to complete refined vehicle trajectory planning. The resulting signal control strategy and trajectory optimization scheme are then distributed in real-time via V2X communication, achieving closed-loop coordination between cloud-based coordination, roadside response, and vehicle-side execution, significantly improving the overall traffic efficiency and safety of the road network.

[0026] 2.1 Traffic Signal Control 2.1.1 Traffic Signal Control Intelligent Agent Traffic signal control agent every time The action is selected during the rolling update phase to ensure real-time control.

[0027] The states, actions, and rewards of the traffic signal control agent are defined as follows: Status: Entrance lane The state can be measured using traffic flow density. Described as: ; in, Indicates the entrance lane Traffic flow vehicle number Indicates the length of the vehicle. This indicates the lane length. All entrances to the intersection are treated in the same way. Therefore, the intersection... The state at a given time can be represented as follows, serving as the input to the signal agent: ; Actions: The traffic signal control agent employs a four-phase control mode, including "North-South Straight," "East-West Straight," "North-South Left Turn," and "East-West Left Turn." Strategy selection signal phase: ; in, The value represents the state. Take action below And the expected cumulative return that can be obtained by continuing to execute the strategy; Reward: The primary objective of the traffic signal control agent is to improve the overall efficiency of the intersection by prioritizing lanes with higher traffic volume. Therefore, the reward function is set as follows: ; in, The lane with the highest vehicle density represents the theoretically optimal action that the traffic signal control agent should take at the current moment. This sparse binary reward structure provides clear and immediate feedback signals to the agent through an explicit reward and punishment mechanism: the agent receives a positive reward when it correctly identifies and allows the lane with the longest queue of vehicles to proceed, and is penalized when it makes an incorrect choice. This reward formula has significant advantages in simplicity and computational efficiency, simplifying the complex optimal signal control problem into an intuitive supervised learning task, directly aligning the learning objective with the key operational goal of improving intersection throughput.

[0028] 2.1.2 D3QN Algorithm The traffic signal control agent uses the D3QN algorithm to optimize signal phase and form a signal control strategy, the structure of which is as follows: Figure 2 As shown, the D3QN algorithm uses the optimal action value function. Decomposed into state value function and dominance function This allows for a more accurate estimation of the value of each action signal. The Q-value of the main network output is then calculated using the following formula: ; in, The parameters of the neural network, This represents the total number of selectable actions in the action space. To guide the updating of the main network's parameters, a target Q-function is defined, representing the number of actions available in the traffic state. Take action signal below The target Q-value is then determined. At this point, the loss function can be defined as follows: ; in, This indicates the number of samples drawn from the experience pool. This represents the sample ordinal number extracted from the experience pool. The D3QN algorithm uses a main network to determine the optimal action signal, while simultaneously using a target network to evaluate the target Q-value of that action signal. The target network has the same structure as the main network, but its parameters... They are different. The Q-value of the target network output can be calculated using the following equation: ; in, The parameters representing the target network, and These represent the signal state and action at the next time step, respectively. The gradient descent algorithm can use the loss value to update the parameters of the main network. ; in, The learning rate; And the parameters of the target network are updated by weighted averaging. : ; It is a hyperparameter that needs to be adjusted manually.

[0029] 2.2 Vehicle trajectory optimization 2.2.1 Vehicle longitudinal decision-making 2.2.1.1 Vehicle longitudinal decision-making intelligent agent The vehicle longitudinal decision-making agent is a multi-objective decision problem that requires optimizing key performance indicators such as safety, compliance, and economy in a high-dimensional, continuous, and uncertain state space.

[0030] The state, action, and reward of the vehicle longitudinal decision-making agent are defined as follows: State space: The state includes the speed of the vehicle and the vehicle in front, the distance to the vehicle in front and the intersection, the current traffic light status and remaining time, and the queue length at the intersection entrance lanes. The vehicle longitudinal decision-making agent obtains the status of vehicles on the same path in real time through V2V (Vehicle-to-Vehicle) communication and receives phase and timing information from the intersection traffic light controller through V2I (Vehicle-to-Infrastructure) communication. This multi-source heterogeneous data collectively constitutes the perceptual basis for the vehicle longitudinal decision-making agent to make multi-objective decisions. Action space: The action is set as possible driving actions such as acceleration, deceleration, and maintaining speed, and the acceleration is adjusted under the constraints of safety mechanisms, with the final speed determined by... Decision, among which , These represent safe speed and road speed limit, respectively. This indicates the recommended speed of the vehicle after receiving the suggested acceleration value from the neural network output; Reward Space: The reward function is divided into three sub-rewards: speed reward, signal compliance reward, and following safety reward. The overall reward function is constructed by a weighted combination of these three sub-rewards. The definitions of speed reward, signal compliance reward, and following safety reward are as follows: ; ; ; ; ; In the formula, For speed rewards, and These are the vehicle's current speed and the desired cruising speed, respectively. The standard deviation parameter represents the sensitivity of the control reward function to the target velocity. and These represent the signal clearance rewards for green light and non-green light conditions, respectively. Indicates the distance between the vehicle and the signalized intersection. The remaining time of the current green light. This represents the remaining time of the current red light. For buffer time, and These represent the signal period and the green light duration, respectively. The length of the queue of vehicles at the intersection. For parking spacing, For driver reaction time, This is the braking gap time. For the deceleration to increase linearly over time, The speed of the vehicle in front. To accelerate the vehicle, Minimum parking distance, As a reward for safe following, Distance to the vehicle in front; Overall reward function The form is expressed as: ; in, , , These represent the weight parameters for speed reward, signal compliance reward, and following safety reward, respectively, used to balance the relative importance of different sub-rewards in the overall objective function. When the traffic light is green in the traffic scenario... = When the traffic light is not green in a traffic scenario, = .

[0031] 2.2.1.2 ID3QN Algorithm To enhance the adaptability and robustness of vehicle longitudinal control under different traffic flow scenarios, this invention introduces a parameterized noise mechanism into the fully connected network of the D3QN algorithm of the vehicle longitudinal decision-making agent, forming the ID3QN algorithm, as follows: Figure 3As shown, the original network parameters of the D3QN algorithm are... Reparameterized to ,in and These are the mean and standard deviation parameters learned by the neural network from the samples, respectively. Let N(0,1) be the random noise independently sampled from the standard normal distribution N(0,1), with the symbol . This represents element-wise multiplication. Correspondingly, the formula for calculating the Q-value of the main network output is: ; in, This represents the optimal action value function, used to determine the optimal action signal, and the target network is used to evaluate the target Q value of the action signal. Represents the state value function. Represents the dominance function. express Traffic signal status at any given time. express Momentary action signals; loss function for: ; in, This indicates the number of samples drawn from the experience pool. The ordinal number of the sample extracted from the experience pool; The target action value function is used to calculate the Q-value of the target network output. The calculation formula is as follows: ; in, The instantaneous reward at time t. As a discount factor, and They are Traffic signal status and action signals at any given time. and These are all parameters in the target network, representing the mean and standard deviation parameters of the noisy linear layer in the target network, respectively. Update the main network parameters using gradient descent algorithm and : , ; Update the target network parameters using a weighted average. and : , ; in, , is a hyperparameter that needs to be adjusted manually.

[0032] This noise injection method maintains a robust estimate of the optimal action value function even under local parameter perturbations, thereby enhancing the robustness and generalization ability of vehicle longitudinal speed control in dynamic traffic environments. This approach not only suppresses overfitting but also improves the algorithm's adaptability to unseen traffic scenarios, providing reliable control performance assurance for reinforcement learning-driven intelligent driving systems.

[0033] In addition, the Adam optimization algorithm is used to update the parameters. and Combining the advantages of the Adaptive Gradient Descent (AdaGrad) algorithm and the root mean square propagation (RMSProp) kinetic mechanism, the learning rate of each parameter can be adaptively adjusted through the exponentially weighted moving average of the first and second moments of the gradient. Specifically, in deep reinforcement learning algorithms, both the main network and the target network are backpropagation neural networks (BPNNs) with identical structures. Therefore, gradient descent based on the loss function can be calculated using the following formula: ; The first-order moment estimate of the Adam optimization algorithm is: ; The second moment estimate is: ; Bias corrections are applied to the first-order and second-order moment estimates of the Adam optimization algorithm: The bias-corrected first-order moment estimate is: ; The bias-corrected second-order moment estimate is: ; in, and These are the first-order moment estimates and second-order moment estimates before correction, respectively. and These are the exponential decay rates for the first-order moment estimate and the second-order moment estimate, respectively.

[0034] parameter and Then update using the following formula: ; in, and They are respectively and The parameters at time, For learning rate, is the numerical stability constant.

[0035] 2.2.2 Vehicle Lateral Decision Making The vehicle outputs lateral control actions through a large model (LLM) to achieve lane changes and path adjustments. The vehicle's lateral control decision-making process includes the following steps: (1) Compile and parse the traffic scene, and integrate multi-source information from vehicle sensors and V2X communication, including vehicle status, surrounding traffic flow characteristics and traffic light status, to form a comprehensive observation vector; (2) Input the integrated observation vector into the large model (LLM) and output the lateral control action, including three types of decisions: maintain the current lane, change lanes to the left and change lanes to the right.

[0036] The overall architecture of vehicle lateral control decision-making follows a closed-loop process of "observation-decision-output," achieving synergistic optimization of traffic efficiency and ecological driving goals while ensuring driving safety.

[0037] 3. Optimized vehicle dispatching The vehicle trajectory optimization scheme and signal control strategy generated in the cloud are sent to the vehicle end and the roadside end in real time via V2X communication. The vehicle end performs longitudinal and lateral control of the vehicle according to the vehicle trajectory optimization scheme, and the roadside end controls the traffic signal according to the signal control strategy, thereby building a closed-loop collaborative mechanism between vehicles, roads and cloud.

[0038] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A vehicle-road-cloud collaborative decision-making method for heterogeneous mixed traffic flow, characterized in that, Includes the following steps: S1. The vehicle terminal collects real-time vehicle status and road condition data, while the roadside terminal collects real-time traffic flow and signal status data at intersections. The vehicle terminal, roadside terminal, and cloud terminal exchange data via V2X communication. S2. Based on the interactive data, the cloud optimizes the vehicle's acceleration and deceleration behavior, as well as lane changes and path adjustments, to complete the vehicle's longitudinal and lateral control decisions and form a vehicle trajectory optimization plan. The longitudinal control decision for the vehicle is as follows: The cloud-based system optimizes vehicle acceleration and deceleration behavior using the ID3QN algorithm within the built-in vehicle longitudinal decision-making agent, generating optimal action signals. The ID3QN algorithm incorporates a parameterized noise mechanism into the D3QN algorithm. This parameterized noise mechanism is as follows: The original network parameters of the D3QN algorithm Reparameterized to ,in and These are the mean and standard deviation parameters learned by the neural network from the samples, respectively. Let N(0,1) be the random noise independently sampled from the standard normal distribution N(0,1), with the symbol . This represents element-wise multiplication; Update parameters using the Adam optimization algorithm and Bias corrections are applied to the first-order and second-order moment estimates of the Adam optimization algorithm. The vehicle's lateral control decision is as follows: The cloud-based system outputs lateral control actions for vehicles through the built-in global decision-making and collaborative scheduling module's LLM, enabling lane changes and path adjustments. S3, based on the interactive data, optimizes the signal phase through the D3QN algorithm in the built-in traffic signal control intelligent agent to form a signal control strategy; S4. The vehicle trajectory optimization plan and signal control strategy generated in the cloud are sent to the vehicle end and the roadside end in real time via V2X communication. The vehicle end performs longitudinal and lateral control of the vehicle according to the vehicle trajectory optimization plan, and the roadside end controls the traffic signal according to the signal control strategy.

2. The vehicle-road-cloud collaborative decision-making method for heterogeneous mixed traffic flow as described in claim 1, characterized in that, In S2, the formulas for bias correction of the first-order moment estimates and second-order moment estimates of the Adam optimization algorithm are as follows: The bias-corrected first-order moment estimate is: ; The bias-corrected second-order moment estimate is: ; in, and These are the first-order moment estimates and second-order moment estimates before correction, respectively. and These are the exponential decay rates for the first-order moment estimate and the second-order moment estimate, respectively.

3. The vehicle-road-cloud collaborative decision-making method for heterogeneous mixed traffic flow as described in claim 2, characterized in that, The parameters and Update using the following formula: ； in, and They are respectively and The parameters at time, For learning rate, is the numerical stability constant.

4. The vehicle-road-cloud collaborative decision-making method for heterogeneous mixed traffic flow as described in claim 1, characterized in that, In S2, the state, action, and reward of the vehicle longitudinal decision-making agent are defined as follows: State space: The state includes the speed of the vehicle and the vehicle in front, the distance to the vehicle in front and the intersection, the current traffic light status and remaining time, and the queue length of the intersection entrance lane. The multi-source heterogeneous data included in the state together constitute the perception basis for the vehicle longitudinal decision-making agent to make multi-objective decisions. Action space: The action is set as possible driving actions such as acceleration, deceleration, and maintaining speed, and the acceleration is adjusted under the constraints of safety mechanisms, with the final speed determined by... Decision, among which , These represent safe speed and road speed limit, respectively. This indicates the recommended speed of the vehicle after receiving the suggested acceleration value from the neural network output; Reward space: The reward function is divided into three sub-rewards: speed reward, signal passing reward, and following safety reward. The overall reward function is composed of a weighted combination of the above three sub-rewards.

5. The vehicle-road-cloud collaborative decision-making method for heterogeneous mixed traffic flow as described in claim 4, characterized in that, In the state space, the vehicle longitudinal decision-making agent obtains the status of vehicles on the same path in real time through V2V communication and receives phase and timing information from the traffic light controller at the intersection through V2I communication.

6. The vehicle-road-cloud collaborative decision-making method for heterogeneous mixed traffic flow as described in claim 4, characterized in that, In the aforementioned reward space, speed reward, signal clearance reward, and following safety reward are defined as follows: ； ； ； ； ； In the formula, For speed rewards, and These are the vehicle's current speed and the desired cruising speed, respectively. The standard deviation parameter represents the sensitivity of the control reward function to the target velocity. and These represent the signal clearance rewards for green light and non-green light conditions, respectively. Indicates the distance between the vehicle and the signalized intersection. The remaining time of the current green light. This represents the remaining time of the current red light. For buffer time, and These represent the signal period and the green light duration, respectively. The length of the queue of vehicles at the intersection. For parking spacing, For driver reaction time, This is the braking gap time. For the deceleration to increase linearly over time, The speed of the vehicle in front. To accelerate the vehicle, Minimum parking distance, As a reward for safe following, Distance to the vehicle in front; Overall reward function The form is expressed as: ； in, , , These represent the weight parameters for speed reward, signal compliance reward, and following safety reward, respectively, used to balance the relative importance of different sub-rewards in the overall objective function. This applies when the traffic light is green in the traffic scenario. = When the traffic light is not green in a traffic scenario, = .

7. The vehicle-road-cloud collaborative decision-making method for heterogeneous mixed traffic flow as described in claim 1, characterized in that, In S2, the formula for calculating the Q-value of the main network output in the improved ID3QN algorithm is as follows: ； in, This represents the optimal action value function, used to determine the optimal action signal, and the target network is used to evaluate the target Q value of the action signal. Represents the state value function. Represents the dominance function. This represents the total number of selectable actions in the action space. express Traffic signal status at any given time. express Momentary action signals; loss function for: ； in, This indicates the number of samples drawn from the experience pool. The ordinal number of the sample extracted from the experience pool; The target action value function is used to calculate the Q-value of the target network output. The calculation formula is as follows: ； in, The instantaneous reward at time t. As a discount factor, and They are Traffic signal status and action signals at any given time. and These represent the mean and standard deviation parameters of the noisy linear layer in the target network, respectively. Update the main network parameters using gradient descent algorithm and : , ; Update the target network parameters using a weighted average. and : , ; in, , is a hyperparameter that is manually adjusted.

8. The vehicle-road-cloud collaborative decision-making method for heterogeneous mixed traffic flow as described in claim 1, characterized in that, In S2, the vehicle lateral control decision-making process includes the following steps: (1) The cloud compiles and analyzes the traffic scene, integrates multi-source information from vehicle sensors and V2X communication, including vehicle status, surrounding traffic flow characteristics and traffic light status, to form a comprehensive observation vector; (2) Input the integrated observation vector into the LLM and output the lateral control action, including three types of decisions: maintain the current lane, change lanes to the left and change lanes to the right.

9. A system for implementing the vehicle-road-cloud collaborative decision-making method for heterogeneous mixed traffic flow as described in any one of claims 1-8, characterized in that, This includes a three-layer collaborative architecture comprised of an intelligent vehicle system, an intelligent roadside system, and a connected cloud platform. These three layers interact with each other in real time via V2X communication. Intelligent vehicle systems are used for longitudinal and lateral control of vehicles based on vehicle trajectory optimization schemes, including: The data acquisition module is used to collect data such as vehicle status and road conditions; Wireless communication module, used to realize vehicle-to-cloud and vehicle-to-infrastructure communication; The in-vehicle display module is used to show the driver information about the vehicle trajectory optimization scheme; Intelligent roadside systems are used for traffic environment monitoring and signal optimization, and for traffic signal control based on signal control strategies; including: The signal acquisition module is used to collect data such as traffic flow and signal status at intersections; The road test communication module is used to realize road-cloud and road-vehicle communication; The data processing module is used to process the collected traffic data; Roadside display panels are used to show drivers guidance information, speed limits, etc. The connected cloud platform is responsible for global state perception and collaborative scheduling. It receives dynamic data from the interaction between the intelligent vehicle system and the intelligent roadside system. It has built-in global decision-making and collaborative scheduling modules, a vehicle longitudinal decision-making agent, and a traffic signal control agent. The vehicle longitudinal decision-making agent is equipped with the ID3QN algorithm, which is an improved version of the D3QN algorithm. The D3QN algorithm introduces a parameterized noise mechanism to complete the longitudinal control decision. The global decision-making and collaborative scheduling module is equipped with LLM to realize lane changing and path adjustment, and complete the lateral control decision. Based on the longitudinal and lateral control decisions, it forms a vehicle trajectory optimization scheme. The traffic signal control agent is equipped with the D3QN algorithm to generate signal control strategies. The vehicle trajectory optimization scheme and the signal control strategy are sent to the intelligent vehicle system and the intelligent roadside system in real time via V2X communication.

Images