Single vehicle control method based on scene style transfer and sac network cooperation
By employing a single-vehicle control method that combines scene style transfer and SAC network collaboration, along with multi-agent communication and transfer learning, the problems of traffic stop-start waves and fuel consumption are solved, achieving optimization of traffic safety and fuel consumption, and improving the robustness of the control strategy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SOUTHEAST UNIV
- Filing Date
- 2023-07-13
- Publication Date
- 2026-07-21
AI Technical Summary
Existing deep reinforcement learning methods are ineffective in suppressing stop-and-go traffic waves in traffic control and fail to simultaneously optimize traffic safety and reduce fuel consumption.
A single-vehicle control method based on scene style transfer and SAC network collaboration is adopted. By constructing a single-vehicle longitudinal control model, combining multi-agent communication, designing a multi-objective reward function, and utilizing transfer learning to improve the robustness of the control strategy under different weather conditions.
It effectively suppresses stop-and-go traffic, optimizes traffic safety, reduces fuel consumption, and improves the robustness of control strategies under different weather conditions.
Smart Images
Figure CN116893681B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of deep reinforcement learning and transfer learning, and in particular to a single-vehicle control method based on scene style transfer and SAC network collaboration. Background Technology
[0002] Deep reinforcement learning plays a crucial role in the field of autonomous driving. By using deep neural networks and reinforcement learning algorithms, autonomous vehicles can learn to perceive their environment, make decisions, and control their movement. Some research focuses on using deep reinforcement learning to build traffic flow models. These models can simulate the dynamic characteristics of traffic flow and are used to predict and plan traffic signal control strategies. However, this approach needs to take into account the complexity and nonlinearity of traffic flow, as well as the influence of various factors on traffic flow, such as vehicle density, vehicle speed, and intersection congestion.
[0003] Transfer learning research in transportation using deep neural networks has also attracted attention. By fine-tuning or adjusting pre-trained neural networks, feature representations and model weights learned in the source domain can be applied to tasks in the target domain. This can accelerate model convergence and improve performance. Applications of transfer learning in the transportation field include traffic flow prediction, traffic congestion control, and driving behavior analysis. By leveraging traffic data and knowledge from other cities or regions, it is possible to better address data shortages or differences in data distribution in the target domain.
[0004] Some studies use reinforcement learning to design reward functions to punish unsafe driving behaviors, but these methods cannot guarantee safety in the early stages of training. Summary of the Invention
[0005] Purpose of the invention: The purpose of this invention is to provide a single-vehicle control method based on scene style transfer and SAC network collaboration that can effectively suppress traffic stop-and-go waves, optimize traffic safety, and reduce fuel consumption.
[0006] Technical solution: The present invention provides a single-vehicle control method based on scene style transfer and SAC network collaboration, comprising the following steps:
[0007] S1 uses the SUMO tool to build a simulation platform, configures road files, vehicle routing files and simulation configuration files; simulates the target vehicle to perceive information about the surrounding environment through V2X communication technology and on-board sensors.
[0008] S2, Construct a single-vehicle longitudinal control model, use an SAC network to obtain traffic status from the simulation platform, and then take actions to control the vehicle's behavior. After the action is executed, the agent receives the reward value fed back from the simulation platform to update the agent parameters. The parameters of the single-vehicle longitudinal control model include the state space, action space and target reward function of the intelligent connected vehicle.
[0009] S3 defines a policy network that outputs numerical acceleration; and a value function network whose output layer represents an estimate of the state values.
[0010] S4 uses the vehicle control experimental platform to call SUMO to simulate different weather conditions and constructs a robustness index to measure the model's ability to suppress stop-and-go traffic waves.
[0011] S5, a transfer learning framework is built on the basis of the single vehicle longitudinal control model;
[0012] S6 utilizes transfer learning to train a converged single-vehicle longitudinal control model under different weather conditions.
[0013] Furthermore, in step S2, the target vehicle, based on its own information, obtains information about the two preceding vehicles. The vehicle's state space is then represented as follows:
[0014] ,
[0015] in, It is the distance between the target vehicle and the second vehicle in front of it. That's the speed of the second car in front. It is the acceleration of the second vehicle in front; It is the distance between the target vehicle and the first vehicle in front of it. That's the speed of the first car in front. It is the acceleration of the first car in front; It is the speed of the controlled target vehicle. It is the acceleration of the target vehicle; the first vehicle in front is closer to the target vehicle than the second vehicle in front.
[0016] Furthermore, in step S2, safety constraints are introduced to avoid rear-end collisions, and the vehicle's motion space is defined as follows:
[0017] The maximum safe acceleration is designed as follows:
[0018] ,
[0019] ,
[0020] in, This refers to the minimum safe distance that a vehicle needs to maintain from the vehicle in front while it is currently driving. This refers to the maximum safe acceleration of the vehicle under its current driving conditions; It is the length of the vehicle in front of the target vehicle. The target vehicle With the car in front relative velocity, The target vehicle n and the vehicle in front The difference in acceleration, It is the driver's reaction time;
[0021] The maximum safe deceleration design is as follows:
[0022] ,
[0023] MADR stands for collision avoidance deceleration rate, which is the maximum rate at which a vehicle can decelerate while avoiding a collision with an obstacle in front of it. This refers to the maximum safe deceleration of a vehicle under its current driving conditions;
[0024] Then the acceleration of the target vehicle at any time satisfy:
[0025] .
[0026] Furthermore, in step S2, the target reward function for intelligent connected vehicles is defined as follows:
[0027] The primary objective is to suppress the propagation of stop-and-go waves by maintaining a safe and feasible headway; a threshold for following distance is set. If the distance is set to 120 meters, the expression for the penalty value is as follows:
[0028] ,
[0029] in, The target vehicle n is at time t and is in contact with the vehicle in front. Real-time distance between them;
[0030] The second objective is traffic efficiency; vehicles will receive a greater reward value the closer they travel to the expected speed.
[0031] ,
[0032] in, It is the vehicle's desired driving speed. It is the real-time driving speed of the target vehicle n at time t;
[0033] The third goal is to reduce fuel consumption, which includes two parts:
[0034] The first part of the reward function is defined as the negative of the square of the target vehicle's acceleration; the larger the acceleration value, the smaller the reward.
[0035] ,
[0036] in, It is the real-time acceleration of the target vehicle n at time t;
[0037] The second part of the reward function is defined as the negative value of the fuel consumption of intelligent connected vehicles; the more fuel consumed, the smaller the reward value will be.
[0038] ,
[0039] in, This represents the amount of fuel consumed by the vehicle at time t.
[0040] Therefore, the complete reward function is shown in the following equation:
[0041] ,
[0042] in, , and These are the weights of the different components of the reward function.
[0043] Furthermore, in step S3, a policy network with one hidden layer is defined, and the output layer uses the tanh activation function to output the numerical value of acceleration; a value function network with a single hidden layer is defined, and the hidden layer uses ReLU as the activation function, and the output layer is a single node representing the estimate of the state value.
[0044] Furthermore, in step S4, the measurement model includes the speed rolling mean and standard deviation;
[0045] Speed rolling standard deviation The value of varies with time and is calculated at each time step k, as shown in the following formula:
[0046] ,
[0047] ,
[0048] Where T is the length of the scrolling time window. Let T be the average velocity starting from the k-th time step. Let be the velocity at the i-th time step. Let T be the rolling standard deviation of the velocity starting at the k-th time step; to measure the overall local velocity change throughout the entire evaluation period, take all rolling standard deviations during this period. The average value is defined as follows:
[0049] ;
[0050] Solving for deceleration rate variation based on Mexican hat wavelet transform. Acceleration rate change and oscillating growth :
[0051] ,
[0052] ,
[0053] ,
[0054] Where N is the number of vehicles in the traffic flow; It is the average deceleration of the vehicle at position j in the traffic flow minus the average deceleration of the vehicle at position j-1. The two average decelerations are calculated based on the start and end times of deceleration of the preceding and following vehicles, respectively. It is the average acceleration of the vehicle at position j in the traffic flow minus the average acceleration of the vehicle at position j-1. The two average accelerations are calculated based on the acceleration start time and acceleration end time of the preceding and following vehicles, respectively. The minimum speed of the vehicle at position j-1 in the traffic flow is minus the minimum speed of the vehicle at position j within a driving cycle.
[0055] Furthermore, in step S5, in the neural network structure of the target scene, each neuron n in the target scene neural network is mapped to each neuron in the source scene neural network through a mapping relationship. The mapping function is expressed as:
[0056] ,
[0057] in, This means mapping the state variables in the target scene to the most similar state variables in the source scene; This means mapping actions in the target scene to the most similar actions in the source scene.
[0058] Compared with the prior art, the significant advantages of this invention are as follows:
[0059] 1. For a single-vehicle longitudinal control model based on deep reinforcement learning, the SAC algorithm is used to combine the information of two downstream vehicles into the state space and apply safety constraints to the action space to ensure traffic safety. The design of the multi-objective reward function weighs the weights of different aspects such as following distance, safety, efficiency, and fuel consumption, further optimizing the longitudinal control performance of the single-vehicle longitudinal control model. It can effectively suppress the propagation of traffic stop-and-go waves and reduce fuel consumption, which has certain practical significance.
[0060] 2. This invention utilizes communication and cooperation among multiple agents to analyze the impact of different agent design parameters and reinforcement learning training parameters on the sensitivity of the algorithm, thereby further improving the performance of the agent system. It combines the SAC algorithm with a parameter sharing structure to enhance cooperation among multiple agents, reduce the computational complexity of the algorithm, and thus achieve the goal of effectively suppressing traffic stop-and-go waves, optimizing traffic safety, and reducing fuel consumption.
[0061] 3. Based on deep reinforcement learning, a single-vehicle longitudinal control model is used in long traffic stop-and-go wave scenarios under different weather conditions. A robustness index for suppressing the effect of traffic stop-and-go waves is constructed, and transfer learning is used to improve the control strategy's rapid learning ability and robustness in new scenarios. Attached Figure Description
[0062] Figure 1 This is the overall flowchart of the present invention;
[0063] Figure 2 This is a schematic diagram of a driving strategy framework based on the SAC algorithm.
[0064] Figure 3 This is a schematic diagram of the control strategy transfer process based on transfer learning. Detailed Implementation
[0065] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.
[0066] like Figure 1 The diagram shown is the overall flowchart of the present invention. The single-vehicle control method based on scene style transfer and SAC network collaboration of the present invention includes the following steps:
[0067] Step one: Build a simulation platform using SUMO, configuring road files, vehicle routing files, and simulation configuration files. Simulate the target vehicle perceiving information about its surrounding environment through V2X communication technology and onboard sensors.
[0068] Step two: Construct a single-vehicle longitudinal control model. Use a SAC (Soft Actor Critic) network to obtain traffic conditions from the simulation platform, then take actions to control the vehicle's behavior. After the action is executed, the agent receives reward values from the simulation platform to update its parameters. Figure 2 As shown, the intelligent agent, acting as the vehicle's control system, is responsible for managing and executing the vehicle's longitudinal motion. The definition of the state space, the representation of the action space, and the setting of the reward function are three key elements of the single-vehicle longitudinal control model based on SAC (System-Agent Control).
[0069] 21) Define the state space of intelligent connected vehicles.
[0070] The state space of an intelligent connected vehicle is represented as follows: (This is based on the target vehicle acquiring its own information and then information from the two preceding vehicles.)
[0071] (1)
[0072] in, It is the distance between the target vehicle and the second vehicle in front of it. That's the speed of the second car in front. It is the acceleration of the second vehicle in front; It is the distance between the target vehicle and the first vehicle in front of it. That's the speed of the first car in front. It is the acceleration of the first car in front; It is the speed of the controlled target vehicle. It is the acceleration of the target vehicle; the first vehicle in front is closer to the target vehicle than the second vehicle in front.
[0073] 22) Define the motion space of intelligent connected vehicles
[0074] This invention introduces safety constraints to limit vehicles from making overly aggressive maneuvers, thereby avoiding rear-end collisions and further improving driving safety.
[0075] 221) Maximum safe acceleration
[0076] To enable the agent to learn a good strategy, maximum safe acceleration and maximum safe deceleration are introduced to avoid traffic accidents.
[0077] The maximum safe acceleration is designed as follows:
[0078] (2)
[0079] (3)
[0080] in, This refers to the minimum safe distance a vehicle must maintain from the vehicle in front while it is currently driving. This refers to the maximum safe acceleration a vehicle can achieve under its current driving conditions. It is the length of the vehicle in front of the target vehicle. The target vehicle With the car in front relative velocity, The target vehicle n and the vehicle in front The difference in acceleration, This is the driver's reaction time, usually taken as 1.5 seconds.
[0081] 222) The maximum safe deceleration design is as follows:
[0082] (4)
[0083] Among them, the Maneuvering Autonomous Deceleration Rate (MADR) is used to evaluate the emergency braking capability of a vehicle, describing the maximum rate at which a vehicle can decelerate while avoiding a collision with an obstacle in front. This refers to the maximum safe deceleration a vehicle can achieve under its current driving conditions.
[0084] Specifically, for safety reasons, the acceleration of the target vehicle at any given time... All must not exceed the maximum safe acceleration / deceleration range, that is:
[0085] (5).
[0086] 23) Define the multi-objective reward function for intelligent connected vehicles.
[0087] This invention employs a long following distance CAV (Connected and Automated Vehicles) control algorithm, which reduces the impact of traffic wave propagation on the target vehicle by increasing the following distance between the target vehicle and the vehicle in front. The distance between the vehicles in front and behind is long enough to even cancel out the propagation of traffic waves.
[0088] The first objective is to suppress the propagation of stop-and-go waves by maintaining a safe and feasible headway. To effectively suppress stop-and-go waves and maintain following distance, a threshold for following distance is set. The following distance is set at 120 meters; once this threshold is exceeded, a significant penalty is imposed. Furthermore, to ensure driving safety, even larger penalties should be applied promptly in the event of a collision.
[0089] (6)
[0090] in, The target vehicle n is at time t and is in contact with the vehicle in front. The real-time distance between them.
[0091] The second objective is traffic efficiency. Vehicles receive a greater reward value the closer they travel to the expected speed.
[0092] (7)
[0093] in, It is the vehicle's desired driving speed. It is the real-time driving speed of the target vehicle n at time t.
[0094] The third objective is to reduce fuel consumption. In this embodiment, the reward function for reducing fuel consumption consists of two parts. Since the vehicle's high fuel consumption mainly stems from frequent acceleration and deceleration at low speeds, the first part of the reward function is defined as the negative of the square of the target vehicle's acceleration; the larger the acceleration value, the smaller the reward.
[0095] (8)
[0096] in, It is the real-time acceleration of the target vehicle n at time t.
[0097] The second part of the reward function is defined as the negative value of the fuel consumption of intelligent connected vehicles, that is, the more fuel consumed, the smaller the reward value will be:
[0098] (9)
[0099] in, This represents the fuel consumed by the vehicle at time t. Calculating a vehicle's fuel consumption requires predefining the physical characteristics and air resistance characteristics involved. Based on the assumptions and parameter values, the instantaneous traction force experienced by the vehicle at time t can be calculated using the following formula:
[0100] (10)
[0101] (11)
[0102] (12)
[0103] (13)
[0104] Among them, equation (10) is the basic dynamic model of the vehicle during its motion; and Let be the vehicle's acceleration and velocity at time t, respectively. , , Let the air resistance, rolling resistance, and gravity acting on the vehicle at time t be respectively. If it is positive, then the fuel consumption per second is:
[0105] (14)
[0106] if If it is negative, then the fuel consumption per second is:
[0107] (15)
[0108] in, It is vehicle acceleration. , , , , , , These are the coefficients of the fitted model.
[0109] Therefore, the complete reward function is shown in the following equation:
[0110] (16)
[0111] in, , and These are the weights of the different components of the reward function. After careful parameter tuning, the following values are used: , , .
[0112] 24) Define the agent design parameters:
[0113] 241) The state space is defined as the information exchange between the target vehicle and the two preceding vehicles based on the information obtained by the target vehicle itself.
[0114] 242) The action space is defined as the limit on the acceleration of the target vehicle to ensure that the acceleration of the target vehicle at any time does not exceed the safe value.
[0115] 243) The weights of the reward function are determined as shown in Table 1 below:
[0116] Table 1. Cawing performance of the final reward function weight parameters
[0117]
[0118] Step 3: Define the policy network and the value function network.
[0119] Define a policy network with one hidden layer, where the output layer uses the tanh activation function and outputs the numerical value of acceleration. Define a value function network with a single hidden layer, where the hidden layer uses ReLU as the activation function, and the output layer is a single node representing the estimate of the state value.
[0120] An auxiliary critic network was introduced into the existing actor-critic network, forming a dual critic network. Both evaluation networks were used to estimate the maximum value of the action. The minimum value between the two estimates was taken to trim the action value and reduce overestimation. The dual evaluation networks were used to... and The improved estimate of the value of a motion can be expressed as:
[0121] (17)
[0122] Where: r is the immediate reward, representing the immediate reward obtained by the agent after performing an action in the current state;
[0123] The discount factor measures the importance of future rewards. It ranges between 0 and 1, representing the degree to which the agent discounts future rewards. A higher discount factor indicates a greater emphasis on long-term rewards, while a lower discount factor emphasizes immediate rewards.
[0124] s' represents the next state;
[0125] a' represents the action to be taken at the next state s';
[0126] The action value of taking action a' in the next state s' of the target network is estimated by using the target network parameters. The target network is a neural network similar to the current network (main network), but with a lower parameter update frequency, used to provide relatively stable target values.
[0127] Step four involves using the SUMO tool on the constructed vehicle control experimental platform to simulate different weather conditions and to build a robustness index to measure the model's effectiveness in suppressing stop-and-go traffic waves.
[0128] 41) Construct robustness indicators to measure the model's effectiveness in suppressing stop-and-go traffic waves. This invention proposes rolling mean and standard deviation of speed to measure the ability of intelligent connected vehicles to absorb aggressive acceleration or deceleration of vehicles ahead.
[0129] 411) Rolling standard deviation Its value changes over time and can be calculated at each time step k, as shown in the following formula:
[0130] (18)
[0131] (19)
[0132] Where T is the length of the scrolling time window. Let T be the average velocity starting from the k-th time step. Let be the velocity at the i-th time step. Let T be the rolling standard deviation of the velocity starting at the k-th time step. To measure the overall local velocity change throughout the entire evaluation period, take all rolling standard deviations during this period. The average value is defined as follows:
[0133] (20)
[0134] 412) Deceleration rate change ; Acceleration rate change ; Oscillating growth .
[0135] The Mexican hat wavelet transform is applied to analyze the characteristics of traffic stop-and-go waves to determine the start time, acceleration / deceleration start and end times, and propagation characteristics of the traffic stop-and-go waves.
[0136] Wavelet analysis can be used to obtain the vehicle's deceleration start time, deceleration end time, acceleration start time, and acceleration end time, thereby enabling [further analysis]. and Calculations are performed to determine if, for a single traffic flow, it can suppress the propagation of traffic stop waves. , and The values should all be, or mostly, negative. The following formula is based on the Mexican hat wavelet transform. , and The calculation formula.
[0137] (twenty one)
[0138] (twenty two)
[0139] (twenty three)
[0140] Where N is the number of vehicles in the traffic flow; It is the average deceleration of the vehicle at position j in the traffic flow minus the average deceleration of the vehicle at position j-1. The two average decelerations are calculated based on the start and end times of deceleration of the preceding and following vehicles, respectively. It is the average acceleration of the vehicle at position j in the traffic flow minus the average acceleration of the vehicle at position j-1. The two average accelerations are calculated based on the acceleration start time and acceleration end time of the preceding and following vehicles, respectively. The minimum speed of the vehicle at position j-1 in the traffic flow is minus the minimum speed of the vehicle at position j within a driving cycle.
[0141] 42) The trained single-vehicle longitudinal control model was applied to different weather conditions for testing, including moderate rain, heavy rain, and snow. Each weather scenario was tested with 100% intelligent connected vehicle penetration. The road friction coefficient was set according to different weather conditions, as shown in Table 2.
[0142] Table 2 Road surface friction coefficients under different weather conditions
[0143]
[0144] Step 5: Based on the single-vehicle longitudinal control model in Step 4, build a transfer learning framework. The process is as follows: Figure 3 As shown.
[0145] In the neural network structure of the target scene, each neuron p in the target scene neural network can be mapped to each neuron in the source scene neural network through a mapping relationship. The mapping function can be expressed as:
[0146] (twenty four)
[0147] in, This means mapping the state variables in the target scene to the most similar state variables in the source scene; This means mapping actions in the target scene to the most similar actions in the source scene.
[0148] In the strategy migration process of this invention, the source scenarios used are all traffic stop-and-go wave scenarios constructed by the vehicle longitudinal control experimental platform in step one. The target scenario adds the road surface friction coefficient to the source scenario to simulate traffic stop-and-go wave scenarios under different weather conditions.
[0149] Step 6: Use transfer learning to train the converged single-vehicle longitudinal control model under different weather conditions, and further compare and analyze the robustness of the control strategy before and after transfer learning.
[0150] The robustness of the single-vehicle longitudinal control model before and after transfer learning is compared using the robustness index from step four, where the rolling standard deviation is one of the metrics. The calculation time window T is the entire vehicle operation process from 0s to 250s, and it is substituted into equation (20); acceleration rate change Deceleration rate change These two indicators require the use of wavelet analysis to identify four key time points: deceleration start time, deceleration end time, acceleration start time, and acceleration end time. Equations (21) and (22) are then used to calculate the average acceleration and average deceleration for each vehicle during the acceleration and deceleration processes; oscillating growth. The minimum value during the operation of each vehicle is directly taken and substituted into equation (23). Based on the above process, the robustness index comparison results of single-vehicle longitudinal control strategies with and without transfer learning under different weather conditions are shown in Table 3.
[0151] Table 3 Robustness improvement effect under different weather scenarios
[0152]
[0153] It should be noted that, as shown in Table 3, the rolling standard deviation... The closer to 0 and the change in deceleration rate Acceleration rate change Oscillating growth The smaller the three indicators, the stronger the robustness.
[0154] Table 3 shows the rolling standard deviation of each vehicle in the traffic flow under the control strategy after transfer learning, in scenarios of moderate rain, heavy rain, and icy / snowy weather. Deceleration rate change Acceleration rate change Oscillating growth All four indicators showed varying degrees of decline compared to the control strategy before transfer learning, which means that robustness has been improved.
Claims
1. A single-vehicle control method based on scene style transfer and SAC network collaboration, characterized in that, The steps include the following: S1. Utilize the SUMO tool to build a simulation platform, configure road files, vehicle routing files, and simulation configuration files; simulate the target vehicle perceiving information about the surrounding environment through V2X communication technology and onboard sensors. S2, Construct a single-vehicle longitudinal control model, use an SAC network to obtain traffic status from the simulation platform, and then take actions to control the vehicle's behavior. After the action is executed, the agent receives the reward value fed back from the simulation platform to update the agent parameters. The parameters of the single-vehicle longitudinal control model include the state space, action space and target reward function of the intelligent connected vehicle. S3 defines a policy network that outputs numerical acceleration; and a value function network whose output layer represents an estimate of the state values. S4 uses the vehicle control experimental platform to call SUMO to simulate different weather conditions and constructs a robustness index to measure the model's ability to suppress stop-and-go traffic waves. S5, a transfer learning framework is built on the basis of the single vehicle longitudinal control model; S6. Transfer learning is used to train a converged single-vehicle longitudinal control model under different weather conditions. In step S2, the target vehicle, based on its own information, obtains information about the two preceding vehicles. The vehicle's state space is then represented as follows: ， in, It is the distance between the target vehicle and the second vehicle in front of it. That's the speed of the second car in front. It is the acceleration of the second vehicle in front; It is the distance between the target vehicle and the first vehicle in front of it. That's the speed of the first car in front. It is the acceleration of the first car in front; It is the speed of the controlled target vehicle. It is the acceleration of the target vehicle; the first vehicle in front is closer to the target vehicle than the second vehicle in front. In step S2, safety constraints are introduced to avoid rear-end collisions, and the vehicle's motion space is defined as follows: The maximum safe acceleration is designed as follows: ， ， in, This refers to the minimum safe distance that a vehicle needs to maintain from the vehicle in front while it is currently driving. This refers to the maximum safe acceleration of the vehicle under its current driving conditions; It is the length of the vehicle in front of the target vehicle. The target vehicle With the car in front relative velocity, The target vehicle n and the vehicle in front The difference in acceleration, It is the driver's reaction time; The maximum safe deceleration design is as follows: ， MADR stands for collision avoidance deceleration rate, which is the maximum rate at which a vehicle can decelerate while avoiding a collision with an obstacle in front of it. This refers to the maximum safe deceleration of a vehicle under its current driving conditions; Then the acceleration of the target vehicle at any time satisfy: 。 2. The single-vehicle control method based on scene style transfer and SAC network collaboration according to claim 1, characterized in that, In step S2, the target reward function for intelligent connected vehicles is defined as follows: The primary objective is to suppress the propagation of stop-and-go waves by maintaining a safe and feasible headway; a threshold for following distance is set. If the distance is set to 120 meters, the expression for the penalty value is as follows: ， in, The target vehicle n is at time t and is in contact with the vehicle in front. Real-time distance between them; The second objective is traffic efficiency; vehicles will receive a greater reward value the closer they travel to the expected speed. ， in, It is the vehicle's desired driving speed. It is the real-time driving speed of the target vehicle n at time t; The third goal is to reduce fuel consumption, which includes two parts: The first part of the reward function is defined as the negative of the square of the target vehicle's acceleration; the larger the acceleration value, the smaller the reward. ， in, It is the real-time acceleration of the target vehicle n at time t; The second part of the reward function is defined as the negative value of the fuel consumption of intelligent connected vehicles; the more fuel consumed, the smaller the reward value will be. ， in, This represents the amount of fuel consumed by the vehicle at time t. Therefore, the complete reward function is shown in the following equation: ， in, , and These are the weights of the different components of the reward function.
3. The single-vehicle control method based on scene style transfer and SAC network collaboration according to claim 1, characterized in that, In step S3, a policy network with one hidden layer is defined, and the output layer uses the tanh activation function to output the numerical value of acceleration; a value function network with a single hidden layer is defined, and the hidden layer uses ReLU as the activation function, and the output layer is a single node representing the estimate of the state value.
4. The single-vehicle control method based on scene style transfer and SAC network collaboration according to claim 1, characterized in that, In step S4, the measurement model includes the speed rolling mean and standard deviation; Speed rolling standard deviation The value of varies with time and is calculated at each time step k, as shown in the following formula: ， ， Where T is the length of the scrolling time window. Let T be the average velocity starting from the k-th time step. Let be the velocity at the i-th time step. Let T be the rolling standard deviation of the velocity starting at the k-th time step; to measure the overall local velocity change throughout the entire evaluation period, take all rolling standard deviations during this period. The average value is defined as follows: ； Solving for deceleration rate variation based on Mexican hat wavelet transform. Acceleration rate change and oscillating growth : ， ， ， Where N is the number of vehicles in the traffic flow; It is the average deceleration of the vehicle at position j in the traffic flow minus the average deceleration of the vehicle at position j-1. The two average decelerations are calculated based on the start and end times of deceleration of the preceding and following vehicles, respectively. It is the average acceleration of the vehicle at position j in the traffic flow minus the average acceleration of the vehicle at position j-1. The two average accelerations are calculated based on the acceleration start time and acceleration end time of the preceding and following vehicles, respectively. The minimum speed of the vehicle at position j-1 in the traffic flow is minus the minimum speed of the vehicle at position j within a driving cycle.
5. The single-vehicle control method based on scene style transfer and SAC network collaboration according to claim 1, characterized in that, In step S5, in the neural network structure of the target scene, each neuron p in the target scene neural network is mapped to each neuron in the source scene neural network through a mapping relationship. The mapping function is expressed as: ， in, This means mapping the state variables in the target scene to the most similar state variables in the source scene; This means mapping actions in the target scene to the most similar actions in the source scene.