Networked autonomous vehicle communication and cooperative control method for mixed traffic flow

Abstract

The invention discloses a mixed traffic flow-oriented networked autonomous vehicle communication and cooperative control method, which is applied to the field of intelligent Internet of Vehicles and aims at solving the problem that in the prior art, limitation of communication resources and information updating requirements of digital twinning are difficult to balance and effective cooperation between CAVs is realized. According to the invention, the distributed digital twin system is constructed in each CAV, the road condition, the communication channel state and the driving strategy of the surrounding vehicles are monitored in real time and dynamically simulated, and the decision deficiency caused by the sensing limitation of a single vehicle is made up, so that the group intelligent collaboration among the vehicles is realized. While the individual driving style is reserved, the CAV can more effectively relieve the disturbance of HDV driving to traffic, and the efficiency and stability of traffic flow are remarkably improved. Besides, the pressure of perception information sharing on a communication system is fully considered in the CAV collaborative driving process, information with the highest decision-making value is optimized under the condition of limited communication resources, and the interaction frequency between vehicles is dynamically adjusted according to the channel state, so that the lightweight design is realized, and the cost is reduced. And the applicability and the reliability of the scheme in an actual application scene are further enhanced.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent vehicle networking, and in particular relates to a self-driving car communication and collaborative control technology. Background Art

[0002] Internet of Vehicles technology and high-performance on-board computing platforms enable connected autonomous vehicles (CAVs) to communicate in real time and efficiently process driving information, allowing them to flexibly respond to complex and changing road environments. This capability not only improves driving safety, but also provides strong technical support for collaborative control between vehicles, and is expected to further enhance the stability of traffic flow and improve road traffic efficiency. Therefore, connected autonomous driving technology is widely regarded as one of the important driving forces for the transformation of modern transportation towards intelligence and efficiency. However, although autonomous driving technology has made significant progress, its full popularization and application is still limited by many factors such as economic costs, laws and regulations, and social acceptance. In the current traffic environment, autonomous vehicles still need to coexist with human-driven vehicles (HDVs), forming a hybrid autonomous driving traffic system.

[0003] In this system, human drivers often interfere with traffic flow due to their limited perception, delayed reaction speed, and inaccurate decision-making. If these interferences are not effectively controlled, they will gradually accumulate and eventually cause traffic congestion. In addition, subjective factors such as individual driving habits and psychological states also increase the unpredictability of human driver behavior, further increasing the difficulty of managing hybrid autonomous driving traffic systems. Therefore, how to reduce HDV disturbances to traffic and improve road stability and traffic efficiency through collaborative control of CAVs while ensuring traffic safety has become an important issue that needs to be urgently addressed in the field of autonomous driving.

[0004] Existing studies have shown that the cooperative control of CAVs can improve traffic performance to a certain extent, but most of these studies are based on the idealized assumption that information is fully shared between vehicles and driving strategies are completely transparent, ignoring the limited communication resources and the instability of physical channels. For example, in a typical intersection scenario with 20 vehicles, the communication bandwidth required to share radar data and camera images between vehicles can be as high as 1Gbps, and as the number of vehicles increases, the amount of information grows exponentially, which not only increases communication delays and packet loss rates, but may also have an adverse effect on driving performance. Therefore, in the cooperative control of connected autonomous driving, fully considering the limitations of communication resources is of great significance to improving the practical feasibility and effectiveness of the strategy.

[0005] Digital twin technology provides a new solution to bridge the gap between communication and transportation systems. As a virtual mapping of the physical world, digital twins can collect data in real time, conduct in-depth simulation analysis and make intelligent decisions, showing unique advantages in solving the joint design problems of complex systems. However, the application of digital twin technology in hybrid autonomous driving transportation systems also faces many challenges. On the one hand, the precise modeling and decision-making of digital twins depend on the comprehensiveness, real-time and accuracy of information, which is in conflict with limited communication resources; on the other hand, due to the personalization of driving behavior, each vehicle needs to maintain its own digital twin, and the decentralization trend has increased the complexity of cooperation between vehicles, which may lead to inconsistent driving decisions. Therefore, how to balance the limitations of communication resources with the information update requirements of digital twins and achieve effective collaboration between CAVs is a technical difficulty in the current hybrid autonomous driving transportation system. Summary of the invention

[0006] In order to solve the above technical problems, the present invention proposes a communication and collaborative control method for networked autonomous driving vehicles for mixed traffic flows. While fully considering the constraints of communication resources, the method of the present invention reduces the disturbance of human drivers to traffic through efficient collaborative perception and control between CAVs, ensures travel safety, and improves the stability of traffic flow and road capacity.

[0007] The technical solution adopted by the present invention is: a communication and cooperative control method for networked autonomous driving vehicles for mixed traffic flow, the application scenario is: in a road section containing closely following HDVs and CAVs, the CAV has the ability to perceive the surrounding road conditions and communication, and manages and allocates communication resources through roadside base stations to share perception information; the specific implementation process includes the following steps:

[0008] S1. Based on the overall road condition characterization within the communication range, the evaluation of the channel state obtained by the CAV through spectrum sensing, and the CAV's fitting of the driving strategies of surrounding vehicles, a distributed digital twin is established on the CAV's onboard computing platform;

[0009] The overall road condition representation within the communication range includes the perception information of the current CAV and the perception information of other CAVs obtained through vehicle-to-vehicle communication; the perception information of each CAV includes the vehicle state of the CAV and the point cloud data of the surrounding driving environment perceived by the CAV;

[0010] The vehicle status includes the speed of the front and rear vehicles, and the distance between the CAV and the front and rear vehicles;

[0011] S2, each CAV senses the surrounding road conditions through the vehicle-mounted radar and generates point cloud data of the surrounding driving environment;

[0012] S3. A multi-agent reinforcement learning method based on the attention mechanism is used to obtain the communication and driving decisions of each CAV. Specifically:

[0013] S31. The digital twin of each CAV includes: an online network, which includes an attention network, a policy network, and a value network;

[0014] S32, taking the overall road condition representation within the communication range corresponding to each CAV as the input feature in the attention network, linearly mapping the input feature into a query vector and a key vector, and obtaining the attention weight of the current CAV to other CAVs;

[0015] S33, calculating the update frequency corresponding to each CAV according to the attention weight and the availability of communication resources;

[0016] S34, based on the update frequency, a communication demand is generated, and the roadside base station allocates a time-frequency resource block for communication between CAVs according to the communication demand. Each CAV communicates according to the allocated video resource block, and obtains a reward function value reflecting the communication performance And update the overall road condition representation in the digital twin;

[0017] S35, the updated overall road condition is represented as the input feature in the attention network, and the hidden state is calculated

[0018] S36, will Input the policy network to obtain driving acceleration

[0019] S37, CAV obtained according to step S36 Drive and calculate reward values ​​reflecting traffic performance

[0020] S38, the vehicle state, attention weight, Put it into the experience replay pool;

[0021] S39. Repeat steps S2-S38, randomly sample the experience in the playback pool, calculate the loss function and backpropagate, and update the attention network, policy network and value network.

[0022] Beneficial effects of the present invention: Compared with the prior art, the present invention builds a distributed digital twin system inside each CAV to conduct real-time monitoring and dynamic simulation of road conditions, communication channel status and driving strategies of surrounding vehicles, thereby making up for the lack of decision-making caused by the limitations of single-vehicle perception, thereby realizing group intelligent collaboration between vehicles. While retaining individual driving styles, CAV can more effectively alleviate the disturbance of HDV driving to traffic and significantly improve the efficiency and stability of traffic flow. In addition, the present invention fully considers the pressure of perception information sharing on the communication system during the CAV collaborative driving process, and realizes lightweight design by selecting the most decision-making information under limited communication resources and dynamically adjusting the interaction frequency between vehicles according to the channel status, further enhancing the applicability and reliability of the solution in actual application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is the application scenario of the present invention.

[0024] Figure 2 This is a flow chart of the hybrid autonomous driving communication and cooperative control algorithm based on lightweight digital twins.

[0025] Figure 3 Schematic diagram of a multi-agent reinforcement learning model based on the attention mechanism. DETAILED DESCRIPTION

[0026] To facilitate those skilled in the art to understand the technical content of the present invention, the present invention is further explained below with reference to the accompanying drawings.

[0027] In order to cope with the disturbance caused by human drivers to the overall traffic flow in hybrid autonomous driving traffic flow, as well as the problem of limited communication resources affecting the interaction of perceptual information required for cooperative driving in practical applications, the present invention proposes a hybrid autonomous driving traffic control method based on lightweight digital twins. Each CAV is equipped with an independent digital twin system to make up for the limitations of single-vehicle perception and simulate the driving behavior of surrounding vehicles. In the case of limited communication resources, the digital twin system filters out the perceptual information that is most critical to vehicle decision-making and traffic control, interacts through asynchronous updates, and provides decision support for vehicles, thereby effectively improving the stability and traffic efficiency of traffic flow in hybrid driving scenarios. Figure 1 As shown, the application scenario of the present invention is a road including a CAV and a HDV that are closely following each other.

[0028] The technical solution of the present invention includes two parts: a lightweight digital twin architecture and a multi-agent reinforcement learning model based on an attention mechanism. Starting from the following behavior between vehicles and its cumulative traffic effects, combined with the characteristics of vehicles being guided by individual interests when driving, the present invention establishes a digital twin system on the on-board computing platform of each vehicle to virtually map the road conditions, channel status and driving strategies of surrounding vehicles, so that the vehicle can grasp more comprehensive road conditions information, deduce the driving behavior of other vehicles, and predict future road and channel conditions. The present invention adopts a multi-agent reinforcement learning model based on an attention mechanism, selects the source and frequency of information interaction for the vehicle in a distributed manner based on the current availability of communication resources, conducts real information interaction in the physical channel, and decides the acceleration and deceleration of the vehicle based on the interaction results.

[0029] The implementation process of the present invention is as follows:

[0030] 1. Lightweight digital twin architecture

[0031] In the present invention, the set of all vehicles is represented by V = V H ∪V C , where V H Denotes the set of HDV, V C represents the set of all CAV vehicles. For any following vehicle v in a following pair i ∈V and its predecessor v i+ ∈V, if the following vehicle is HDV, that is, v i ∈V H , then due to the limited perception range of human drivers, the observation information of the following vehicle at time t is in is the speed of the front vehicle, is the distance between the two vehicles. Based on this perception information, the following vehicle adjusts its speed to ensure that it closely follows the leading vehicle while maintaining safe driving. Need to meet:

[0032]

[0033] in, and They represent the maximum deceleration of the front and rear vehicles during emergency braking, and Δt is the time from when the driver of the rear vehicle discovers the front vehicle braking to when he / she responds to the braking. During this time, the rear vehicle maintains the original speed. Since HDV's driving decision is based only on the current state of the front vehicle, it is difficult to make advance predictions about traffic conditions, resulting in poor performance in terms of traffic flow stability and traffic capacity. CAV can expand its perception range through on-board cameras and radars. If v i ∈V C , then its observation information is in and For the rear car v i- The speed and distance between the two vehicles are dynamically measured by the camera. The point cloud data of the surrounding driving environment generated by radar perception. In addition, CAV can also exchange its perception information through vehicle-to-vehicle communication, giving the vehicle a wider field of view.

[0034] However, frequent interactions will bring great burden to the communication system. Therefore, the present invention establishes an independent digital twin system for each vehicle to achieve the update and maintenance of lightweight perception information, so that each vehicle can efficiently obtain the perception information of other vehicles under limited communication resources, thereby improving the efficiency of collaborative driving control. The construction process of the lightweight digital twin proposed in the present invention mainly includes the following stages:

[0035] The first is information collection. The information sources include the vehicle's on-board camera and radar perception, the perception of surrounding vehicles obtained through vehicle-to-vehicle communication, the evaluation of communication resource availability when interacting with perception information, and the fitting of other vehicles' driving strategies during driving. Based on the collected information, each CAV builds a distributed digital twin system on its own on-board computing platform. in Represents the vehicle's own perception and the collection of other vehicles' perception information obtained through vehicle-to-vehicle communication, which is used to form v i Overall representation of traffic conditions; Indicates the vehicle's assessment of the channel status obtained through spectrum sensing, which is used to support the vehicle's dynamic adjustment information update requirements; i Yes i The fitting of the driving strategies of surrounding vehicles is used to infer the driving behaviors of other vehicles and improve the collaborative effect.

[0036] The second is the deduction stage. The digital twin architecture proposed in the present invention is different from the traditional digital twin system, which requires real-time and accurate maintenance and update of information to support decision-making. The present invention believes that not all data from other vehicles are of equal importance. Therefore, the updates in the system of the present invention will be selectively synchronized to consider the correlation of channel states and the importance of exchanged information, thereby reducing the communication burden and building a more efficient and lightweight system. In the deduction stage, the present invention focuses on how observation data from different vehicles affect the collaborative behavior of a single CAV. In order to more clearly represent the information exchange process, the present invention uses an update interval is used to measure the time that has passed since the data was generated. Therefore, It can be expressed as Through continuous interaction and feedback with the physical world, the present invention analyzes the underlying relationship between traffic utility and update interval to identify the optimal information exchange that balances cooperative driving efficiency and communication constraints.

[0037] The last stage is the decision and feedback stage. According to the inferred update interval, v i Get observation data from selected vehicles for continuous updates Based on the update v i Distributed acceleration decisions are made, acting on the physical world and collecting feedback on overall traffic performance. This process not only supports real-time vehicle-following decisions, but also provides guidance for long-term driving strategies. As new information is continuously collected, the vehicle will respond and adjust its interaction strategy and driving strategy in the digital twin, allowing the vehicle to dynamically adapt to changing traffic conditions and communication resource constraints.

[0038] 2. Multi-agent reinforcement learning method based on attention mechanism

[0039] The present invention proposes a multi-agent reinforcement learning method based on the attention mechanism, which is used to optimize the interaction of perception information between vehicles under the condition of limited communication resources, and make driving decisions based on the interaction information. This method improves the stability and traffic capacity of the road as much as possible on the basis of ensuring driving safety.

[0040] like Figure 3 As shown in the figure, each CAV is modeled as an intelligent agent connected by a wireless channel, and its communication and driving are defined as partially observable Markov processes. At each time t, each vehicle first updates its local observation information The digital twin then updates the overall road condition representation through vehicle-to-vehicle communication. Based on the information in the updated twin system, v i Further generate driving decisions and determine its acceleration and deceleration behavior. Therefore, the core of this problem can be divided into two parts: the goal and frequency of information update Optimization and driving acceleration decisions generate.

[0041] In the attention mechanism, the input features of each car is linearly mapped into query vector and key vector for extracting relevant features. i ∈V C For vehicles j ∈V C The attention weight is calculated as follows:

[0042]

[0043] Among them, q iis the query vector, k j are the key vectors and d is the dimension of these vectors. Determine the vehicle v i and vehicle v j The importance of information exchange between i The sum of attention weights for all vehicles is equal to 1.

[0044] Although the attention mechanism can effectively evaluate the importance of observation information from different CAVs, its evaluation process relies on receiving data from other vehicles first, and thus cannot fundamentally reduce redundant communication interactions. To solve this problem, the present invention makes full use of the characteristics of the Markov process and takes the attention score of the previous moment as part of the current state. Compared with directly using the attention weight, this method uses its reciprocal and rounds down to determine the frequency of information updates for each vehicle. The specific expression is as follows:

[0045]

[0046] Where M represents the maximum number of vehicles that each vehicle can communicate with at any time, taking into account delay and bandwidth limitations, and is calculated as follows:

[0047]

[0048] Among them, B max is the total bandwidth upper limit of the system, D represents the amount of perceived data, τ max It is the maximum allowed delay for successful transmission of sensing information. v i and v j The expected value of the channel gain between the two channels is ij is the physical distance between the two vehicles, α is the path loss exponent, A Gaussian variable that obeys Rayleigh distribution represents the channel gain. To minimize the average communication delay, the roadside base station allocates orthogonal sub-channels to each link between vehicles according to the communication requirements of each link to reduce co-channel interference. This method reasonably allocates the information update frequency of each vehicle by weighing the bandwidth, delay, and physical communication conditions between vehicles, thereby minimizing redundant communication interactions while ensuring communication efficiency.

[0049] counter Recording the vehicle v i The length of time since the last communication with other vehicles. When i Need to be with v j Communication to update its digital twin system Subsequently, each CAV updates its And calculate the hidden state

[0050]

[0051] concatenate means concatenating multiple vectors, and tanh is the hyperbolic tangent function, which is used for data activation. The process is recorded as the attention network Att i (ω Att ),ω Att is the network parameter. After that, each vehicle can make independent decisions. The reward function of the vehicle From the perspective of transportation and communication, it can be divided into two parts, namely in Reflects traffic performance and is defined as

[0052]

[0053] Indicates v i The absolute value of the change in following distance is used to measure the stability of traffic flow. v i The driving speed at time t is used to measure the road capacity. It is to judge whether the following distance meets the minimum safe distance, which is the basis of driving safety.

[0054] Reward function for communication performance Defined as

[0055]

[0056] This function is used to evaluate the communication delay of vehicles when interacting with each other to perceive information. Whether the delay upper limit τ of successful transmission is met max , thereby measuring the efficiency of the communication system.

[0057] Due to the acceleration is a continuous variable. The present invention designs an online attention network, a strategy network, and a value network in the digital twin system of each CAV. Strategy network μ i (ω μ ) is used to decide the acceleration Value Network It is used to evaluate the quality of actions output by the policy network. In addition, in order to enhance the stability of the policy, target network copies are created for each of the three networks. and And the parameters of the online network are synchronized regularly through the soft update mechanism.

[0058] The loss function of the value network is:

[0059]

[0060] Where γ is the decay factor. The loss function of the attention network and the executor network can be calculated by the following formula:

[0061]

[0062] With the above loss function, the gradient can be calculated by the chain rule to update the relevant network parameters. This training process optimizes the model to derive information exchange and driving strategies that meet the communication resource constraints.

[0063] 3. Hybrid autonomous driving traffic control algorithm process based on lightweight digital twin

[0064] The method of the present invention is a collaborative control method for CAVs in a hybrid autonomous driving scenario, which fully considers the communication overhead of information sharing during vehicle collaborative driving, adaptively selects the objects and frequencies of perception information sharing between vehicles, and implements collaborative control based on the information. The algorithm steps are as follows:

[0065] Step 1: Initialize road driving status, vehicle following relationship, communication resources, V2V link, etc.

[0066] Step 2: Build a lightweight digital twin system in each CAV’s onboard computing platform At the same time, initialize the online attention network, policy network, and evaluation network Att in the twin system i (ω Att ), μ i (ω μ ), and the corresponding target network and Initialize counter C i ;

[0067] Step 3: CAV updates its perception through cameras and radar

[0068] Step 4: Vehicle-based attention score calculate As an indicator of update intervals;

[0069] Step 5: Determine which vehicles need to obtain interaction information from at the current moment through the update interval and update counter;

[0070] Step 6: According to the communication requirements, the roadside base station allocates orthogonal sub-channels for the V2V links that need to communicate. The vehicles communicate according to the allocated communication resources and obtain communication-related reward values. And update the representation of road conditions in the digital twin

[0071] Step 7. Based on the updated Computing hidden states for decision making in digital twin systems

[0072] Step 8. Enter the Online Strategy Network Get driving acceleration

[0073] Step 9: The vehicle drives according to the decision-making driving behavior in the digital twin system and calculates the traffic-related reward value

[0074] Step 10: Store the experience in the experience revisit pool, randomly sample, calculate the loss function, and update the online attention network, policy network, and value network;

[0075] Step 11, at a certain interval, soft update the target network with the parameters of the online network;

[0076] Step 12, return to Step 3;

[0077] Compared with other CAV collaborative control algorithms, the present invention has the following innovations: First, for the coexistence scenario of CAV and HDV, the present invention establishes the functional relationship between CAV perceptual information interaction and collaborative control effect, proposes a lightweight digital twin architecture, and designs a traffic performance-oriented information interaction strategy. According to the real-time traffic conditions and the availability of communication resources, CAV adaptively selects the objects and frequencies of perceptual information interaction required for control through digital twin deduction, and provides an optimized information transmission scheme for collaborative control under limited communication resources. Secondly, the present invention proposes a multi-agent reinforcement learning method based on the attention mechanism for distributed collaborative control of CAV. While retaining the individual driving style and interests of CAV, this method realizes multi-vehicle collaboration through group intelligence, effectively alleviating the disturbance caused by manual driving behavior to traffic flow, and improving the stability and traffic efficiency of traffic flow.

[0078] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the scope of protection of the present invention is not limited to such specific statements and embodiments. For those skilled in the art, the present invention may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A communication and cooperative control method for networked autonomous driving vehicles in mixed traffic flow, characterized in that: The application scenario is: in a road section with closely following HDVs and CAVs, the CAV has the ability to perceive the surrounding road conditions and communicate, and manage and allocate communication resources through roadside base stations to share perception information; the specific implementation process includes the following steps: S1. Based on the overall road condition characterization within the communication range, the evaluation of the channel state obtained by the CAV through spectrum sensing, and the CAV's fitting of the driving strategies of surrounding vehicles, a distributed digital twin is established on the CAV's onboard computing platform; The overall road condition representation within the communication range includes the perception information of the current CAV and the perception information of other CAVs obtained through vehicle-to-vehicle communication; the perception information of each CAV includes the vehicle state of the CAV and the point cloud data of the surrounding driving environment perceived by the CAV; The vehicle status includes the speed of the front and rear vehicles, and the distance between the CAV and the front and rear vehicles; S2, each CAV senses the surrounding road conditions through the vehicle-mounted radar and generates point cloud data of the surrounding driving environment; S3. A multi-agent reinforcement learning method based on the attention mechanism is used to obtain the communication and driving decisions of each CAV. Specifically: S31. The digital twin of each CAV includes: an online network, which includes an attention network, a policy network, and a value network; S32, taking the overall road condition representation within the communication range corresponding to each CAV as the input feature in the attention network, linearly mapping the input feature into a query vector and a key vector, and obtaining the attention weight of the current CAV to other CAVs; S33, calculating the update frequency corresponding to each CAV according to the attention weight and the availability of communication resources; S34, based on the update frequency, a communication demand is generated, and the roadside base station allocates a time-frequency resource block for communication between CAVs according to the communication demand. Each CAV communicates according to the allocated video resource block, and obtains a reward function value reflecting the communication performance And update the overall road condition representation in the digital twin; S35, the updated overall road condition is represented as the input feature in the attention network, and the hidden state is calculated S36, will Input the policy network to obtain driving acceleration S37, CAV obtained according to step S36 Drive and calculate reward values ​​reflecting traffic performance S38, the vehicle state, attention weight, Put it into the experience replay pool; S39. Repeat steps S2-S38, randomly sample the experience in the playback pool, calculate the loss function and backpropagate, and update the attention network, policy network, and value network.

2. The method for communication and cooperative control of networked autonomous driving vehicles for mixed traffic flow according to claim 1, characterized in that: The calculation formula for the availability of the communication resources in step S3 is: Among them, B max is the total bandwidth upper limit of the system, D represents the amount of perceived data, τ max is the maximum allowable delay for successful transmission of sensing information, is the expected value of the channel gain between the i-th CAV and the j-th CAV, d0 is the reference distance, d ij is the physical distance between the i-th CAV and the j-th CAV, α is the path loss exponent, represents the channel gain, V C Represents a collection of CAVs.

3. The method for communication and cooperative control of networked autonomous driving vehicles for mixed traffic flow according to claim 2, characterized in that: The update frequency is calculated as: in, represents the update frequency between the i-th CAV and the j-th CAV at time t, It represents the attention weight of the i-th CAV to the j-th CAV at time t-1.

4. The method for communication and cooperative control of networked autonomous driving vehicles for mixed traffic flow according to claim 3, characterized in that: In step S34, the digital twin is updated to represent the overall road condition. Specifically, the i-th CAV is recorded as v i , using a counter Recorded from v i The time since the last communication with other CAVs, c ij Indicates that v i The time since the last communication with the jth CAV, j = 1, 2, ..., V c ;when When i Communication with the jth CAV is required to update the overall road condition representation in its digital twin.

5. The method for communication and cooperative control of networked autonomous driving vehicles for mixed traffic flow according to claim 4, characterized in that: The availability of communication resources is specifically defined as the maximum number of vehicles that the current CAV can communicate with at any time, taking into account delay and bandwidth limitations.

6. The method for communication and cooperative control of networked autonomous driving vehicles for mixed traffic flow according to claim 5, characterized in that: The loss function specifically includes: The loss function of the value network is: Where γ is the attenuation factor, v i The reward function of The loss functions of the attention network and the policy network are calculated by the following formula:

7. The method for communication and cooperative control of networked autonomous driving vehicles for mixed traffic flow according to claim 6, characterized in that: Vehicle Reward Function It includes two parts: the reward function of traffic performance and the reward function of communication performance, namely in Reflects traffic performance and is defined as Indicates v i The absolute value of the change in following distance is used to measure the stability of traffic flow; v i The driving speed at time t is used to measure the road capacity. It is to judge whether the following distance meets the minimum safety distance; Reward function for communication performance Defined as Used to evaluate the communication delay when vehicles interact with each other to perceive information Whether the delay upper limit τ of successful transmission is met max , thereby measuring the efficiency of the communication system.

Images