Method and device for optimizing vehicle-to-vehicle communication

Abstract

The present application provides a method and device for optimizing inter-vehicle communication, which includes: when the target vehicle is in the process of automatic driving, detecting the real-time traffic environment through the on-board sensors on the target vehicle; when it is determined that a detour is required based on the real-time traffic environment, receiving vehicle information sent by surrounding vehicles through the C‑V2X on-board communication module on the target vehicle; determining the real-time traffic situation and the state of surrounding vehicles based on the real-time traffic environment and vehicle information; determining the optimal communication strategy based on the real-time traffic situation and the state of surrounding vehicles using the QMIX algorithm; and dynamically optimizing the communication network of the target vehicle based on the optimal communication strategy. The method and device can adaptively optimize the communication network based on the real-time traffic environment and driving requirements, reduce invalid communications and redundant information transmission, and significantly improve the communication efficiency of the entire vehicle network.

Description

Technical Field

[0001] The present application relates to the field of vehicle communication technology, and more specifically, to a method and device for optimizing inter-vehicle communication. Background Art

[0002] At present, with the continuous development of intelligent transportation systems, vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) technologies have become increasingly mature, providing the possibility of close coordination between people, roads and vehicles. Vehicle-to-vehicle communication allows vehicles to exchange driving information in real time, thereby effectively avoiding collisions, optimizing driving routes and improving traffic efficiency. However, in practice, it is found that the existing vehicle-to-vehicle communication (V2V) and vehicle-to-infrastructure communication (V2I) are usually based on dedicated short-range communication (DSRC) or cellular vehicle-to-everything (C-V2X) technology. The topology of the communication network between vehicles lacks adaptability and cannot be dynamically adjusted in real time according to factors such as road traffic conditions, vehicle density, and driving trajectory changes, resulting in uneven distribution of communication resources and potential low communication efficiency. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a method and device for optimizing inter-vehicle communications, which can adaptively optimize the communication network according to the real-time traffic environment and driving requirements, reduce invalid communications and redundant information transmission, and significantly improve the communication efficiency of the entire vehicle network.

[0004] The first aspect of the present application provides a method for optimizing inter-vehicle communication, comprising:

[0005] When the target vehicle is in the process of automatic driving, the real-time traffic environment is detected by the on-board sensor of the target vehicle;

[0006] When it is determined that a detour is required according to the real-time traffic environment, the C-V2X vehicle-mounted communication module on the target vehicle receives vehicle information sent by surrounding vehicles through the C-V2X vehicle-mounted communication module;

[0007] Determine the real-time traffic situation and the state of surrounding vehicles according to the real-time traffic environment and the vehicle information;

[0008] According to the real-time traffic conditions and the surrounding vehicle status, the optimal communication strategy is determined using the QMIX algorithm; wherein the optimal communication strategy at least includes a communication parameter adjustment strategy, a communication connection adjustment strategy, and a communication topology adjustment strategy;

[0009] The communication network of the target vehicle is dynamically optimized according to the optimal communication strategy.

[0010] In the above implementation process, the method can adaptively optimize the communication network according to the real-time traffic environment and driving requirements, reduce invalid communication and redundant information transmission, and significantly improve the communication efficiency of the entire vehicle network.

[0011] Further, after dynamically optimizing the communication network of the target vehicle according to the optimal communication strategy, the method further includes:

[0012] Obtaining the current communication quality and the task completion status of the optimal communication strategy;

[0013] The QMIX algorithm is updated according to the current communication quality and the task completion status.

[0014] Further, the determining of the optimal communication strategy using the QMIX algorithm according to the real-time traffic situation and the surrounding vehicle status includes:

[0015] Establishing a QMIX algorithm; wherein the QMIX algorithm includes a DRQN ​​neural network and a hybrid network;

[0016] Constructing a state space vector of each of the surrounding vehicles according to the surrounding vehicle states;

[0017] Calculating the Q value of each of the surrounding vehicles through the DRQN ​​neural network and the state space vector;

[0018] Calculating a global joint Q value through the hybrid network, the Q value of each of the surrounding vehicles and the global state information;

[0019] An argmax operation is performed according to the global joint Q value to determine the optimal communication strategy.

[0020] Furthermore, the DRQN ​​neural network at least includes an MLP input layer, a GRU recurrent layer, an MLP output layer, a strategy π function and a random exploration ε function;

[0021] All values ​​of parameters of each network module in the hybrid network are non-negative numbers.

[0022] Further, the Q value of each of the surrounding vehicles is calculated by the DRQN ​​neural network and the state space vector, including:

[0023] integrating the state space vectors into a high-dimensional vector;

[0024] Extracting features from the high-dimensional vector through the MLP input layer to obtain a feature vector and a hidden state at a previous time step;

[0025] Determining a current hidden state according to the feature vector and the hidden state of the previous time step through the GRU cycle layer;

[0026] The Q value of each of the surrounding vehicles is calculated according to the current hidden state through the MLP output layer.

[0027] A second aspect of the present application provides an inter-vehicle communication optimization device, the inter-vehicle communication optimization device comprising:

[0028] A detection unit, used to detect the real-time traffic environment through a vehicle-mounted sensor on the target vehicle when the target vehicle is in the process of automatic driving;

[0029] A receiving unit, configured to receive vehicle information sent by surrounding vehicles through the C-V2X vehicle-mounted communication module through the C-V2X vehicle-mounted communication module on the target vehicle when it is determined that a detour is required according to the real-time traffic environment;

[0030] A determination unit, configured to determine the real-time traffic situation and the state of surrounding vehicles according to the real-time traffic environment and the vehicle information;

[0031] A strategy determination unit, configured to determine an optimal communication strategy using a QMIX algorithm according to the real-time traffic situation and the surrounding vehicle status; wherein the optimal communication strategy at least includes a communication parameter adjustment strategy, a communication connection adjustment strategy, and a communication topology adjustment strategy;

[0032] A communication optimization unit is used to dynamically optimize the communication network of the target vehicle according to the optimal communication strategy.

[0033] Furthermore, the inter-vehicle communication optimization device further includes:

[0034] An acquisition unit, configured to acquire current communication quality and task completion status of the optimal communication strategy after dynamically optimizing the communication network of the target vehicle according to the optimal communication strategy;

[0035] An algorithm updating unit is used to update the QMIX algorithm according to the current communication quality and the task completion status.

[0036] Furthermore, the strategy determination unit includes:

[0037] Establishing a subunit for establishing a QMIX algorithm; wherein the QMIX algorithm includes a DRQN ​​neural network and a hybrid network;

[0038] A construction subunit, configured to construct a state space vector of each of the surrounding vehicles according to the state of the surrounding vehicles;

[0039] A calculation subunit, used for calculating the Q value of each of the surrounding vehicles through the DRQN ​​neural network and the state space vector;

[0040] The calculation subunit is further used to calculate a global joint Q value through the hybrid network, the Q value of each of the surrounding vehicles and the global state information;

[0041] A determination subunit is used to perform an argmax operation according to the global joint Q value to determine an optimal communication strategy.

[0042] Furthermore, the DRQN ​​neural network at least includes an MLP input layer, a GRU recurrent layer, an MLP output layer, a strategy π function and a random exploration ε function;

[0043] All values ​​of parameters of each network module in the hybrid network are non-negative numbers.

[0044] Furthermore, the computing subunit includes:

[0045] An integration module, used for integrating the state space vector into a high-dimensional vector;

[0046] An extraction module, used for performing feature extraction on the high-dimensional vector through the MLP input layer to obtain a feature vector and a hidden state of the previous time step;

[0047] A determination module, configured to determine a current hidden state according to the feature vector and the hidden state of the previous time step through the GRU cycle layer;

[0048] A calculation module is used to calculate the Q value of each of the surrounding vehicles according to the current hidden state through the MLP output layer.

[0049] A third aspect of the present application provides an electronic device, comprising a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the inter-vehicle communication optimization method described in any one of the first aspect of the present application.

[0050] A fourth aspect of the present application provides a computer-readable storage medium storing computer program instructions, wherein when the computer program instructions are read and executed by a processor, the vehicle-to-vehicle communication optimization method described in any one of the first aspect of the present application is executed.

[0051] The beneficial effects of the present application are: the topology structure and communication parameters of the inter-vehicle communication network can be adaptively adjusted, so that the communication network can be flexibly changed according to the real-time traffic environment and driving requirements, reducing invalid communication and redundant information transmission, and significantly improving the communication efficiency of the entire vehicle network;

[0052] It can also dynamically adjust the transmission power, operating frequency and transmission rate according to the real-time communication needs and environmental conditions through the intelligent agent. Especially when high-intensity communication is not required, the transmission power can be intelligently reduced to significantly reduce the energy consumption of communication equipment. For electric vehicles and other models that rely on battery life, it can effectively reduce energy consumption and extend driving mileage.

[0053] It can also quickly respond to and adapt to new communication needs in the face of complex and changing situations such as sudden traffic incidents, road construction, and bad weather, ensuring the timely delivery of key information, thereby improving network adaptability, enhancing driving safety, and improving traffic efficiency;

[0054] It can also formulate differentiated communication strategies for different autonomous driving tasks (such as convoy cooperative driving, emergency obstacle avoidance and path planning, etc.), so that communication resources can serve specific task requirements more reasonably and efficiently, improve the success rate of task execution and overall traffic smoothness, and truly realize task-oriented communication;

[0055] It can also improve network stability and connectivity through dynamic optimization of network structure, connection strategy and communication parameters, and reduce communication interruptions and resource waste caused by frequent changes in network structure, making the entire vehicle communication network more robust and stable.

[0056] It can also optimize the agent decision-making and communication strategies through reinforcement learning algorithms, so that vehicles can make optimal communication decisions under various road conditions and mission conditions, thereby improving the overall experience of drivers and passengers, and providing strong support for the reliable operation of intelligent traffic management and autonomous driving systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0058] Figure 1 A flow chart of a method for optimizing inter-vehicle communication provided in an embodiment of the present application;

[0059] Figure 2 A flowchart of another method for optimizing inter-vehicle communication provided in an embodiment of the present application;

[0060] Figure 3 A schematic diagram of the structure of a DRQN ​​neural network provided in an embodiment of the present application;

[0061] Figure 4A schematic diagram of a QMIX algorithm architecture provided in an embodiment of the present application;

[0062] Figure 5 A schematic diagram of the structure of an inter-vehicle communication optimization device provided in an embodiment of the present application;

[0063] Figure 6 A schematic diagram of the structure of another vehicle-to-vehicle communication optimization device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0065] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0066] Example 1

[0067] Please see Figure 1 , Figure 1 A flow chart of a method for optimizing inter-vehicle communication provided in this embodiment. The method for optimizing inter-vehicle communication includes:

[0068] S101. When the target vehicle is in the process of automatic driving, the real-time traffic environment is detected by the on-board sensors of the target vehicle.

[0069] S102. When it is determined that a detour is necessary based on the real-time traffic environment, the C-V2X vehicle-mounted communication module on the target vehicle receives vehicle information sent by surrounding vehicles through the C-V2X vehicle-mounted communication module.

[0070] S103: Determine the real-time traffic situation and surrounding vehicle status based on the real-time traffic environment and vehicle information.

[0071] S104: Determine the optimal communication strategy using the QMIX algorithm according to the real-time traffic conditions and the status of surrounding vehicles.

[0072] In this embodiment, the optimal communication strategy at least includes a communication parameter adjustment strategy, a communication connection adjustment strategy, and a communication topology adjustment strategy.

[0073] S105. Dynamically optimize the communication network of the target vehicle according to the optimal communication strategy.

[0074] In this embodiment, the method is applicable to the adaptive network topology communication between vehicles. Specifically, the method can regard different autonomous driving or non-autonomous driving vehicles on the road as an intelligent agent (each intelligent agent is equipped with an on-board communication device with a certain communication range and maximum transmission power) based on the QMIX algorithm, so that when the inter-vehicle communication network is normally connected, the network structure discovery time is optimized, energy consumption is reduced, and the dynamic changes of the driving paths of the vehicles, changes in the traffic environment and the task requirements of autonomous driving can be adapted.

[0075] In this embodiment, the execution subject of the method may be a computing device such as a computer or a server, and no limitation is made in this embodiment.

[0076] In this embodiment, the execution subject of the method may also be a smart device such as a smart phone, a tablet computer, etc., which is not limited in this embodiment.

[0077] It can be seen that the implementation of the inter-vehicle communication optimization method described in this embodiment can adaptively optimize the communication network according to the real-time traffic environment and driving requirements, reduce invalid communications and redundant information transmission, and significantly improve the communication efficiency of the entire vehicle network.

[0078] Example 2

[0079] Please see Figure 2 , Figure 2 A flow chart of a method for optimizing inter-vehicle communication provided in this embodiment. The method for optimizing inter-vehicle communication includes:

[0080] S201. When the target vehicle is in the process of automatic driving, the real-time traffic environment is detected by the on-board sensors of the target vehicle.

[0081] S202. When it is determined that a detour is necessary based on the real-time traffic environment, the C-V2X vehicle-mounted communication module on the target vehicle receives vehicle information sent by surrounding vehicles through the C-V2X vehicle-mounted communication module.

[0082] In this embodiment, the method can regard each autonomous or non-autonomous vehicle on the road as an agent in a multi-agent communication reinforcement learning framework. Each agent is equipped with C-V2X vehicle communication equipment. The C-V2X module allows vehicles to exchange data in real time through two main communication modes:

[0083] Direct communication: Vehicles can communicate directly with each other without the need for base stations or other network facilities. This method is essential for timely transmission of immediate and critical safety information such as location, speed, acceleration changes (i.e. driving intention), road conditions, etc., which helps improve driving safety and prevent traffic accidents, especially for early warning in cases of limited vision or sudden situations.

[0084] Network communication: Vehicles communicate with roadside infrastructure or cloud servers through cellular networks to obtain more extensive traffic information, such as the status of traffic lights, road construction warnings, weather forecasts, etc. At the same time, they can also upload the vehicle's own data to help optimize the overall traffic flow and realize intelligent traffic management.

[0085] S203: Determine the real-time traffic situation and surrounding vehicle status based on the real-time traffic environment and vehicle information.

[0086] In this embodiment, each agent has a specific state space and observation space to describe its own state and external environment information; that is, vehicle information corresponds to the state space, while real-time traffic conditions and surrounding vehicle states correspond to the observation space.

[0087] In this embodiment, the above-mentioned state space may include multi-dimensional attributes of the agent itself, such as:

[0088] Connectivity (communication distance to other vehicles, link quality, number of neighbors, etc.);

[0089] Energy consumption (current transmit power, battery status, etc.);

[0090] Response speed (information transmission delay, instruction execution speed);

[0091] Network stability (topology change rate, connection stability score).

[0092] In this embodiment, the above observation space is used to refer to the information set that the intelligent agent can obtain in real time through vehicle-mounted sensors and communication devices, such as:

[0093] Self-state perception: its own position and driving direction, speed change, actual measurement value of spatial environment perception data, etc.

[0094] Perception of the surrounding environment: Through its own sensor detection or communication with nearby vehicles, the intelligent agent can obtain partial or full status information of surrounding vehicles, such as their position, speed, driving direction, spatial environment perception data, etc., to form a cognitive model of the local environment.

[0095] S204, establishing a QMIX algorithm; wherein the QMIX algorithm includes a DRQN ​​neural network and a hybrid network.

[0096] In this embodiment, the DRQN ​​neural network includes at least an MLP input layer, a GRU recurrent layer, an MLP output layer, a strategy π function, and a random exploration ε function;

[0097] All the values ​​of the parameters of each network module in the hybrid network are non-negative.

[0098] In this embodiment, the local Q-value function (DRQN neural network) and the hybrid network (MIX neural network) in the QMIX algorithm: Based on the QMIX algorithm, a hybrid network is designed to decompose the global reward function into the local Q-value function of each agent. The Q-value function of each agent depends only on its local observations and executable actions, and the hybrid network ensures that the combination of local Q-values ​​monotonically approaches the global Q-value. In this way, even if each agent makes decisions based only on local information, the optimization of the overall network performance can be guaranteed.

[0099] In this embodiment, the learning and decision-making process of the QMIX algorithm is as follows:

[0100] (1) Initialization:

[0101] Create an experience pool of size N and initialize the action value network of each agent Initialize the joint action-value network Q ta , Target Action Value Network and target joint action value network .

[0102] (2) Sampling and experience updating:

[0103] In a parallel simulation environment, use the action value network of each agent to sample states, actions and reward sequence . Store these data in the prioritized experience pool. Each time experience sampling is performed, the parameters of each network are updated based on the data in the experience pool.

[0104] (3) Gradient update:

[0105] Every M times the action value network of the agent and the joint action value network Q ta After the gradient of and the joint action value network Q ta The gradients of are assigned to the target action value network and target joint action value network Q ta .

[0106] In this embodiment, the training method includes the following settings:

[0107] (1) Random initialization: The agents are given different position distributions in each scenario, and some unmanned platforms in the cluster are randomly destroyed with a certain probability to simulate a possible road topology communication environment.

[0108] (2) Parameter setting: including parameter initialization, learning rate, number of training rounds, batch size, random exploration coefficient, experience pool size, neural network depth and hidden layer dimension, etc.;

[0109] (3) Training method: Through distributed sampling and centralized training, the action value network of each unmanned platform is trained using the gradient descent method. and the joint action value network Q ta Conduct training.

[0110] S205: Construct a state space vector of each surrounding vehicle according to the state of the surrounding vehicles.

[0111] S206. Integrate the state space vector into a high-dimensional vector.

[0112] S207, extracting features from the high-dimensional vector through the MLP input layer to obtain a feature vector and a hidden state of the previous time step.

[0113] S208. Determine the current hidden state through the GRU cycle layer according to the feature vector and the hidden state of the previous time step.

[0114] S209, calculate the Q value of each surrounding vehicle according to the current hidden state through the MLP output layer.

[0115] S210, calculating a global joint Q value through the hybrid network, the Q value of each surrounding vehicle and the global state information.

[0116] Please see Figure 3 , Figure 3 A schematic diagram of the structure of a DRQN ​​neural network is shown. The specific structure can be designed as follows:

[0117] (1) Constructing the input vector: The state space information of the agent itself (connectivity, energy consumption, response speed, network stability) is quantified and integrated into a high-dimensional vector Ot as the input of the DRQN ​​network.

[0118] ;

[0119] in, , , , They respectively represent the numerical representation of the connectivity, energy consumption, response speed and network structure stability characteristics at time step t.

[0120] (2) Input layer (MLP):

[0121] First, the state information of the car is integrated into a high-dimensional vector (o t ,u t−1 ).

[0122] This high-dimensional vector is input into a multi-layer perceptron (MLP) for preliminary nonlinear transformation and feature extraction of the original input features.

[0123] (3) Recurrent layer (GRU):

[0124] Then, the feature vector processed by MLP and the hidden state h of the previous time step are combined. t−1 They are fed together into a Gated Recurrent Unit (GRU).

[0125] GRU updates the hidden state h based on the current input and historical information t , this hidden state contains valuable information about past behaviors and current state.

[0126] (4) Output layer (MLP):

[0127] Finally, the updated hidden state h t This is fed into the second MLP layer, which maps the hidden state to the Q-value estimate for each possible action.

[0128] The mathematical expression can be expanded to:

[0129] ;

[0130] ;

[0131] The DRQN ​​structure can use recurrent neural networks to capture time dependencies, while using MLP for feature extraction and Q-value calculation, so as to better estimate the local Q-value function and help the agent make more intelligent decisions during reinforcement learning.

[0132] (5) Strategy π and random exploration ε:

[0133] The policy π is a function that maps states to the probability distribution of actions. In reinforcement learning, the policy π defines how the agent chooses actions in the action space (adjusting the transmission power, operating frequency, transmission rate, choosing to establish or disconnect communication connections with neighboring vehicles, and routing strategy decisions) to respond to the state of the environment. In the DRQN ​​network, the policy π is determined by the output of the network, that is, the result Q after processing by MLP and GRU.

[0134] Random exploration (ε-greedy policy) means that when selecting an action, there is a probability of ε to randomly select an action, and there is a probability of 1-ε to select an action according to the strategy π. In reinforcement learning, in order to prevent the agent from falling into a local optimal solution and being unable to further explore the environment, some randomness is usually added on the basis of the strategy π.

[0135] Through the operation of the strategy π function and the random exploration ε, the Q value of each action u in the current state is obtained .

[0136] In this embodiment, the explanation of the hybrid network (MIX neural network) is as follows:

[0137] The local Q-values ​​of all agents in their specific states are combined into a global joint Q-value Q tot , to guide the decision-making of the entire cluster. In the multi-agent reinforcement learning framework, each agent has its own local Q-value function Q i (τ i ,u i ), where τ i represents the state description of agent i, u i It represents the actions that the agent can take in this state.

[0138] The input of the MIX network model is the Q value output by the DRON network of each agent. A multi-layer fully connected network is used to finally output the joint working Q value. In order to apply centralized learning to obtain distributed strategies, all the values ​​of the parameters of each network module in the MIX hybrid network are non-negative numbers, and there is no restriction on the offset. This constraint can ensure that the monotonicity constraint is met, so that the optimal strategy of the hybrid network is the optimal strategy of each agent, that is, taking argmax of the joint action value function is equivalent to taking argmax of each local action value function. The distributed strategy is greedy through the local Q i , get the optimal action, as shown below:

[0139] ;

[0140] In order to make MIX network use system status information efficiently t , we introduced a hypernetwork structure in the design, which uses the global state information s t It is used as input to provide dynamic weight and offset adjustment for each layer of the MIX network. It is added to each layer of the MIX network and outputs the weight and offset of the hybrid network.

[0141] In order to ensure the non-negativity of the weights, a linear network with an absolute value activation function is used to ensure that the output is not negative. The same method is used for the offset but without the non-negativity constraint.

[0142] Through a two-layer network and a ReLU activation function, the offset of the last layer of the MIX network is converted into a nonlinear mapping network.

[0143] Since the status information t is mixed to Q through the hypernetworktot , rather than just as an input to the hybrid network. This has the advantage that if used as an input, then s t The coefficients of are all positive, which results in the inability to fully utilize the state information to improve system performance, which is equivalent to discarding half of the information in order to make more use of the state information s.

[0144] The final cost function of the QMIX model is:

[0145] ;

[0146] The update of the QMIX model uses the traditional DQN idea, where b represents the number of samples sampled from the experience memory, and y tot The formula is as follows:

[0147] ;

[0148] in, represents the target network, where r' represents the reward at the next moment, a' represents the action that the agent may take at the next time step, and s' is the next state. The parameters is the set of weights and biases of each layer of the target network. By continuously optimizing these parameters, the target network is updated to more accurately predict the value of different actions in various future states, thereby guiding the agent to make better decisions. tot The computational complexity of the argmax operation no longer grows exponentially with the number of agents, but grows linearly with the number of agents, greatly improving the efficiency of the algorithm.

[0149] Please see Figure 4 , Figure 4 A schematic diagram of the QMIX algorithm architecture is shown.

[0150] S211. Perform an argmax operation based on the global joint Q value to determine the optimal communication strategy.

[0151] In this embodiment, the optimal communication strategy at least includes a communication parameter adjustment strategy, a communication connection adjustment strategy, and a communication topology adjustment strategy.

[0152] S212. Dynamically optimize the communication network of the target vehicle according to the optimal communication strategy.

[0153] In this embodiment, the method has a preset action space, which mainly includes the adjustment of the working parameters of the communication equipment, the connection management with the adjacent vehicles, and the possible routing strategy decision (i.e., dynamic optimization content). The purpose of these actions is to adapt to the dynamically changing traffic environment and task requirements, optimize the communication network topology between vehicles, so as to achieve the goals of improving communication efficiency and reducing energy consumption, for example:

[0154] (1) Adjust the working parameters of communication equipment:

[0155] Transmit power: The intelligent C-V2X module can adjust the transmit power of its on-board communication equipment to adapt to different communication distances, environmental interference conditions and energy consumption requirements; when long-distance communication is required or channel conditions are poor, the transmit power may need to be increased to ensure the reliability of information transmission; when short-distance communication or channel conditions are good, the transmit power can be appropriately reduced to save energy;

[0156] Operating frequency: The C-V2X module of the intelligent body can select the most appropriate operating frequency; this may involve avoiding frequency bands with severe interference, or choosing the same or compatible frequency as neighboring vehicles to improve communication efficiency;

[0157] Transmission rate: The agent can adjust the transmission rate according to the urgency of the communication task, the amount of data, and the channel conditions to balance communication efficiency and reliability;

[0158] (2) Choose to establish or disconnect communication with nearby vehicles:

[0159] According to the current traffic conditions, task requirements and the status of communication equipment, the intelligent agent can actively choose which neighboring vehicles to establish communication connections with, or disconnect unnecessary connections to optimize the network topology and ensure that important information can be transmitted efficiently and reliably;

[0160] (3) Routing strategy decision:

[0161] In the presence of multi-hop communication (forwarding information through other vehicle hops) or when there are multiple available communication paths, the agent needs to make routing decisions and select the best communication path to send the information; this may involve considering factors such as the communication quality, latency, energy consumption, reliability of the path, as well as possible network congestion.

[0162] S213: Obtain the current communication quality and the task completion status of the optimal communication strategy.

[0163] S214: Update the QMIX algorithm according to the current communication quality and task completion status.

[0164] In this embodiment, a global reward function can be designed in the method to reflect factors such as communication efficiency, energy consumption, connectivity, task response speed, etc. of the entire vehicle network. The reward function takes into account the following aspects:

[0165] (1) Connectivity: Reward agents for maintaining or improving network connectivity and punish behaviors that cause network disconnection or information delays;

[0166] (2) Energy consumption: Encourage agents to adopt energy-saving communication strategies, such as appropriately reducing transmission power and using sleep mode.

[0167] (3) Response speed: rewards rapid response to changes in autonomous driving mission requirements, such as rapid dissemination of emergency information and timely update of path planning instructions;

[0168] (4) Network structure stability: Encourage smooth transition of network topology to avoid communication interruption or resource waste caused by frequent changes.

[0169] In this embodiment, the method can use the above four points to construct a mathematical global reward function R to quantify the communication efficiency, energy consumption, connectivity, response speed and network structure stability of the vehicle network. The following is the mathematical expression:

[0170] R=w1⋅R conn +w2⋅R energy +w3⋅R speed +w4⋅R stability ;

[0171] Among them, w1, w2, w3, and w4 are the weights of the corresponding reward items, reflecting the relative importance of these factors in the global optimization goal. These weights can be adjusted according to actual application requirements. The following is a detailed explanation of the mathematical expression of each reward item:

[0172] (1) Connectivity reward R conn

[0173] The design can be measured by indicators such as the average path length, maximum path length, and minimum connected subset size of the network:

[0174] ;

[0175] Among them, d(i, j) represents the shortest path length from vehicle i to vehicle j, N is the total number of vehicles, α and β are weight parameters, and S is the largest connected subset in the vehicle network. The negative sign indicates that the path length is as short as possible, the maximum path length penalty term avoids the occurrence of communication islands, and the connected subset reward term encourages the overall connectivity of the network.

[0176] (2) Energy consumption reward R energy

[0177] The design can be based on indicators such as the average transmission power of vehicle communication equipment and the proportion of sleep time:

[0178] ;

[0179] Among them, P i (t) is the transmission power of vehicle i at the current time t, T sleep (t) is the total time that all vehicles are in sleep mode at the current moment, T total (t) is the total time at the current moment, and γ is the weight parameter. The negative sign indicates that the lower the transmit power and the higher the sleep time ratio, the better.

[0180] (3) Response speed reward R speed

[0181] It can be measured by the speed of emergency information dissemination, the response time of path planning instruction updates, etc.:

[0182] ;

[0183] in, is the time required for emergency information to propagate from the source to vehicle i, is the time required for vehicle i to respond after receiving a new path planning instruction, and δ is a weight parameter. The positive sign indicates that the faster the propagation speed and response time are, the better.

[0184] (4) Network structure stability reward R stability

[0185] It can be measured by indicators such as the frequency of network topology changes, the number of disconnections, and the number of reconnections:

[0186] ;

[0187] Among them, ∆E(t) is the change in the network topology structure in time t (such as the addition or deletion of edges or changes in weight), E(t) is the number of edges in the network at the current moment, ∆C(t) is the number of times the connection is disconnected or reestablished in time t, C(t) is the number of connections at the current moment, ρ is the weight parameter, and T is the observation time window. The negative sign indicates that the network structure change is expected to be as small as possible.

[0188] For example, suppose a group of self-driving vehicles are driving on the road, and a traffic accident or a large vehicle is driving slowly ahead, causing the team to take a detour. At this time, the vehicle agent needs to adjust the communication strategy according to the real-time traffic environment and driving path changes. The specific example process is as follows:

[0189] (1) State assessment: Each intelligent agent detects traffic jams ahead through on-board sensors and obtains emergency information from surrounding vehicles through C-V2X direct communication.

[0190] (2) Observation update: The agent updates its observation space to understand the details of traffic congestion and the status of surrounding vehicles.

[0191] (3) Decision-making: Based on the observed information, the agent uses the QMIX algorithm to evaluate different communication strategies, including adjusting the transmission power to maintain communication with necessary vehicles or increasing the communication frequency with the vehicle ahead to obtain the latest route information.

[0192] (4) Action execution: The agent adjusts the parameters of the communication equipment, such as reducing the transmission power to save energy while increasing the communication frequency with key vehicles.

[0193] (5) Connection management: The agent may disconnect from vehicles that it no longer needs to communicate with to reduce the burden on the network.

[0194] (6) Reward feedback: Depending on the communication quality and task completion, the agent receives a reward or penalty signal to update its behavior strategy.

[0195] (7) Continuous optimization: The agent continues to learn and adjust its strategies to cope with subsequent traffic changes and task requirements.

[0196] Through this process, the inter-vehicle communication network can adaptively adjust its topology and communication parameters to improve communication efficiency, reduce energy consumption, and enhance the robustness and adaptability of the network.

[0197] In summary, the innovative points of this method include:

[0198] (1) Adaptive network topology structure: Different from the existing technology in which the inter-vehicle communication network topology is fixed or adjusted based on preset rules, this method proposes an adaptive network topology communication method based on the reinforcement learning Qmix algorithm, which can dynamically adjust the communication topology structure between vehicles according to the real-time traffic environment, driving path changes, and autonomous driving task requirements, thereby realizing the intelligentization and real-time optimization of the network structure.

[0199] (2) Dynamic communication parameter adjustment: The intelligent agent can autonomously adjust the key operating parameters of the communication equipment, including transmission power, operating frequency and transmission rate, to adapt to different communication requirements and energy consumption constraints, especially for energy-sensitive vehicles such as electric vehicles, effectively reducing unnecessary energy consumption.

[0200] (3) Flexible connection management: The intelligent agent can selectively establish or disconnect communication connections with neighboring vehicles based on current traffic conditions, task requirements, and communication equipment status, thereby enhancing the flexibility and resource utilization of the communication network and ensuring the efficient transmission of important information at critical moments.

[0201] (4) Intelligent routing strategy: When multi-hop communication or multiple paths are available, the intelligent agent can use reinforcement learning algorithms to make routing strategy decisions and select the optimal communication path, taking into account communication quality, latency, energy consumption, reliability, and network congestion, thereby improving communication efficiency and overall network performance.

[0202] (5) Global reward function design: By designing a global reward function that comprehensively reflects multiple key indicators such as communication efficiency, energy consumption, connectivity, task response speed, and network stability, the intelligent agent is guided to learn the optimal communication strategy, and balanced optimization of all aspects is ensured through weight adjustment.

[0203] In this embodiment, the execution subject of the method may be a computing device such as a computer or a server, and no limitation is made in this embodiment.

[0204] In this embodiment, the execution subject of the method may also be a smart device such as a smart phone, a tablet computer, etc., which is not limited in this embodiment.

[0205] It can be seen that the implementation of the inter-vehicle communication optimization method described in this embodiment can adaptively optimize the communication network according to the real-time traffic environment and driving requirements, reduce invalid communications and redundant information transmission, and significantly improve the communication efficiency of the entire vehicle network.

[0206] Example 3

[0207] Please see Figure 5 , Figure 5 FIG. 1 is a schematic diagram of the structure of a vehicle-to-vehicle communication optimization device provided in this embodiment. Figure 5 As shown, the inter-vehicle communication optimization device includes:

[0208] The detection unit 310 is used to detect the real-time traffic environment through the on-board sensor of the target vehicle when the target vehicle is in the process of automatic driving;

[0209] The receiving unit 320 is used to receive vehicle information sent by surrounding vehicles through the C-V2X vehicle-mounted communication module through the C-V2X vehicle-mounted communication module on the target vehicle when it is determined that a detour is required according to the real-time traffic environment;

[0210] A determination unit 330, configured to determine the real-time traffic situation and surrounding vehicle status according to the real-time traffic environment and vehicle information;

[0211] A strategy determination unit 340 is used to determine an optimal communication strategy using a QMIX algorithm according to real-time traffic conditions and surrounding vehicle states; wherein the optimal communication strategy at least includes a communication parameter adjustment strategy, a communication connection adjustment strategy, and a communication topology adjustment strategy;

[0212] The communication optimization unit 350 is used to dynamically optimize the communication network of the target vehicle according to the optimal communication strategy.

[0213] In this embodiment, the explanation of the inter-vehicle communication optimization device can refer to the description in Example 1 or Example 2, and will not be further elaborated in this embodiment.

[0214] It can be seen that the implementation of the inter-vehicle communication optimization device described in this embodiment can adaptively optimize the communication network according to the real-time traffic environment and driving requirements, reduce invalid communications and redundant information transmission, and significantly improve the communication efficiency of the entire vehicle network.

[0215] Example 4

[0216] Please see Figure 6 , Figure 6 FIG. 1 is a schematic diagram of the structure of a vehicle-to-vehicle communication optimization device provided in this embodiment. Figure 6 As shown, the inter-vehicle communication optimization device includes:

[0217] The detection unit 310 is used to detect the real-time traffic environment through the on-board sensor of the target vehicle when the target vehicle is in the process of automatic driving;

[0218] The receiving unit 320 is used to receive vehicle information sent by surrounding vehicles through the C-V2X vehicle-mounted communication module through the C-V2X vehicle-mounted communication module on the target vehicle when it is determined that a detour is required according to the real-time traffic environment;

[0219] A determination unit 330, configured to determine the real-time traffic situation and surrounding vehicle status according to the real-time traffic environment and vehicle information;

[0220] A strategy determination unit 340 is used to determine an optimal communication strategy using a QMIX algorithm according to real-time traffic conditions and surrounding vehicle states; wherein the optimal communication strategy at least includes a communication parameter adjustment strategy, a communication connection adjustment strategy, and a communication topology adjustment strategy;

[0221] The communication optimization unit 350 is used to dynamically optimize the communication network of the target vehicle according to the optimal communication strategy.

[0222] As an optional implementation, the inter-vehicle communication optimization device further includes:

[0223] An acquisition unit 360, for acquiring the current communication quality and the task completion status of the optimal communication strategy after dynamically optimizing the communication network of the target vehicle according to the optimal communication strategy;

[0224] The algorithm updating unit 370 is used to update the QMIX algorithm according to the current communication quality and task completion status.

[0225] As an optional implementation, the policy determination unit 340 includes:

[0226] Establishing subunit 341, used to establish QMIX algorithm; wherein the QMIX algorithm includes DRQN ​​neural network and hybrid network;

[0227] A construction subunit 342, configured to construct a state space vector of each surrounding vehicle according to the state of the surrounding vehicles;

[0228] A calculation subunit 343, used to calculate the Q value of each surrounding vehicle through the DRQN ​​neural network and the state space vector;

[0229] The calculation subunit 343 is further used to calculate the global joint Q value through the hybrid network, the Q value of each surrounding vehicle and the global state information;

[0230] The determination subunit 344 is used to perform an argmax operation according to the global joint Q value to determine the optimal communication strategy.

[0231] In this embodiment, the DRQN ​​neural network includes at least an MLP input layer, a GRU recurrent layer, an MLP output layer, a strategy π function, and a random exploration ε function;

[0232] All the values ​​of the parameters of each network module in the hybrid network are non-negative.

[0233] As an optional implementation, the calculation subunit 343 includes:

[0234] An integration module, used to integrate the state space vector into a high-dimensional vector;

[0235] The extraction module is used to extract features from high-dimensional vectors through the MLP input layer to obtain feature vectors and the hidden state of the previous time step;

[0236] A determination module is used to determine the current hidden state based on the feature vector and the hidden state of the previous time step through the GRU recurrent layer;

[0237] The calculation module is used to calculate the Q value of each surrounding vehicle according to the current hidden state through the MLP output layer.

[0238] In this embodiment, the explanation of the inter-vehicle communication optimization device can refer to the description in Example 1 or Example 2, and will not be further elaborated in this embodiment.

[0239] It can be seen that the implementation of the inter-vehicle communication optimization device described in this embodiment can adaptively optimize the communication network according to the real-time traffic environment and driving requirements, reduce invalid communications and redundant information transmission, and significantly improve the communication efficiency of the entire vehicle network.

[0240] An embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the inter-vehicle communication optimization method in embodiment 1 or embodiment 2 of the present application.

[0241] An embodiment of the present application provides a computer-readable storage medium storing computer program instructions. When the computer program instructions are read and executed by a processor, the vehicle-to-vehicle communication optimization method in Embodiment 1 or Embodiment 2 of the present application is executed.

[0242] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0243] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0244] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.

[0245] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0246] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0247] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. A method for optimizing inter-vehicle communication, characterized in that: include: When the target vehicle is in the process of automatic driving, the real-time traffic environment is detected by the on-board sensor of the target vehicle; When it is determined that a detour is required according to the real-time traffic environment, the C-V2X vehicle-mounted communication module on the target vehicle receives vehicle information sent by surrounding vehicles through the C-V2X vehicle-mounted communication module; Determine the real-time traffic situation and the state of surrounding vehicles according to the real-time traffic environment and the vehicle information; According to the real-time traffic conditions and the surrounding vehicle status, the optimal communication strategy is determined using the QMIX algorithm; wherein the optimal communication strategy at least includes a communication parameter adjustment strategy, a communication connection adjustment strategy, and a communication topology adjustment strategy; The communication network of the target vehicle is dynamically optimized according to the optimal communication strategy.

2. The vehicle-to-vehicle communication optimization method according to claim 1, characterized in that: After dynamically optimizing the communication network of the target vehicle according to the optimal communication strategy, the method further includes: Obtaining the current communication quality and the task completion status of the optimal communication strategy; The QMIX algorithm is updated according to the current communication quality and the task completion status.

3. The vehicle-to-vehicle communication optimization method according to claim 1, characterized in that: Determining the optimal communication strategy using the QMIX algorithm according to the real-time traffic situation and the surrounding vehicle status includes: Establishing a QMIX algorithm; wherein the QMIX algorithm includes a DRQN ​​neural network and a hybrid network; Constructing a state space vector of each of the surrounding vehicles according to the surrounding vehicle states; The Q value of each of the surrounding vehicles is calculated by the DRQN ​​neural network and the state space vector; the Q value of the surrounding vehicle is the local Q value function Q of the surrounding vehicle i (τ i ,u i ), where τ i Represents the state description of the surrounding vehicle i, u i It represents the actions that the surrounding vehicle i can take in this state; Calculating a global joint Q value through the hybrid network, the Q value of each of the surrounding vehicles and the global state information; An argmax operation is performed according to the global joint Q value to determine the optimal communication strategy.

4. The vehicle-to-vehicle communication optimization method according to claim 3, characterized in that: The DRQN ​​neural network at least includes an MLP input layer, a GRU recurrent layer, an MLP output layer, a strategy π function and a random exploration ε function; All values ​​of parameters of each network module in the hybrid network are non-negative numbers.

5. The vehicle-to-vehicle communication optimization method according to claim 4, characterized in that: The step of calculating the Q value of each of the surrounding vehicles by using the DRQN ​​neural network and the state space vector includes: integrating the state space vectors into a high-dimensional vector; Performing feature extraction on the high-dimensional vector through the MLP input layer to obtain a feature vector and a hidden state of the previous time step; Determining a current hidden state according to the feature vector and the hidden state of the previous time step through the GRU cycle layer; The Q value of each of the surrounding vehicles is calculated according to the current hidden state through the MLP output layer.

6. A vehicle-to-vehicle communication optimization device, characterized in that: The inter-vehicle communication optimization device comprises: A detection unit, used to detect the real-time traffic environment through a vehicle-mounted sensor on the target vehicle when the target vehicle is in the process of automatic driving; A receiving unit, configured to receive vehicle information sent by surrounding vehicles through the C-V2X vehicle-mounted communication module through the C-V2X vehicle-mounted communication module on the target vehicle when it is determined that a detour is required according to the real-time traffic environment; A determination unit, configured to determine the real-time traffic situation and the state of surrounding vehicles according to the real-time traffic environment and the vehicle information; A strategy determination unit, configured to determine an optimal communication strategy using a QMIX algorithm according to the real-time traffic situation and the surrounding vehicle status; wherein the optimal communication strategy at least includes a communication parameter adjustment strategy, a communication connection adjustment strategy, and a communication topology adjustment strategy; A communication optimization unit is used to dynamically optimize the communication network of the target vehicle according to the optimal communication strategy.

7. The inter-vehicle communication optimization device according to claim 6, characterized in that: The inter-vehicle communication optimization device further includes: An acquisition unit, configured to acquire current communication quality and task completion status of the optimal communication strategy after dynamically optimizing the communication network of the target vehicle according to the optimal communication strategy; An algorithm updating unit is used to update the QMIX algorithm according to the current communication quality and the task completion status.

8. The inter-vehicle communication optimization device according to claim 6, characterized in that: The strategy determination unit comprises: Establishing a subunit for establishing a QMIX algorithm; wherein the QMIX algorithm includes a DRQN ​​neural network and a hybrid network; A construction subunit, configured to construct a state space vector of each of the surrounding vehicles according to the state of the surrounding vehicles; A calculation subunit is used to calculate the Q value of each of the surrounding vehicles through the DRQN ​​neural network and the state space vector; the Q value of the surrounding vehicle is the local Q value function Q of the surrounding vehicle i (τ i ,u i ), where τ i Represents the state description of the surrounding vehicle i, u i It represents the actions that the surrounding vehicle i can take in this state; The subunit is further used to calculate a global joint Q value through the hybrid network, the Q value of each of the surrounding vehicles and the global state information; A determination subunit is used to perform an argmax operation according to the global joint Q value to determine an optimal communication strategy.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the inter-vehicle communication optimization method according to any one of claims 1 to 5.

10. A readable storage medium, characterized in that: The readable storage medium stores computer program instructions, and when the computer program instructions are read and executed by a processor, the vehicle-to-vehicle communication optimization method according to any one of claims 1 to 5 is executed.

Images