A communication-sensitive multi-agent collaboration method

Abstract

The present invention proposes a communication-sensitive multi-agent collaboration method, which is applied to the field of vehicle network communication technology. The method of the present invention includes: the agent uses an information filtering module to encode the locally observed message into a hidden state vector, that is, a compressed filtered message, and before sending the message, the shared medium access control module is used to score the message, and the edge computing node sorts the message value according to the score, and sends a confirmation message to the agent with the topK message value. After the agent receives the confirmation, it sends the message; the edge computing node receives all messages, uses the message distillation module to extract valid information, and summarizes and distributes it to each agent; in the process of summarizing the agent messages, the graph convolution aggregation method based on the graph information bottleneck is used to retain the agent topology structure information, and the summarized message is distilled. The present invention reduces the transmission of redundant and invalid information between agents, saves communication resources, and improves the decision-making ability and collaboration efficiency of multiple agents.

Description

Technical Field

[0001] The present invention relates to the fields of Internet of Vehicles communication technology and multi-agent collaboration technology, and in particular to a multi-agent collaboration method in a scenario where communication resources are limited. Background Art

[0002] The control system architecture of an autonomous vehicle is mainly composed of a perception module, a decision module, and a control module. The perception module is responsible for perceiving the surrounding environment. Under the conditions of the Internet of Vehicles, the vehicle acts as both a sensor and an actuator. The vehicle not only perceives the surrounding environment through on-board sensors, but also uses the Internet of Vehicles technology to interact with other vehicles or road facilities to share the perceived environmental information. In this way, a complete understanding of the surrounding environment is established in a collaborative manner. The decision module is mainly responsible for driving action planning. According to the surrounding environment information obtained by the perception module, effective information is extracted from the environmental information, the current environmental state is judged based on the effective information, action planning is made based on the environmental state, and action decisions are output. The control module is responsible for converting the action decisions output by the decision module into vehicle control instructions, where the control of the vehicle is divided into lateral control and longitudinal control. Lateral control refers to controlling the vehicle's heading by controlling the steering wheel angle, and longitudinal control refers to controlling the vehicle's speed by controlling the vehicle's throttle, gear, and brake.

[0003] In the Internet of Vehicles environment, vehicles can fully and accurately perceive the surrounding environment through V2X (Vehicle to X). The planning system of the autonomous vehicle not only needs to obtain the current status of other traffic elements through the perception module, but also needs to obtain a wider range of traffic status through V2X in order to make correct and efficient driving decision planning. On the basis of the Internet of Vehicles, the coordinated scheduling of vehicles can be realized, and the coordination of multiple intelligent vehicles based on communication can be realized. However, V2X transmits messages through wireless communication, so the transmission of messages will be limited by communication resources, and the problem of communication resource limitation needs to be considered during the transmission process. In the case of limited communication resources, the messages to be transmitted need to be filtered and compressed, and in order to improve the collaborative effect of vehicles, the intelligent vehicles need to be able to extract effective information from all received messages to assist the vehicles in making driving decisions. In the process of information filtering and compression, the vehicle needs to be able to distinguish which information is valid and which information is redundant from the message, so that the vehicle can retain valid information during the process of message filtering and compression, reduce the impact of information filtering and compression on intelligent vehicle communication, and ensure the effect of vehicle collaboration. At the same time, in the process of information extraction, intelligent vehicles need to be able to judge the information that is valid for the current vehicle from the large number of messages received, and extract effective information from all the messages received to assist the vehicle in making driving decisions.

[0004] Existing document 1 (Chinese invention patent with publication number CN110471297A) disclosed a multi-agent collaborative control method, system and device on November 19, 2019. By setting the interaction value function as an intrinsic incentive value function, each agent can explore influential states and behavior points more frequently; by stimulating the interaction between agents, multiple agents are encouraged to cooperate to effectively solve complex cooperative tasks. However, in this technical solution, multiple agents do not have spontaneous communication behavior during the collaboration process, and there is no instant communication of effective information between multiple agents. A single agent cannot obtain effective observation information of other agents in the current environment to assist the current agent in making decisions. This will prevent the agent from making the optimal action decision that can improve the global reward.

[0005] Existing document 2 (Chinese invention patent with publication number CN109726903A) disclosed a distributed multi-agent collaborative decision-making method based on the attention mechanism on May 7, 2019. The distributed multi-agent observation information is integrated based on the attention mechanism, and the relationship between local information is autonomously discovered to construct global decision information for a single agent. However, in this technical solution, the limitation of communication resources is not taken into account in the process of communication between agents. The communication between two agents in the real environment is limited by the current communication resources. The communication between agents does not meet the bandwidth limitation in the current environment or the number of communicating agents is too large, which will make it impossible for the agents to communicate effectively and the agents will not be able to receive correct information. This will prevent the agent from making the optimal action decision that meets the current environmental state.

[0006] Existing document 3 (Chinese invention patent application with publication number CN108921298A) disclosed a reinforcement learning multi-agent communication and decision-making method on November 30, 2018, in which the state characteristics of all agents are input as communication information into the VLAD (local aggregation description vector) layer for soft allocation and clustering, and the clustered communication information is distributed to each agent, and each agent aggregates its own state characteristics with the received clustered communication information to make action decisions, thereby improving the decision-making level of the agent. However, in this technical solution, the agent makes action decisions based on the received clustered information and aggregates its own state characteristics. This will make it impossible to perform targeted information aggregation according to the environment in which a specific agent is located during the information aggregation process, and extract more effective information for the current agent. At the same time, the limitation of communication resources in real scenarios is not considered during the information transmission process. This will cause the agent to receive insufficient effective information or redundant information, and it will be unable to make the optimal action decision that conforms to the current environmental state.

[0007] Existing document 4 (Chinese invention patent application with publication number CN112434792A) disclosed a reinforcement learning algorithm for collaborative communication and control of multi-agent systems on March 2, 2021. It provides a reinforcement learning algorithm for multi-agent systems that send and receive messages for information sharing through a communication network with a certain topology structure, so that the multi-agent system can build communication strategies and control strategies on each agent through training. However, in this technical solution, the communication between agents is based on a fixed topology structure, and the agents cannot communicate freely. The agents cannot send messages selectively, which will prevent valid information for a specific agent from being sent directly to the specific agent, and cannot prevent the agent from sending redundant information to overwhelm valid information. The effectiveness of the information received by the agent will be limited, which is not conducive to the agent making the optimal action decision based on the received information.

[0008] In summary, the existing technology lacks consideration for the limitations of communication resources in real scenarios, resulting in insufficient effective information received by the intelligent agent and inability to make optimal action decisions. Therefore, it is necessary to further consider the collaborative communication method of vehicles when communication resources are limited, so as to improve the efficiency of vehicles in obtaining effective information and making better action decisions. Summary of the invention

[0009] In view of the problems existing in the above-mentioned prior art, the present invention takes into account the communication situation of intelligent vehicles under limited communication resources, and provides a communication-sensitive multi-agent collaboration method, which performs information filtering based on information bottlenecks to meet bandwidth limitations, performs information distillation based on a multi-head attention mechanism to assist the intelligent agent in extracting valid information, and scores and controls the sending of the generated information based on a shared medium access control module to reduce the occupation of communication resources by invalid messages and prevent valid messages from being overwhelmed at the receiving end.

[0010] A communication-sensitive multi-agent collaboration method of the present invention comprises the following steps: in an autonomous driving scenario, vehicles are dispersed in a collaboration area as agents, and a multi-agent communication module is set for each agent. The multi-agent communication module comprises an information filtering module, a message distillation module, and a shared medium access control module.

[0011] Step 1: Use the variational autoencoder VAE to build an information filtering module. The information filtering module encodes the message vector of the local observation obtained by the input agent into a hidden state vector and sends it out as a compressed and filtered message. When training, the information filtering module satisfies the constraint of the message vector variance.

[0012] Step 2: Before sending a message, the agent uses the shared medium access control module to score the message and send it out; an edge computing node is set in the environment, which receives the scores sent by all agents, sorts the message values ​​according to the scores, and sends confirmation information to the agents with the topK message values. After receiving the confirmation information, the agent sends the compressed and filtered message out;

[0013] The score generation layer of the shared medium access control module is a fully connected network. Agent i i The generated message score will be s i Expressed as: s i =Sigmoid(m i W i ), where W i is the parameter matrix of the fully connected network;

[0014] Step 3: The edge computing node receives messages sent by all agents and uses the message distillation module to extract valid information from all received messages. The message distillation module generates three types of vectors based on the multi-head attention mechanism, namely query vector, target vector and value vector. The attention weights are calculated based on the three types of vectors, and the weighted messages of the received messages are calculated to obtain valid information. After extracting the valid information, the edge computing node summarizes it and distributes it to each agent in the scene.

[0015] During the communication process of the intelligent agents, graph convolution communication is used to retain the topological structure of the communication process and summarize the messages. At the same time, the graph information bottleneck is introduced to distill the summarized messages and topological structures of the intelligent agents. The loss function Loss for distilling the topological structure is 拓扑 as follows:

[0016] Loss 拓扑 =KL[P 邻接节点汇总分布 ,P 目标汇总分布 ]

[0017] Among them, P 目标汇总分布 represents the target distribution of the sampling distribution of adjacent nodes when the agent performs message aggregation, P 邻接节点汇总分布 Represents the actual distribution of the sampling distribution of adjacent nodes when the agent performs message aggregation.

[0018] Compared with the prior art, the advantages and positive effects of the present invention are:

[0019] (1) The method of the present invention uses the edge computing node to confirm which agents send messages to reduce the number of agents sending messages, saving communication resources and reducing the interference of redundant information on the extraction of effective information. At the same time, during the communication process, the agents use graph convolutional communication to aggregate messages and retain the topological structure of the communication process. The process of message propagation in the graph will converge into a distribution rather than a specific vector, which helps the communication system to deal with the problems of message interference and message loss in unreliable channels and improves the robustness of the collaborative communication process. In addition, during the communication process, the agents also use the graph information bottleneck method to distill the aggregated messages and topological structures to discard invalid information parts and invalid graph topological structure information, thereby improving the decision-making ability and collaborative efficiency of multiple agents.

[0020] (2) Compared with the prior art, in which the default message in the distributed multi-agent collaboration process is that it can be transmitted losslessly between agents, the present invention takes into account the problem of limited communication resources between agents in the actual multi-agent collaboration scenario, and solves the bandwidth limitation and media access contention problems in the communication resource limited scenario. For bandwidth limitation, the method of the present invention filters and compresses the message before sending it to meet the bandwidth limitation; the information filtering module will identify the valid information in the message during the information filtering and compression process, so as to retain the valid information. In order to enable the agents to communicate more efficiently, the present invention designs a message distillation module based on the multi-head attention mechanism to help the agents extract valuable information from all received messages. The multi-head attention mechanism can make the agents pay attention to multiple features and extract more abundant valid features. The method of the present invention also scores the sent messages through the shared media access control module to assist the edge computing node in confirming which agents send messages, save communication resources, and reduce the interference of redundant information on the extraction of valid information, so as to improve the decision-making ability and collaboration level of the agents. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a framework diagram for realizing a multi-agent collaboration method of the present invention. DETAILED DESCRIPTION

[0022] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0023] In actual autonomous driving scenarios, vehicles are dispersed in collaborative areas as intelligent agents. In actual communication processes, because communication resources are limited, it is necessary to solve bandwidth limitations and media access contention problems. The communication-sensitive multi-agent collaboration method provided by the present invention installs a multi-agent communication module in each agent to perform information vector filtering, message distillation, and shared media access, thereby ensuring that vehicles can perform efficient multi-agent collaboration in scenarios with limited communication resources and improving the efficiency of multi-agent collaborative communication.

[0024] like Figure 1 As shown, the communication-sensitive multi-agent collaboration method provided by the present invention sets an information filtering module, a shared medium access control module and a message distillation module in each vehicle agent. The implementation of the present invention is described in three steps below.

[0025] The first step is to implement an information filtering method based on information bottleneck, namely, an information filtering module. The present invention implements the information filtering module based on the VAE method, and the information filtering module converts the local observation values ​​obtained by the intelligent agent into i , i.e. the message vector, encoded as the hidden state vector h i , compress the message and output the compressed message m i To communicate with other agents.

[0026] In the process of multi-agent collaboration, the local observation information of each agent is shared through communication between multiple agents. The agent makes action decisions based on the local observation information and the received information, and makes the decision more optimal for the global situation by receiving the local observation information of other agents. Communication resources are limited, so the message transmission process will be subject to bandwidth restrictions. The transmitted message needs to be compressed to meet the required bandwidth restrictions in order to effectively transmit the message. The information vector needs to be compressed and filtered before transmission, and the information vector will cause the problem of incomplete effective information after compression. In order to enable the vehicle to obtain effective information under the communication bandwidth restriction, it is necessary to design an effective information vector compression and filtering algorithm so that the transmitted information vector meets the communication bandwidth restriction and retains effective information during the compression and filtering process. The information filtering module will directly affect the performance of the multi-agent system because the agent's decision is based on the received message and its own local observation. Therefore, the quality of compression and filtering directly determines the quality of the received message. In order to ensure performance under bandwidth restrictions, valuable information in the message must be saved in the filtered message.

[0027] In order to ensure that the compressed message can meet the bandwidth limit, the relationship between the bandwidth limit and the message vector used in the communication must be analyzed. Based on the source coding theorem and the maximum data rate, the relationship between the bandwidth and entropy of the message can be constructed.

[0028] The following is a brief introduction to the derivation process with reference to relevant papers.

[0029] The present invention constructs the relationship between bandwidth limitation and transmission message based on information entropy. When the bandwidth limitation is B and the signal level is K, the information entropy H(M) of message M is less than or equal to 2Blog2K / n, where n is the transmission rate, indicating that the communication system transmits n symbols per second. According to the maximum entropy principle, Gaussian distribution has the maximum information entropy. If the random variable X is normally distributed, and its mean μ and variance σ are the same as the message vector M, then the information entropy of the normally distributed variable is Therefore, the information entropy of a message M with a mean of μ and a variance of σ is less than or equal to In summary, the conditions for transmitting messages to meet bandwidth constraints and the maximum entropy of transmitted messages can be written as:

[0030]

[0031]

[0032] Therefore, if H(X) is less than Then H(M) will satisfy the bandwidth constraint.

[0033] In this limited bandwidth scenario, the important question is how to train the information filtering module so that it can compress the variance of the information to a value less than the variance in the constraint. The key issue in information compression filtering is how to retain valuable information in the message during the compression process. The present invention applies the method in VAE in the information filtering module. VAE uses hidden state vectors and reparameterization techniques to improve the representation and generation capabilities of hidden state vectors. For example, if X is the input of VAE and Y is the output of VAE, the purpose of VAE is to maximize the mutual information of X and Y, because VAE generates Y based on the input of X. The hidden state is the intermediate state in the process of generating Y. The hidden state H is generated by X, and Y is generated by H. VAE compresses the hidden variable vector H to a certain mean and variance during training. In this process, VAE will learn to preserve valuable information while removing redundant information in the hidden state vector H. Applying the VAE algorithm to the communication of intelligent agents can improve the representation ability of compressed and filtered messages. During the training process, the intelligent agent can learn an effective method of compressing and filtering messages to meet the bandwidth limitation.

[0034] The limited bandwidth constraint is converted into a constraint on the variance of the message vector. Based on the relationship between the entropy of the message vector and the bandwidth limit, the variance limit of the message vector can be calculated. The information filtering module compresses the message vector according to the variance limit so that it meets the bandwidth limit requirements. The information filtering module has a fully connected layer to fit the mean and variance of each communication message vector. Each message vector has a separate distribution that is related to the observation of the agent. Therefore, each piece of information can focus on conveying valuable information in the current environment, making communication more efficient, thereby improving the overall reward of all agents. During training, an information compression loss term is added to the entire loss function. The information filtering module uses the KL divergence between the message vector distribution and the target distribution as the information compression loss term to filter and compress the message vector. After calculating the target distribution, according to VAE, if the target message vector distribution is N(μ 目标 , σ 目标 ), then in order to achieve information filtering, the following loss function term can be added:

[0035] Loss 过滤 =KL(N(μ,σ)||N(μ 目标 ,σ 目标 ))

[0036] Among them, μ and σ represent the mean vector and variance vector of the communication message respectively.

[0037] The core function of the information filtering module is to retain valuable information in the transmission message vector while filtering and compressing information to meet bandwidth constraints, thereby maintaining the collaborative ability of multiple agents. This is a trade-off between the collaborative ability of multiple agents and information filtering and compression. If the information is not filtered and compressed, the agent will obtain more information about the environment, thereby making better decisions and obtaining better multi-agent collaborative effects. During the communication process, the agent will perform multiple rounds of communication on the graph topology of the traffic agent, so that the agent can obtain information directly from neighboring nodes or from neighboring nodes with multiple hops away. However, after multiple rounds of message propagation and aggregation, the message will contain a large amount of redundant information and cannot communicate directly. In order to meet the bandwidth constraints, the information must be filtered and compressed. In order to meet the bandwidth constraints and prevent invalid information from drowning valid information, it is very important to effectively filter and compress the information. In the process of filtering and compressing the information, valid information is retained while discarding worthless information, thereby effectively utilizing communication resources and transmitting valid information between agents. The information filtering module of the present invention causes the information to lose the least valid information during the compression and filtering process, and retains the valid information in the filtered and compressed message.

[0038] The second step is to implement the scoring-based shared media access control method, namely the shared media access control module. An edge computing node is set up in the environment. The edge computing node is also an intelligent agent, which can be a vehicle in the scene.

[0039] The communication resources that can be used in the process of communication between agents are limited. The message transmission between agents will occupy limited communication resources. If all agents send a message to other agents at each time step, then when the agents share the communication medium, the probability of transmission conflicts between messages will increase. In order to avoid the problem of conflicts when transmitting messages on a shared transmission medium, the carrier sense multiple access (CSMA) protocol is used to avoid conflicts between transmitted messages. CSMA is a media access control protocol in which a node verifies whether there are other messages being transmitted before transmitting a message on a shared transmission medium. Therefore, if the conflict rate is high, the agent will wait for a long time before sending a message, because the agent can only send a message when the transmission medium is idle. In addition, some messages are worthless to other agents. Transmitting worthless messages will waste communication resources, and receiving too many redundant messages will drown out valuable messages. In order to reserve communication resources for more important messages, the method of the present invention will score the message value using a shared medium access control module before preparing to transmit the message, and decide whether to transmit the message to other agents. The shared medium access control module can be specifically described as s i =f s (m i ), where f s is the score generation layer, s i Agent i is based on message m i The score generation layer is a fully connected network. The encoded message vector is used as the input of the fully connected network. The fully connected network learns how to evaluate the value of the message, thereby generating the message score corresponding to the message. The message sending behavior decision is made based on the message score. If the message is worthless, the agent will not transmit the message to other agents. The overall mathematical representation of the shared medium access control module for agent i is:

[0040] s i =Sigmoid(m i W i )

[0041] Among them, s i is the decision score of the access control action, m i is the encoded message vector, W i is the parameter matrix of the learnable fully connected network.

[0042] In the method of the present invention, the edge computing node receives the messages sent by all agents and the scores of the messages, sorts the message values ​​according to the scores, and sends confirmation information to the agents with topK message values. After receiving the confirmation information, the agent sends the compressed and filtered message. In this way, the global observation information can be comprehensively considered, and only valuable information vector agents will send it, which saves communication resources, reduces the interference of redundant information on the extraction of effective information, and improves the decision-making ability and coordination level of the agent.

[0043] The third step is to implement the information distillation method based on the multi-head attention mechanism, namely the message distillation module. The agent uses the information filtering module to compress and filter the observed message vector and then sends it out. After other agents receive the message, they use the message distillation module to extract valid information from all the received messages.

[0044] Multi-agent collaboration shares local observations through a communication system, allowing the agents to obtain more global information. The quality of information transmission between agents directly affects the multi-agent collaboration effect. At the same time, how to effectively process received messages is also very important for the multi-agent collaboration effect. Because each agent is in a different position in the environment, the agents face different environments. Therefore, the importance of each message from other agents is different for each agent, and the agents will receive a lot of messages. In order to improve the ability of agents to process received messages and improve the collaborative effect of multi-agents, the present invention designs a message distillation module. The message distillation module can extract valid information for the current agent from all received messages according to the environment in which the agent is located, so as to prevent too much redundant information from affecting the agent's action decision during the decision-making process.

[0045] The message distillation module of the present invention extracts effective information from the received message based on the multi-head attention mechanism. In the process of information extraction, three types of vectors are generated, namely query vector, target vector and value vector. The present invention uses the same network structure to generate three types of vectors, and different parameters are learned during the training process to generate different vectors. The formula for generating vectors is:

[0046] v=mW

[0047] Where v is the generated vector, m is the encoded message vector, and W is the learnable parameter matrix.

[0048] The query vector is based on the message m of agent i itself i The generated one can be described as where f q () is the query vector generation layer, is the message m from agent i iThe target vector can be used to calculate the attention weight of each message. The target vector is the output of the target vector generation layer. The target vector generation layer takes the received message as input and outputs the target vector corresponding to the received message. It can be described as where f t () is the target vector generation layer, m k is the message of agent k, According to the message m k Generated target vector. In order to calculate the attention weight, the message distillation module will calculate the query vector of the current agent i The target vector of the message received by the current agent from agent k The attention weight α of agent i receiving the message i It can be calculated by the following formula:

[0049]

[0050] Where N is the number of agents, d m represents the message dimension, α ik represents the weight of agent i receiving message k.

[0051] The value vector is a preliminary extraction of the received message and can be described as where f v () is the value vector generation layer. The message obtained by agent i through the message distillation module can be obtained by weighting the value vector with the calculated weight. Agent i can extract message c through the following formula i ,as follows:

[0052]

[0053] At the same time, the present invention introduces graph convolution communication in the process of multi-agent communication, further retains the topological structure of the overall communication process, can improve the overall collaborative performance in scenarios where decisions have local dependencies, and can improve the robustness of the collaborative process. In order to perform efficient information distillation, a graph information bottleneck method is also introduced, and information distillation and aggregation are performed when the agent node receives the message through the graph information bottleneck. Compared with the traditional information bottleneck, the graph information bottleneck not only adds an information bottleneck in the information dimension, but also adds an information bottleneck in the topological structure. The information bottleneck of the information dimension is mainly used to filter and compress the information vector in the process of multi-layer graph convolution communication. In the process of graph convolution communication, after multiple rounds of graph convolution communication, the information vector propagated in the graph will become too bloated due to multiple information aggregations, and cannot continue to propagate in the graph, affecting the efficiency of graph convolution communication. Therefore, it is necessary to add a graph information bottleneck in the process of graph convolution communication, filter and compress the messages propagated in the graph, ignore the invalid information part in the information vector, and retain the valuable information part to continue to propagate in the graph. Through the graph information bottleneck, the mutual information of the messages in the graph nodes before and after multiple rounds of graph convolutional communication is reduced under the premise of maximizing the overall collaborative reward, and the nodes in the graph convolutional communication are encouraged to only gather and disseminate valuable necessary information for auxiliary decision-making, so that only compact information vectors are propagated in the graph. For the information bottleneck of the topological structure, it is mainly used to filter and compress the topological structure information, discard invalid graph topological structure information, and only retain the most compact topological structure information to assist the intelligent agent in making decisions. The loss function term Loss for distilling the topological structure is 拓扑 as follows:

[0054] Loss 拓扑 =KL[P 邻接节点汇总分布 ,P 目标汇总分布 ]

[0055] Among them, P 目标汇总分布 represents the target distribution of the sampling distribution of adjacent nodes when the agent performs message aggregation, P 邻接节点汇总分布 Represents the actual distribution of the sampling distribution of adjacent nodes when the agent performs message aggregation.

[0056] At the same time, as messages propagate in the graph, they will converge into distributions rather than specific vectors, which will help the communication system cope with message interference and message loss in unreliable channels and improve the robustness of the collaborative communication process.

[0057] In order to reduce the computational load of the agent, the present invention uses a message distillation module on the edge computing node in the scene to identify effective information, summarize it and distribute it to the agent in the scene. The agent can directly use the messages distributed by the received edge computing node to assist in decision-making. The way in which the edge computing node identifies effective information is the same as above.

[0058] Experimental description and experimental results:

[0059] The multi-vehicle cooperative communication system was tested at a traffic intersection without traffic lights. The vehicles were restricted by limited communication resources during the communication process. During the test, the success rate, communication rate and waiting time of vehicles passing through the traffic intersection without collision were mainly recorded. These three indicators show the cooperative performance of the current multi-vehicle cooperative communication system in the scenario of limited communication resources. The success rate indicates whether the multi-vehicle cooperative communication system can effectively carry out cooperative scheduling between multiple vehicles. A high success rate indicates that the multi-vehicle cooperative communication system can effectively communicate between vehicles, so that vehicles can obtain valuable information to assist vehicles in making driving decisions. The communication rate indicates whether the vehicle can reduce the sending of worthless messages and avoid worthless messages occupying communication resources. The waiting time indicates whether the vehicle can efficiently pass through the traffic intersection and improve the traffic efficiency of the vehicle through the multi-vehicle cooperative scheduling system. In order to test the multi-vehicle cooperative scheduling system in a more complex environment, the test environment includes four traffic intersections. The vehicles in the four traffic intersections can communicate with each other and make driving decisions in a coordinated manner. In this environment, the vehicle will receive a large number of messages from other vehicles, among which there is a large amount of redundant information. The vehicle needs to efficiently extract effective information to assist in driving decisions. At the same time, in this environment, the transmission control module is also necessary, because the transmission control module can reduce the sending of worthless messages, reduce the occupation of limited communication resources, and reduce the number of messages received by the receiving vehicle, so that the receiving vehicle can extract effective information faster for assisting driving decisions. During the communication process between vehicles, the bandwidth of communication between vehicles is limited. The filtering and compression module can effectively filter and compress the information vector to meet the communication bandwidth limit, and retain effective information in the information vector during the filtering and compression process.

[0060] The embodiment of the present invention uses 20 vehicles to simulate the performance of the method of the present invention in the environment of a traffic intersection, and compares it with IC3Net and IMAC. During the training process, the vehicle addition rate will increase with the increase of training rounds. The multi-vehicle collaborative scheduling system using the method of the present invention can maintain an average success rate during the training process, which shows that the multi-vehicle collaborative scheduling system has learned the basic rules of collaborative scheduling and compressed and filtered the message vector. Compared with IC3Net, the success rate during the training period (from 0 to 1950 rounds) is very close, so the information filtering module does not lose valid information during the filtering and compression of the information vector. During the period of limiting the communication bandwidth (1951 to 2000 rounds), a higher success rate can be obtained, which is higher than other methods, indicating that the multi-vehicle collaborative scheduling system using the method of the present invention can perform effective information filtering and compression, meet the communication bandwidth limit, and perform effective communication between vehicles. At the same time, during the evaluation process, the communication rate of the method of the present invention is lower than that of other methods, indicating that the method of the present invention can reduce unnecessary communication and reserve communication resources for valuable information.

Claims

1. A communication-sensitive multi-agent collaboration method, characterized in that: In the autonomous driving scenario, vehicles are dispersed in the collaborative area as agents, and a multi-agent communication module is set for each agent. The multi-agent communication module includes an information filtering module, a message distillation module and a shared medium access control module. The method includes the following steps: Step 1: Use the variational autoencoder VAE to build an information filtering module. The information filtering module encodes the message vector of the local observation obtained by the input agent into a hidden state vector and sends it out as a compressed and filtered message. When training, the information filtering module satisfies the constraint of the message vector variance. Step 2: Before sending a message, the agent uses the shared medium access control module to score the message and send it out; an edge computing node is set in the environment, which receives the scores sent by all agents, sorts the message values ​​according to the scores, and sends confirmation information to the agents with the topK message values. After receiving the confirmation information, the agent sends the compressed and filtered message out; The score generation layer of the shared medium access control module is a fully connected network. The encoded message vector is used as the input of the fully connected network. The fully connected network learns how to evaluate the message value, thereby generating the message score corresponding to the message. The message sending behavior decision is made based on the message score. The agent i is assigned a message vector m according to the message vector m. i The decision score s of the generated access control action i Expressed as: s i =Sigmoid(m i W i ), where W i is the parameter matrix of the fully connected network; Step 3: The edge computing node receives messages sent by all agents, and uses the message distillation module to extract valid information from all received messages. The message distillation module generates three types of vectors based on the multi-head attention mechanism, namely query vector, target vector and value vector, calculates attention weights based on the three types of vectors, calculates weighted messages of received messages, and obtains valid information. After extracting the valid information, the edge computing node summarizes it and distributes it to each agent in the scene. During the communication process of the agents, graph convolution communication is used to retain the topological structure of the communication process, summarize the messages, and introduce the graph information bottleneck to distill the summarized messages and the topological structure. The loss function Loss for distilling the topological structure is 拓扑 as follows: Loss 拓扑 =KL[P 邻接节点汇总分布 ,P 目标汇总分布 ] Among them, P 目标汇总分布 represents the target distribution of the sampling distribution of adjacent nodes when the agent performs message aggregation, P 邻接节点汇总分布 Represents the actual distribution of the sampling distribution of adjacent nodes when the agent performs message aggregation.

2. The method according to claim 1, characterized in that In the step 1, the information filtering module compresses the message vector according to the variance limit so that it meets the bandwidth limit requirement; Assume that the target message vector follows the distribution N(μ 目标 ,σ 目标 ), the KL divergence between the message vector distribution N(μ,σ) and the target message vector distribution is taken as the information compression loss Loss 过滤 as follows: Loss 过滤 =KL(N(μ,σ)||N(μ 目标 ,s 目标 )), Among them, μ and σ represent the mean vector and variance vector of the communication message respectively.

3. The method according to claim 1, characterized in that: In step 3, the message distillation module generates a query vector based on its own message, generates a corresponding target vector based on the message received from each agent, and calculates the attention weight based on the query vector and the target vector of each agent; The value vector is a preliminary extraction of the received message, and the value vector is weighted by the calculated attention weight to obtain effective information; Assume that the query vector calculated by agent i is The target vector calculated for the received N agent messages is Then the attention weight α of agent i receiving the message i Calculated as: Among them, d m Indicates the message, α ik represents the weight of agent i receiving message k.

Images