Vehicle platoon communication resource allocation method and device, and electronic equipment

Abstract

The embodiment of the application provides a kind of vehicle platoon communication resource allocation method, device and electronic equipment, determine the state information of current agent;State information is input into strategy Actor network, to make Actor network output corresponding resource allocation action;Determine the local reward of each agent under multiple evaluation dimensions based on the execution result of resource allocation action under current communication network;Determine the global reward of current agent under each corresponding evaluation dimension based on the local reward of each agent under multiple evaluation dimensions;Optimize corresponding Actor network based on the global reward of current agent under each evaluation dimension.The global reward is determined by the local reward of all agents under current communication network, and then the Actor network is optimized, so that the Actor network can make the optimal choice at group level when allocating resources, so that the corresponding indicators of each agent are optimal, and finally all agents achieve global cooperation optimization and local reward optimization.

Description

Technical Field

[0001] This application relates to the field of vehicle control, and specifically to a method, apparatus, and electronic device for allocating communication resources in vehicle platooning. Background Technology

[0002] With the widespread adoption of C-V2X (Cellular Vehicle-to-Everything) and the development of 5G IoT technology, more and more intelligent driving vehicles will rely on C-V2X to achieve real-time information exchange. Vehicle platooning systems are a form of vehicle-to-everything (V2X) and future intelligent driving, which helps reduce fuel consumption, improve traffic efficiency, and lower transportation costs. A vehicle platoon consists of multiple vehicles arranged continuously like train carriages, allowing multiple vehicles to drive together. This platooning reduces road congestion and V2X communication pressure. In each platoon, the leading vehicle is called the Platoon Leader (PL), and the other vehicles are called Platoon Members (PMs). Each platoon is considered an intelligent agent, and the Platoon Leader is responsible for communicating with its members and other platoons.

[0003] The C-V2X vehicle-to-everything (V2X) network faces numerous challenges in wireless resource allocation, including millisecond-level latency requirements, simultaneous access by a large number of devices, massive data transmission, complex and ever-changing wireless network interference environments, and diverse service demands. Current communication decision-making relies solely on individual agents training and learning individual metrics. While this can yield optimal individual rewards, it also leads to mutual interference between agents, making it difficult to achieve optimal cooperation at the group level. Summary of the Invention

[0004] In view of this, this application provides a method, apparatus and electronic device for allocating communication resources in vehicle platooning, in order to solve the problem that the allocation of communication resources in the prior art is difficult to achieve optimal cooperation at the group level.

[0005] In a first aspect, embodiments of this application provide a method for allocating vehicle platooning communication resources, including:

[0006] Determine the current state information of the agent;

[0007] The state information is input into the policy Actor network so that the Actor network outputs the corresponding resource allocation action.

[0008] Based on the execution results of the resource allocation actions, determine the local rewards of each agent in the current communication network under multiple evaluation dimensions;

[0009] The global reward of the current agent in each corresponding evaluation dimension is determined based on the local rewards of each agent in multiple evaluation dimensions.

[0010] The Actor network is optimized based on the global reward of the current agent in each evaluation dimension.

[0011] In one optional embodiment, determining the current state information of the agent includes:

[0012] Acquire the first communication information sent by each formation member within the current intelligent agent and the second communication information sent by the leader of other formations;

[0013] The current state information of the agent is determined based on the first communication information and the second communication information.

[0014] In one optional embodiment, determining the global reward of the current agent in each corresponding evaluation dimension based on the local rewards of each agent in multiple evaluation dimensions includes:

[0015] Obtain the state information of each agent in the current communication network;

[0016] The state information is input into the attention network to obtain the first weight parameters and the first bias parameters of each agent.

[0017] Based on the first weight parameter, the first bias parameter, and the local rewards under multiple evaluation dimensions of each agent, the global reward of the current agent under each evaluation dimension is determined.

[0018] In one optional embodiment, the Actor network corresponding to the global reward optimization of the current agent in each evaluation dimension includes:

[0019] Obtain the second weight parameters and second bias parameters for each evaluation dimension of the current agent;

[0020] The joint reward of the current agent is determined based on the second weight parameter, the second bias parameter, and the global reward corresponding to each evaluation dimension of the current agent.

[0021] The parameters of the Actor network are optimized with the joint reward as the objective.

[0022] In one alternative embodiment, the optimized Actor network is deployed in the autonomous vehicle to determine the optimal resource allocation action based on the vehicle's state information.

[0023] In one optional embodiment, the evaluation dimensions include one or more of the following combinations:

[0024] Success rate of collaborative sensing information transmission in CAMs;

[0025] Timeliness of the intelligent agent updating status information to the roadside unit;

[0026] Information transmission power.

[0027] In one optional embodiment, the status information includes one or more of the following combinations:

[0028] Channel state information between the intelligent agent and the roadside unit;

[0029] Channel state information between the formation leader and formation members;

[0030] Total interference to the channel from other intelligent agents;

[0031] The remaining size of CAMs that need to be transmitted within a specified time;

[0032] Remaining transmission time.

[0033] Secondly, embodiments of this application provide a vehicle platooning communication resource allocation device, comprising:

[0034] The determination module is used to determine the current state information of the agent;

[0035] The strategy module is used to input the state information into the strategy Actor network so that the Actor network outputs the corresponding resource allocation action;

[0036] The evaluation module is used to determine the local rewards of each agent in the current communication network under multiple evaluation dimensions based on the execution results of the resource allocation action.

[0037] The determining module is further configured to determine the global reward of the current agent in each corresponding evaluation dimension based on the local rewards of each agent in multiple evaluation dimensions;

[0038] The optimization module is used to optimize the corresponding Actor network based on the global reward of the current agent in each evaluation dimension.

[0039] Thirdly, embodiments of this application provide an electronic device, including a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to execute the method described in any of the first aspects above.

[0040] Fourthly, embodiments of this application provide a computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to perform the method described in any of the first aspects.

[0041] Fifthly, embodiments of this application provide a computer program product comprising executable instructions that, when executed on a computer, cause the computer to perform the method described in any of the first aspects.

[0042] The scheme provided in this application determines the current agent's state information; inputs the state information into a policy Actor network, causing the Actor network to output corresponding resource allocation actions; determines the local rewards of each agent in the current communication network across multiple evaluation dimensions based on the execution results of the resource allocation actions; determines the global reward of the current agent in each corresponding evaluation dimension based on the local rewards of each agent in multiple evaluation dimensions; and optimizes the corresponding Actor network based on the global rewards of the current agent in each evaluation dimension. By determining the global reward through the local rewards of all agents in the current communication network, and then optimizing the Actor network, the Actor network can make optimal choices at the group level when allocating resources, achieving optimal corresponding indicators for each agent, and ultimately achieving optimal global cooperation and optimal local rewards for all agents. Attached Figure Description

[0043] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is an example schematic diagram of a vehicle platooning communication resource allocation method provided in an embodiment of this application;

[0045] Figure 2 A flowchart illustrating a vehicle platooning communication resource allocation method provided in this application embodiment;

[0046] Figure 3 A schematic diagram illustrating another vehicle platooning communication resource allocation method provided in this application embodiment;

[0047] Figure 4 A schematic diagram illustrating another vehicle platooning communication resource allocation method provided in this application embodiment;

[0048] Figure 5 This is a schematic diagram of the structure of a vehicle platooning communication resource allocation device provided in an embodiment of this application;

[0049] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0050] To better understand the technical solution of this application, the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0051] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0052] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0053] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0054] Inter-platform information exchange mainly falls into two categories: information exchange between vehicles within a platoon and information exchange between vehicles in different platoons. Different platoons exchange information via Vehicle-to-Infrastructure (V2I) communication between the platoon leader (PL) and the roadside unit (RSU) to obtain information about the surrounding environment and rationally plan platoon routes. Vehicles within the same platoon exchange Cooperative Awareness Messages (CAMs) through Vehicle-to-Vehicle Communication (V2V) to understand the relative speeds and distances of surrounding vehicles, thereby maintaining formation and improving traffic efficiency.

[0055] The C-V2X vehicle network faces many problems such as millisecond-level delay requirements for communication, simultaneous access of a large number of devices, massive data transmission, complex and variable wireless network interference environments, and diverse service requirements, which all pose huge challenges to wireless resource allocation. Multi-Agent Reinforcement Learning (MARL), as an emerging solution, can effectively address the deficiencies of traditional reinforcement learning methods, enable effective resource collaboration between vehicle formations, handle dynamic and changing environments and nonlinear systems, and learn optimal decision-making strategies to maximize system performance. However, existing solutions based on deep reinforcement learning methods still have deficiencies.

[0056] Each formation is regarded as an agent for interaction between multiple agents. However, in order for each agent to make better decisions, it needs to know the strategies and state information of all other agents except itself. In an actual vehicle network environment, as the number of formations increases, the amount of information that each formation needs to obtain will be very large, and the communication overhead will increase significantly. In addition, each agent trains and learns individual metrics independently. Although it can obtain the optimal individual reward, there will be mutual interference between agents, and it is difficult to achieve cooperation optimization at the group level. Although existing methods already have methods with multiple evaluation functions for sub-metrics, there is still a lack of multi-agent collaborative optimization in each metric dimension, and the problem of multi-agent multi-metric collaboration has not been solved.

[0057] To address the above problems, the embodiments of this application provide a method for allocating communication resources for vehicle formations, which optimizes the Actor network with the optimal solution of the global reward to achieve the optimization of communication resource allocation. Referring to Figure 1 , the formation vehicle network system based on C-V2X has V2I and V2V links, including a Road Side Unit (RSU) and multiple vehicle formations. Let the set represent the set of vehicle queues, and each vehicle formation consists of some intelligent driving vehicles (formation members) that can communicate via V2V. Let represent the vehicles included in the jth formation, numbered sequentially starting from the Platoon Leader (PL). To perform frame-by-frame resource allocation, the time range is discretized with a time interval Δt as the minimum unit, and the positive integer The index represents the time period t. The bandwidth of the communication system is divided into multiple orthogonal sub-channels of size W, denoted by the set K = {1, 2, ..., k}. In platoon-based vehicle networking, there are two communication modes: intra-platoon communication and inter-platoon communication. Vehicles periodically exchange cooperative sensing information (CAMs) via V2V links. In inter-platoon communication mode, roadside units exchange intersection safety information and control information of each platoon with each platoon (platoon leader PL) via V2I links.

[0058] Figure 2 This is a flowchart illustrating a vehicle platooning communication resource allocation method provided in an embodiment of this application. The method can be executed by a processing device, such as... Figure 2 As shown, the method may include:

[0059] Step 201: Determine the current state information of the agent.

[0060] Each vehicle platoon (platoon leader) acts as an intelligent agent to interact with the vehicle-to-everything (V2X) environment. Each agent can acquire first communication information sent by each platoon member within its own platoon, as well as second communication information sent by the platoon leader. Based on the first and second communication information, the agent's current state information can be determined. Specifically, the state information may include: channel state information between the agent and the roadside unit. Channel state information between the formation leader and formation members The total interference of other agents on formation j on channel k at the previous moment The remaining size of CAMs that need to be transferred within the specified time. Remaining teleportation time Status information can be represented by the following functions:

[0061] Step 202: Input the state information into the policy Actor network so that the Actor network outputs the corresponding resource allocation action.

[0062] Resource allocation actions can be represented by the following functions: in, This indicates which sub-channel was selected for communication by formation j. Indicates the communication mode selection for this channel (when... When, it indicates that the formation has decided to use this sub-channel for inter-formation V2I communication; when When this is the case, it indicates that the sub-channel will be used for V2V communication within the formation. Indicates the transmission power.

[0063] In the Actor-Critic algorithm framework, the Actor network is an approximate model of the policy function, used to determine the action an agent should take given a state. In the vehicle communication resource allocation scenario, the input of the Actor network is the agent's current state information, and the output is the most suitable resource allocation action for that state. This resource allocation action is used to determine the sub-channel, communication mode, and transmit power.

[0064] Step 203: Determine the local rewards of each agent in the current communication network under multiple evaluation dimensions based on the execution results of the resource allocation action.

[0065] Each vehicle platoon acts as an agent, attempting to connect to the optimal available sub-channels, with the following metrics: (1) the platoon leader promptly distributes cooperative sensing information to subsequent platoon members; and (2) maintaining and updating communication connections with roadside units (RSUs). Evaluation dimensions set accordingly may include: the success rate of cooperative sensing information (CAM) transmission, the timeliness of agents updating status information to roadside units, and information transmission power, etc.

[0066] Optionally, the success rate of cooperative sensing information (CAM) transmission can be used as the evaluation dimension, and its corresponding local reward can be expressed as: Using the timeliness of the agent updating state information to the roadside unit as the evaluation dimension, the corresponding local reward can be expressed as: Where λ1, λ2, λ3, and λ4 are the weighting factors for each item. B represents the remaining size of the cooperative sensing information that needs to be transmitted. j This indicates the size of the cooperative sensing information that needs to be transmitted. For the timeliness of information of formation j, It is the instantaneous rate of V2I communication between vehicle platoon j and roadside unit R. H is the minimum rate for V2I communication between vehicle platoon j and roadside unit R, and H() is a function that adjusts the transmit power to the same range as other items in the reward function. Let represent the transmit power level of the formation leader j. G() is a step function representing positive gain, defined as follows: Here, G0 > 0.

[0067] Step 204: Determine the global reward of the current agent in each corresponding evaluation dimension based on the local rewards of each agent in multiple evaluation dimensions.

[0068] The processing device combines the local rewards corresponding to each agent in each evaluation dimension to obtain the global reward for that evaluation dimension. Based on the two local reward functions mentioned above, the function of the global reward can be decomposed accordingly as follows: in, Let p represent the global reward under evaluation dimension M, where p is the formation set. The second term in the global reward expression represents minimizing interference for all formations. Average interference is chosen as one element of the global reward function because each vehicle formation tends to select sub-channels and transmit power levels that cause less interference to other formations, thereby achieving optimal group cooperation and overall resource allocation.

[0069] Step 205: Optimize the corresponding Actor network based on the global reward of the current agent in each evaluation dimension.

[0070] The processing device obtains the second weight parameters and second bias parameters corresponding to the global reward of each evaluation dimension. The final joint reward can be obtained by combining the global reward of each evaluation dimension with the corresponding second weight parameters and second bias parameters. The Actor network parameters can be optimized by taking the joint reward as the target.

[0071] To achieve efficient collaboration among multiple agents and multiple indicators, and to adapt to the diverse service requirements in actual vehicle-to-everything (V2X) communication, such as low transmission latency, rapid transmission of large data packets, and low power consumption, this application's embodiments design an indicator decomposition strategy. Based on different sub-indicators (evaluation dimensions), the overall reward function is decomposed into multiple sub-reward functions. Each sub-reward function corresponds to a separate evaluation network, which is trained separately to optimize each sub-indicator. The global network is then split into multiple Mixing networks, using an attention-based mechanism to aggregate the agent's Q-value for each sub-indicator. This approach addresses the specificity of each agent, achieving optimal performance for each agent's corresponding indicator, and ultimately enabling all agents to achieve global optimal cooperation and local optimal reward.

[0072] In an optional embodiment, the output process of the aforementioned local and global rewards can be referred to Figure 3 .like Figure 3 As shown, in a vehicle platoon-based communication network, there are N agents and M evaluation metrics at time t. The local action value (Q-value) function of the m-th (m∈M) metric is: The local reward of the output is The global action value (Q-value) function for the m-th indicator is: The output global reward is

[0073] The overall evaluation network can be divided into two parts: a global evaluation network and a local evaluation network. In the local evaluation network, each agent includes one Actor network and M Critic networks, where M is the number of metrics, and each Critic network corresponds to an evaluation dimension. At each time t, the current state information of each agent j (j∈N) in the Actor network is... As input, no other agent's state or actions are required to evaluate an individual's performance. The output of an Actor network is a resource allocation action. and After inputting the Critic network for each metric, the maximum Q-value for each metric m can be obtained. Q values ​​of each indicator The input will be a global evaluation network, which contains M mixing networks, each corresponding to an evaluation dimension. The state information s of all agents will be used. t and the Q-values ​​of all agents under each metric m. As input, after summing the Q-values ​​of all agents, the global reward for each metric m is output.

[0074] When optimizing the Actor network, the processing device can determine the target value for network optimization based on the above reward function, the function expression of which can be expressed as: The immediate reward for the current resource allocation action, where γ∈[0,1] is the discount factor. For long-term expected returns. The loss function is calculated based on the basic algorithm, and gradient descent is used for multiple iterations to optimize the parameters in the Actor network. During the iterative optimization process, iteration can stop when the variance of the resource allocation action output by the Actor network converges. Alternatively, iteration can stop when the rate of change of the Actor network parameters falls below a certain threshold.

[0075] In one optional embodiment, when the processing device determines the global reward based on the local rewards of each agent, it inputs state information into the attention network to obtain the first weight parameters and first bias parameters of each agent; then, based on the first weight parameters, first bias parameters, and local rewards under multiple evaluation dimensions, it determines the global reward of the current agent under each evaluation dimension. (Refer to...) Figure 4 Attention-weighted Q-value hybrid networks combine the attention mechanism with Q-value hybrid networks, introducing a supernetwork, namely the global state s t The input is fed into a dedicated neural network θ to obtain the network's weight and bias parameters. The local rewards of each agent in each evaluation dimension are unified through a dimension reshaping module, and then the values ​​are weighted and adjusted through a value attention layer, taking into account the differences between agents and enabling a relatively accurate estimation of the joint value function. The global reward is obtained by inputting the attention-weighted Q-value of each agent. Its function expression is as follows:

[0076]

[0077] In this embodiment, the global reward function is decomposed into multiple sub-indicators (such as maximizing CAM transmission and minimizing transmission power), and each sub-indicator is trained with an independent Critic network, thereby achieving optimization of each sub-indicator. Furthermore, an attention-weighted Q-value hybrid network is used to mix the Q-values ​​of each agent, thereby achieving optimization of the corresponding indicator for each agent, realizing optimal handling of complex scenarios with multiple service requirements coexisting in vehicle communication networks.

[0078] Figure 5 This is a schematic diagram of a vehicle platooning communication resource allocation device provided in an embodiment of this application. Figure 5 As shown, the device may include:

[0079] The determination module 510 is used to determine the current state information of the intelligent agent.

[0080] The strategy module 520 is used to input state information into the strategy Actor network so that the Actor network outputs the corresponding resource allocation action.

[0081] Evaluation module 530 is used to determine the local rewards of each agent in the current communication network under multiple evaluation dimensions based on the execution results of resource allocation actions.

[0082] The determination module 510 is also used to determine the global reward of the current agent in each corresponding evaluation dimension based on the local rewards of each agent in multiple evaluation dimensions.

[0083] The optimization module 540 is used to optimize the corresponding Actor network based on the global reward of the current agent in each evaluation dimension.

[0084] Corresponding to the above embodiments, this application also provides an electronic device. Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 600 may include a processor 601, a memory 602, and a communication unit 603. These components communicate through one or more buses. Those skilled in the art will understand that the structure of the electronic device shown in the figure does not constitute a limitation on the embodiment of this application. It may be a bus topology or a star topology, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0085] The communication unit 603 is used to establish a communication channel, enabling the electronic device to communicate with other devices. It can receive user data sent by other devices or send user data to other devices.

[0086] The processor 601 serves as the control center of the electronic device, connecting various parts of the device via interfaces and lines. It executes software programs, instructions, and / or modules stored in the memory 602, and calls data stored in the memory to perform various functions and / or process data. The processor may be composed of integrated circuits (ICs), such as a single packaged IC or multiple packaged ICs with the same or different functions connected together. For example, the processor 601 may consist only of a central processing unit (CPU). In this embodiment, the CPU may have a single processing core or include multiple processing cores.

[0087] The memory 602 is used to store the execution instructions of the processor 601. The memory 602 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0088] When the execution instructions in memory 602 are executed by processor 601, the electronic device 600 is able to perform some or all of the steps in the above embodiments.

[0089] In a specific implementation, this application also provides a computer storage medium, wherein the computer storage medium may store a program, which, when executed, may include some or all of the steps of the vehicle platooning communication resource allocation method provided in this application. The storage medium may be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0090] In a specific implementation, this application also provides a computer program product, wherein the computer program product includes executable instructions, which, when executed on a computer, cause the computer to perform some or all of the steps in various embodiments of the vehicle platooning communication resource allocation method provided in this application.

[0091] This application also provides a non-transitory computer-readable storage medium that stores computer instructions that cause the computer to execute the vehicle platooning communication resource allocation method provided in this application.

[0092] The aforementioned non-transitory computer-readable storage medium may be any combination of one or more computer-readable media. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or flash memory, optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0093] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including—but not limited to—electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0094] The program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0095] Those skilled in the art will clearly understand that the techniques in the embodiments of this application can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of this application, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application or some parts of the embodiments.

[0096] The same or similar parts between the various embodiments in this specification can be referred to mutually. In particular, the device embodiments and terminal embodiments are basically similar to the method embodiments, so the description is relatively simple, and the relevant parts can be referred to the description in the method embodiments.

Claims

1. A method for allocating communication resources in vehicle platooning, characterized in that, include: Determine the current state information of the intelligent agent, which is a vehicle platoon; The state information is input into the policy Actor network so that the Actor network outputs the corresponding resource allocation action. Based on the execution results of the resource allocation actions, determine the local rewards of each agent in the current communication network under multiple evaluation dimensions; The global reward of the current agent in each corresponding evaluation dimension is determined based on the local rewards of each agent in multiple evaluation dimensions. Optimize the corresponding Actor network based on the global reward of the current agent under each evaluation dimension; The process of determining the global reward of the current agent in each corresponding evaluation dimension based on the local rewards of each agent in multiple evaluation dimensions includes: Obtain the state information of each agent in the current communication network; The state information is input into the attention network to obtain the first weight parameters and the first bias parameters of each agent. Based on the first weight parameter, the first bias parameter, and the local rewards under multiple evaluation dimensions of each agent, the global reward of the current agent under each evaluation dimension is determined. The Actor network optimized based on the global reward of the current agent in each evaluation dimension includes: Obtain the second weight parameters and second bias parameters for each evaluation dimension of the current agent; The joint reward of the current agent is determined based on the second weight parameter, the second bias parameter, and the global reward corresponding to each evaluation dimension of the current agent. The parameters of the Actor network are optimized with the joint reward as the objective.

2. The method according to claim 1, characterized in that, Determining the current state information of the agent includes: Acquire the first communication information sent by each formation member within the current intelligent agent and the second communication information sent by the leader of other formations; The current state information of the agent is determined based on the first communication information and the second communication information.

3. The method according to claim 1, characterized in that, The optimized Actor network is deployed in autonomous vehicles to determine the optimal resource allocation actions based on the vehicle's state information.

4. The method according to claim 1, characterized in that, The evaluation dimensions include one or more of the following: Success rate of collaborative sensing information transmission in CAMs; Timeliness of the intelligent agent updating status information to the roadside unit; Information transmission power.

5. The method according to claim 1, characterized in that, The status information includes one or more of the following combinations: Channel state information between the intelligent agent and the roadside unit; Channel state information between the formation leader and formation members; Total interference to the channel from other intelligent agents; The remaining size of CAMs that need to be transmitted within a specified time; Remaining transmission time.

6. A vehicle platooning communication resource allocation device, characterized in that, include: A determination module is used to determine the current state information of the intelligent agent, which is a vehicle platoon. The strategy module is used to input the state information into the strategy Actor network so that the Actor network outputs the corresponding resource allocation action; The evaluation module is used to determine the local rewards of each agent in the current communication network under multiple evaluation dimensions based on the execution results of the resource allocation action. The determining module is further configured to determine the global reward of the current agent in each corresponding evaluation dimension based on the local rewards of each agent in multiple evaluation dimensions; The optimization module is used to optimize the corresponding Actor network based on the global reward of the current agent in each evaluation dimension; The determining module is specifically used to acquire the state information of each agent in the current communication network; input the state information into the attention network to obtain the first weight parameter and the first bias parameter of each agent; and determine the global reward of the current agent in each evaluation dimension based on the first weight parameter, the first bias parameter and the local reward under multiple evaluation dimensions. The optimization module is specifically used to obtain the second weight parameters and second bias parameters of each evaluation dimension of the current agent; determine the joint reward of the current agent based on the second weight parameters, second bias parameters and global reward corresponding to each evaluation dimension of the current agent; and optimize the parameters of the Actor network with the joint reward as the target.

7. An electronic device, characterized in that, The device includes a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device performs the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform the method according to any one of claims 1 to 5.

Images