Multi-agent reinforcement learning multi-task agent decision-making method and system for crossroad without signal lamp

Abstract

The invention discloses a multi-agent reinforcement learning multi-task agent decision-making method and system for a crossroad without signal lamps, and belongs to the technical field of intelligent network connection vehicle multi-task agent decision-making control. According to the method, vehicle-road cooperation is achieved through wireless communication, an automatic driving vehicle uploads state and task data to a road side unit, the road side unit adopts a multi-task mixed attention mechanism, data are processed through a self-attention module and a cross attention module, and decision features are generated and input into a strategy network. And finally generating a longitudinal control instruction by combining an offline and online mixed sampling strategy training model, and issuing and executing the longitudinal control instruction through wireless communication. According to the method, the sensing range can be expanded, the key target can be dynamically focused, the training efficiency and the decision-making stability are improved, and the collaborative decision-making capability and the traffic safety of the automatic driving vehicle in the mixed traffic environment are remarkably enhanced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent networked vehicle multi-task agent decision control, in particular to a multi-agent reinforcement learning multi-task agent decision method and system for a no-signal intersection. BACKGROUND

[0002] With the complexity of the transportation network, the phenomenon of mixed traffic of multiple types of traffic participants in urban roads is increasingly prominent. The intersection controlled by no signal has become a key area where traffic conflicts occur frequently and the traffic efficiency is limited. Existing automatic driving decision methods mostly rely on the perception and autonomous decision of a single vehicle. In the limited visibility and complex interactive environment of the intersection, it is difficult to fully grasp the state information of the surrounding traffic participants, thereby restricting the safety of vehicle traffic and the improvement of overall traffic efficiency.

[0003] To make up for the deficiency of local perception of a single vehicle, vehicle-road cooperation technology has developed rapidly in recent years. Through the global state information provided by the road infrastructure, automatic driving vehicles can obtain a more complete environmental cognition, so as to realize cooperative operation under a unified decision control framework, which has become an important research direction of intelligent networked transportation system.

[0004] In the cooperative decision method, multi-agent reinforcement learning provides a new technical path for multi-vehicle interaction modeling and strategy optimization. The typical MADDPG algorithm solves the non-stationarity and environmental dynamic change problem to a certain extent through centralized training and distributed execution mechanism. However, when dealing with multi-task switching and dynamic changes of task priority, this kind of algorithm often has difficulty in effectively extracting key conflict target features, resulting in slow learning convergence speed and insufficient strategy generalization ability. Existing researches usually only assign weights to surrounding targets in the spatial dimension, lacking dynamic modeling of different task attention priorities, thereby limiting the application effect in high dynamic mixed traffic scenarios.

[0005] In addition, reinforcement learning still has bottlenecks in sample efficiency and training stability. The traditional experience replay mechanism adopts indiscriminate uniform sampling and fails to highlight key experiences, which can easily cause training process shock and slow convergence. Although the priority experience replay improves the utilization rate of important samples by weighting the sample temporal difference error, in the high-dimensional action space and multi-agent cooperative task, it is easy to introduce sample bias and cause strategy overfitting, thereby affecting the stability of the algorithm. Therefore, how to improve the sample utilization efficiency while ensuring sample diversity and design a hybrid sampling strategy with online and offline characteristics has become a problem to be solved.

[0006] In summary, in the typical mixed traffic application scenario of the signal-free intersection, the prior art still lacks a multi-agent decision-making method and system that can fully utilize the global state information provided by vehicle-road cooperation, dynamically focus on key tasks and target features, and balance sample utilization efficiency and learning stability, thereby achieving efficient cooperative traffic and ensuring traffic safety. SUMMARY

[0007] The purpose of the present application is to provide a multi-agent reinforcement learning multi-task agent decision-making method for signal-free intersections, which can dynamically focus on key tasks and core target features based on the global state information provided by vehicle-road cooperation, effectively improve sample utilization efficiency and learning process stability, and thus improve the cooperative decision-making efficiency and traffic safety of autonomous vehicles in mixed traffic environments.

[0008] To achieve the above purpose, the technical scheme adopted by the present application is as follows:

[0009] A multi-agent reinforcement learning multi-task agent decision-making method for signal-free intersections, comprising the following steps:

[0010] S1: The autonomous vehicle uploads the state data and task data of the vehicle to the roadside unit in real time through wireless communication;

[0011] S2: The multi-task hybrid attention mechanism of the roadside unit encodes the above data into embedded vectors of uniform dimensions, and obtains the output result through the self-attention module and the cross-attention module ;

[0012] S3: The output result is input into the action policy network, and the hybrid sampling strategy is used for training;

[0013] S4: The final action decision value is obtained and issued to the target vehicle for execution through wireless communication.

[0014] Further, in S1:

[0015] The state data includes vehicle position, speed, heading angle and environmental information;

[0016] The task data includes left turn, right turn and straight ahead.

[0017] Further, S2 includes:

[0018] S21: Encode the state and task data into embedded vectors of uniform dimensions through a multi-layer perceptron with shared weights;

[0019] S22: constructing an input of a self-attention module, calculating an attention weight, and performing multi-head fusion and normalization to obtain a self-attention vector ;

[0020] S23: constructing an input of a cross-attention module, calculating a task attention weight and multi-head fusion to obtain an output result .

[0021] Further, the embedding vector in S21 includes:

[0022] a state embedding vector , and an expression is:

[0023]

[0024] wherein, MLP is a multi-layer perceptron, is a vehicle at a time state data;

[0025] a task embedding vector , and an expression is:

[0026]

[0027] wherein is a task type of the i-th vehicle.

[0028] Further, the self-attention vector in S22 , and an expression is:

[0029]

[0030] wherein, indicates a number of the remaining vehicles in the scene, indicates an attention weight of the agent to the surrounding vehicles .

[0031] Further, the output result in S23 , and an expression is:

[0032]

[0033] wherein, indicates a number of attention heads in the cross-attention module, is a task attention weight of the vehicle i to each of the other vehicles.

[0034] Further, the training by using the hybrid sampling strategy in S3 includes:

[0035] constructing a candidate sample set;

[0036] prioritizing the samples;

[0037] injecting the latest samples and updating the action policy network and the evaluation network.

[0038] Another object of the present application is to provide a multi-agent reinforcement learning multi-task agent decision system for a signal-free intersection, which implements the multi-agent reinforcement learning multi-task agent decision method for a signal-free intersection.

[0039] a roadside decision unit: deployed at an intersection and connected with a vehicle-mounted execution unit through a V2I communication link,

[0040] The roadside decision unit receives state data uploaded by an autonomous vehicle in the intersection through V2I communication, runs a multi-task hybrid attention mechanism and a policy network to generate a longitudinal control instruction, and issues the instruction to the target vehicle through a V2I communication link.

[0041] a vehicle-mounted execution unit: mounted on an autonomous vehicle, used to upload vehicle state data to the roadside decision unit, receive, verify and execute the issued longitudinal control instruction, and realize longitudinal motion control through a vehicle control system.

[0042] Another object of the present application is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the multi-agent reinforcement learning multi-task agent decision method for a signal-free intersection.

[0043] Another object of the present application is to provide an electronic device comprising a memory for storing a computer program and a processor for running the computer program to make the electronic device execute the multi-agent reinforcement learning multi-task agent decision method for a signal-free intersection.

[0044] In summary, the present application has at least one of the following beneficial technical effects:

[0045] 1. Through the vehicle-road cooperation V2I communication architecture, global perception and centralized decision making are realized by the roadside unit, which completely breaks through the single-vehicle intelligent perception blind area and greatly improves the traffic safety in complex intersection environments.

[0046] 2. The multi-task hybrid attention mechanism is innovatively introduced, which enables the vehicle to focus on potential conflict targets in the spatial dimension and dynamically filter key objects in the task dimension, thereby realizing precise decision attention allocation and significantly optimizing multi-vehicle cooperation efficiency.

[0047] 3. The offline and online hybrid sampling strategy is adopted, which greatly improves the utilization efficiency of training samples and the convergence speed of the model through priority sorting and real-time data fusion, and enhances the robustness and environmental adaptability of the decision strategy.

[0048] 4. The system is designed for mixed traffic scenarios, only controls the networked autonomous vehicle, fully considers the current traffic situation, and has high engineering practicability and landing promotion value.

[0049] In summary, the application improves the cooperative decision efficiency and traffic safety of the autonomous vehicle in the mixed traffic environment. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 The flowchart of the application;

[0051] Figure 2 The traffic scene schematic diagram of the signal-free intersection of the application;

[0052] Figure 3 The flowchart of the multi-task hybrid attention mechanism of the application;

[0053] Figure 4 The flowchart of the hybrid sampling strategy of the application. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical scheme and advantages of the application clearer and more understandable, the application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the application and do not limit the application. In addition, the technical features involved in each embodiment of the application described below can be combined with each other as long as they do not conflict with each other.

[0055] The embodiment provides a multi-agent reinforcement learning multi-task agent decision method for a signal-free intersection, as shown in Figure 1 , comprising the following steps:

[0056] S1: The autonomous vehicle uploads the state data and task data of the vehicle to the roadside unit in real time through a wireless communication mode;

[0057] S2: The multi-task hybrid attention mechanism of the roadside unit encodes the above data into embedded vectors of uniform dimension, and obtains the output result through a self-attention module and a cross-attention module;

[0058] S3: input the output result to the action policy network, and train it by using a hybrid sampling strategy;

[0059] S4: obtain the final action decision value , and is issued to the target vehicle through wireless communication.

[0060] The following will be described in detail in conjunction with embodiments.

[0061] S1: The autonomous vehicle uploads the state data and task data of the vehicle to the roadside unit in real time through wireless communication.

[0062] In the intersection scene without signal control, the traffic participants include autonomous vehicles and human-driven vehicles. The present application fully considers the actual existence of the existing traffic driving system, avoids idealized assumptions under full automation conditions, so that the scene is closer to the real traffic environment. The autonomous vehicles in the intersection without signals interact with the roadside decision unit through wireless communication (such as V2I communication), and the autonomous vehicles travel according to the expected driving path of their respective tasks, and only perform longitudinal control within the path trajectory range.

[0063] Figure 2 An exemplary traffic scene diagram in an intersection without signals is shown. The scene adopts a centralized decision-making mode of roadside master node-vehicle slave node, wherein the roadside unit serves as the master node for performing fusion and decision generation of global state information; the intelligent networked autonomous vehicles in the intersection serve as the slave node for executing the decision instructions issued by the roadside unit.

[0064] The autonomous vehicle uploads its state data, such as vehicle position, speed, heading angle, etc., to the roadside decision unit through V2I communication, and the roadside decision unit uniformly fuses the state data from multiple vehicles to generate longitudinal decision instructions and issue them to each vehicle, realizing the extension of the perception range and unified decision control. Compared with single-vehicle perception, the perception range of the vehicle is effectively extended, and in the presence of line-of-sight obstruction or sensor blind area, the ability to predict potential conflicts in advance is significantly improved.

[0065] S2: The multi-task hybrid attention mechanism of the roadside unit encodes the above data into embedded vectors of a unified dimension, and obtains the output results through a self-attention module and a cross-attention module. As shown in Figure 3 , specifically:

[0066] S21: Encode the state and task data into embedded vectors of a unified dimension through a multi-layer perception machine with shared weights.

[0067] (1) Autonomous vehicles at unsignalized intersections interact with the roadside decision unit via V2I communication. Within the scenario, vehicles upload their local vehicle status data to the roadside unit via V2I communication. The roadside unit integrates the received data and inputs it into the multilayer perceptron, thereby representing and modeling the status information of each connected vehicle. The status data of each connected vehicle is represented as follows:

[0068]

[0069] in:

[0070] Indicates vehicle At any moment Two-dimensional position coordinates;

[0071] Indicates vehicle At any moment longitudinal velocity;

[0072] Indicates vehicle At any moment longitudinal acceleration;

[0073] Indicates vehicle At any moment The heading angle;

[0074] This refers to environmental data sensed by the vehicle's sensors, including the position and speed data of surrounding traffic participants.

[0075] vehicle At any moment Status data The input is first fed into a multilayer perceptron (MLP), which shares weights across all vehicles. The MLP encodes the original high-dimensional, heterogeneous state information into a uniform-dimensional state embedding vector. The expression is:

[0076]

[0077] MLP stands for Multilayer Perceptron. For vehicles At any moment Status data.

[0078] (2) Under the multi-task setting, different vehicles may perform different tasks (such as turning left, turning right, or going straight), and each task will have different focuses on information. After the roadside unit receives the state data uploaded by the autonomous vehicle, the multi-task hybrid attention mechanism will perform more targeted processing and feature extraction on the data from different vehicles and different tasks.

[0079] Different tasks performed by different vehicles are also encoded by a multilayer perceptron, and a task embedding vector of uniform dimension is obtained through the task encoding input module. The expression is:

[0080]

[0081] in Let i be the task type for the i-th vehicle.

[0082] S22: Construct the input for the self-attention module, calculate the attention weights, and perform multi-head fusion and normalization to obtain the self-attention vector. .

[0083] Vehicles in the scene Time Embedded features of state data As input to the query, the self-attention module matches the keys and values ​​with features common to the ego vehicle and neighboring vehicles.

[0084] The self-attention module establishes global dependencies within vehicle features and obtains vehicle attention weights through weighted calculations of queries, keys, and values. The dimensions of the query vector and key matrix are... The dimension of the value matrix is .

[0085] Self-defined vehicle features as query vectors:

[0086]

[0087] Self-vehicle and its The joint representation of each adjacent vehicle is used as the key matrix and value matrix:

[0088]

[0089] The attention weights of the ego vehicle to other vehicles are calculated, and the attention matrix is ​​obtained by scaling the dot product and normalizing it using softmax: first, the dot product is scaled... Scale, then normalize using softmax. Subsequently, use this attention matrix to... Perform a weighted summation on the th... Each attention point is used to obtain the output:

[0090]

[0091] The outputs from all attention heads are fused by a linear layer:

[0092]

[0093] The final self-attention vector is denoted as:

[0094]

[0095] where, denotes the number of other vehicles in the scene, denotes the attention weight of the agent to the surrounding vehicles, and it satisfies the normalization:

[0096]

[0097] This way, the most relevant vehicle features to the current decision are highlighted adaptively, irrelevant or interfering information is weakened, and the sensitivity of the policy network to key environmental factors is enhanced.

[0098] S23: Construct the cross-attention module input, calculate the task attention weight and multi-head fusion, and obtain the output result .

[0099] Encode the task (one-hot vector) of vehicle i into a fixed-dimensional vector , input it to the cross-attention module as the query, and match it again with the output of the self-attention module as the key and value. Establish the association between the task and the output of the self-attention module to obtain the task attention weight, and emphasize the objects and clues most relevant to the current task.

[0100] The query input for a specific task is denoted as:

[0101]

[0102] The output of the self-attention module is used as the key and value of the cross-attention module input. The dimensions of the query vector and the key matrix are , and the dimension of the value matrix is :

[0103]

[0104] Similarly, the similarity between the task query and the output of the self-attention module is calculated using the scaled dot product, scaled by , and then normalized by softmax. For the th attention head, the obtained is the task attention weight of vehicle i to each of the other vehicles:

[0105]

[0106] The output of the cross-attention module is:

[0107]

[0108] wherein, denotes the number of attention heads in the cross-attention module, is the task attention weight of vehicle i to each of the other vehicles.

[0109] S3: output the result Input the action policy network and train it using the hybrid sampling strategy;

[0110] As shown in Figure 4 , in the model training phase of the multi-task agent decision system, a hybrid sampling strategy is designed to improve the utilization efficiency of experience samples, accelerate the convergence speed of the policy and value networks, and improve the stability and robustness of decision-making.

[0111] Under the hybrid sampling strategy, we use the temporal difference error as an importance measure of experience samples and introduce it into the experience priority calculation. Before each sampling, the priority values of each sample in the experience pool are updated, and the samples are sorted according to the priority. When updating the network parameters, the multi-task agent decision system combines the offline samples sorted by priority with the online samples of the latest round to form a hybrid batch for training, thereby balancing experience diversity and real-time performance. Training using the hybrid sampling strategy includes:

[0112] 1. Construct a candidate sample set. During the data collection phase, each experience sample generated by environmental interaction can be represented as a five-tuple:

[0113]

[0114] wherein, is the current state, is the executed action, is the immediate reward, is the next state, is the absolute value of the TD error of each experience sample, indicating the estimation error of the current policy for that sample. All samples are stored in the experience replay pool D.

[0115] 2. Prioritize the samples.

[0116] In the offline sample priority sorting phase, the hybrid sampling strategy first uniformly extracts l small batches of k samples from the experience replay pool D (e.g., l = 2), denoted as:

[0117]

[0118] Combine multiple batches to get candidate set :

[0119]

[0120] According to the candidate set Calculate the sample priority of each sample in the candidate set and sort them in descending order, and keep the top k samples after sorting:

[0121]

[0122] 3. Inject the latest sample and update the action policy network and evaluation network.

[0123] In the sample priority batch Introduce the online sample produced by the latest round (i.e. the latest experience sample at the current time step), replace the last sample in the priority batch : :

[0124]

[0125] After replacement, get the final training batch :

[0126]

[0127] In this way, it is ensured that the policy network and the value network can simultaneously absorb high-quality historical experience and the latest interaction information when updating, thereby improving the adaptability to environmental changes.

[0128] S4: Get the final action decision value , and issue it to the target vehicle for execution through wireless communication.

[0129] The original action decision value generated by the policy network calculation from S3. The value is usually a continuous, normalized numerical value. After signal conversion and denormalization, the original value output by the policy network is mapped to the actual physical quantity that the vehicle controller can understand, and the final action decision value refers to the longitudinal acceleration instruction .

[0130] Physical meaning of decision value:

[0131] > 0: instruct the vehicle to accelerate;

[0132] ​​​​ = 0: instruct the vehicle to maintain current speed;

[0133] <0: instruct the vehicle to decelerate (including braking).

[0134] The roadside unit will generate the acceleration instruction Bind with the target vehicle ID, encapsulated into a specific data packet. Through the V2I communication link, in the form of broadcast or directional sending, it is issued to the target autonomous vehicle (AV) within milliseconds of delay. The vehicle unit of the target vehicle receives the instruction and performs safety verification, and the instruction is sent to the execution mechanism of the vehicle after verification. The vehicle produces corresponding longitudinal motion to complete the final link of cooperative decision-making.

[0135] Safety verification includes instruction rationality and context consistency. Instruction rationality refers to whether the acceleration value is within the physical limit and execution comfort range of the vehicle, and context consistency refers to whether the instruction conflicts with the current vehicle state (such as current speed, tire grip) and perception information.

[0136] In summary, the present application realizes global perception and centralized decision-making of roadside units through wireless communication, breaks through the single-vehicle perception blind area, and significantly improves intersection safety. The innovative introduction of a multi-task hybrid attention mechanism dynamically focuses on key targets in the spatial and task dimensions, optimizing multi-vehicle cooperation efficiency. Hybrid sampling strategy is used to prioritize training high-value experience and integrate real-time data, improving model convergence speed and decision robustness. The present application is designed for mixed traffic scenarios, controlling only connected and autonomous vehicles, and has high efficiency and practical value, effectively improving the efficiency and safety of cooperative decision-making in mixed traffic.

[0137] The embodiment also provides a multi-agent reinforcement learning multi-task agent decision-making system for a non-signalized intersection, which executes the multi-agent reinforcement learning multi-task agent decision-making method for a non-signalized intersection, and includes a roadside decision unit and a vehicle-mounted execution unit. Specifically as follows:

[0138] The roadside decision unit is deployed at the intersection and connected to the vehicle-mounted execution unit through a V2I communication link,

[0139] The roadside decision unit receives state data uploaded by autonomous vehicles in the intersection through V2I communication, runs a multi-task hybrid attention mechanism and a policy network to generate longitudinal control instructions, and issues the instructions to the target vehicle through a V2I communication link;

[0140] The vehicle-mounted execution unit is mounted on an autonomous vehicle and used to upload vehicle state data to the roadside decision unit, receive, verify and execute the issued longitudinal control instructions, and realize longitudinal motion control through a vehicle control system.

[0141] The embodiment also provides a computer readable storage medium storing a computer program, the computer program being executed by a processor to implement the method.

[0142] The embodiment also provides an electronic device including a memory and a processor, the memory being configured to store a computer program, and the processor being configured to execute the computer program to enable the electronic device to perform the method.

[0143] Embodiments of the application can be implemented in methods, systems, or computer program products. Thus, the application can be in the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the application can be in the form of a computer program product on one or more computer usable storage media (including, but not limited to, disk memory, CD-ROMs, optical storage media, etc.) embodying computer readable program code.

[0144] The application is described in reference to the flowcharts and / or block diagrams of the methods, apparatus (systems) and computer program products according to embodiments of the application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, a special purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions, which are executed via the processor of the computer or other programmable data processing apparatus, generate a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 means for carrying out the functions specified in the flowcharts and / or block diagrams.

[0145] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer readable memory produce a manufacture including an instruction means, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 means for carrying out the functions specified in the flowcharts and / or block diagrams.

[0146] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable data processing apparatus to produce a computer implemented process, so that the instructions executed on the computer or other programmable data processing apparatus provide a process for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 means for carrying out the functions specified in the flowcharts and / or block diagrams.

[0147] The content not described in detail in the specification of the present application belongs to the prior art known to the person skilled in the art. It is indicated here that the above description helps the person skilled in the art to understand the present application, but does not limit the protection scope of the present application. Any implementation of equivalent replacement, modification, improvement and deletion of the above description without departing from the essential content of the present application falls within the protection scope of the present application.

Claims

1. A multi-agent reinforcement learning multi-task agent decision-making method for traffic light-free intersections, characterized in that, Includes the following steps: S1: Autonomous vehicles upload vehicle status data and task data to the roadside unit in real time via wireless communication; S2: The multi-task hybrid attention mechanism of the roadside unit encodes the above data into a uniform-dimensional embedding vector, and obtains the output result through a self-attention module and a cross-attention module. ; S3: Output the result Input the action policy network and train it using a hybrid sampling strategy; S4: Obtain the final action decision value It is then transmitted to the target vehicle for execution via wireless communication.

2. The multi-agent reinforcement learning multi-task agent decision-making method for traffic light-free intersections according to claim 1, characterized in that, In S1: The status data includes vehicle position, speed, heading angle, and environmental information; The task data includes left turn, right turn, and straight ahead.

3. The multi-agent reinforcement learning multi-task agent decision-making method for traffic light-free intersections according to claim 1, characterized in that, S2 include: S21: Encode state and task data into a uniform-dimensional embedding vector using a multilayer perceptron with shared weights; S22: Construct the input for the self-attention module, calculate the attention weights, and perform multi-head fusion and normalization to obtain the self-attention vector. ; S23: Construct the input for the cross-attention module, calculate the task attention weights and multi-head fusion, and obtain the output result. .

4. The multi-agent reinforcement learning multi-task agent decision-making method for traffic light-free intersections according to claim 3, characterized in that, The embedding vectors mentioned in S21 include: State embedding vector The expression is: MLP stands for Multilayer Perceptron. For vehicles At any moment Status data; Task embedding vector The expression is: ,in Let i be the task type for the i-th vehicle.

5. A multi-agent reinforcement learning multi-task agent decision-making method for traffic light-free intersections according to claim 3, characterized in that, The self-attention vector described in S22 The expression is: ,in, This indicates the number of other vehicles in the scene. Represents intelligent agents surrounding vehicles Attention weights.

6. The multi-agent reinforcement learning multi-task agent decision-making method for traffic light-free intersections according to claim 3, characterized in that, The output results described in S23 The expression is: ,in, This indicates the number of attention heads in the cross-attention module. It represents the task attention weight of vehicle i for each of the other vehicles.

7. A multi-agent reinforcement learning multi-task agent decision-making method for traffic light-free intersections according to claim 1, characterized in that, The training using a hybrid sampling strategy described in S3 includes: Construct a candidate sample set; Prioritize the samples; Inject the latest samples and update the action policy network and evaluation network.

8. A multi-agent reinforcement learning multi-task agent decision-making system for traffic light-free intersections, characterized in that, The system executes a multi-agent reinforcement learning multi-task agent decision-making method for traffic light-free intersections as described in any one of claims 1-7, including: Roadside decision-making unit: Deployed at the intersection and connected to the vehicle-mounted execution unit via a V2I communication link. The roadside decision unit receives status data uploaded by autonomous vehicles within the intersection via V2I communication, runs a multi-task hybrid attention mechanism and policy network to generate longitudinal control commands, and sends them to the target vehicle via the V2I communication link. Onboard execution unit: mounted on the autonomous vehicle, used to upload vehicle status data to the roadside decision unit, and to receive, verify and execute the issued longitudinal control commands, thereby realizing longitudinal motion control through the vehicle control system.

9. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements a multi-agent reinforcement learning multi-task agent decision-making method for traffic light-free intersections according to any one of claims 1-7.

10. An electronic device, characterized in that, It includes a memory and a processor, wherein the memory is used to store computer programs, and the processor runs the computer programs to enable the electronic device to perform the multi-agent reinforcement learning multi-task agent decision-making method for traffic light-free intersections as described in any one of claims 1-7.

Images