Unmanned vehicle behavior decision-making method for pedestrian-vehicle mixed road environment

Abstract

The invention discloses an unmanned vehicle behavior decision-making method for a pedestrian-vehicle mixed road environment, and belongs to the technical field of intelligent decision-making of unmanned vehicles. Comprising the following steps: acquiring historical data of different types of traffic participants; establishing graph representation of a multi-agent interaction environment, wherein the graph representation comprises a node feature matrix, an adjacent matrix and a supplementary adjacent matrix; performing traffic scene interaction feature extraction by using the supplementary adjacent matrix, and fusing the traffic scene interaction feature with the node feature matrix to obtain an updated node feature matrix; and inputting the adjacent matrix and the updated node feature matrix into a decision model to complete a behavior decision. According to the method, the edge features and the edge types are projected to the same feature space, the feature matrix containing the edges is formed after splicing, traffic scene interaction features are extracted and fused with node features for information enhancement, the problem that GAT cannot process the edge features in an interaction graph is solved, understanding of vehicle behaviors is helped, and the prediction capacity is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent decision-making of unmanned vehicles, and in particular to a behavior decision-making method of an unmanned vehicle used in a mixed road environment of people and vehicles. Background Art

[0002] The behavioral decision-making model of driverless vehicles is the core link in realizing autonomous driving. Its goal is to simulate the decision-making process of human drivers under the premise of ensuring safety in order to adapt to complex traffic environments. In actual traffic environments, there are many types of traffic participants, including vehicles, non-motor vehicles, and pedestrians. Human drivers can comprehensively consider the different characteristics of different types of traffic participants to make safe and reasonable driving behavior decisions. If driverless vehicles want to achieve decision-making capabilities similar to or even superior to those of humans, they must take into account different types of traffic participants in the traffic environment, rather than just a single type.

[0003] Graph Attention Network (GAT), as a graph neural network, provides powerful information integration and decision support capabilities for autonomous vehicles, allowing vehicles to effectively understand and respond to the complex traffic environment around them, thereby improving the performance and safety of autonomous driving systems. When an autonomous vehicle makes a lane change decision, vehicles that are closer to it may be assigned a larger weight than vehicles that are farther away. In addition, a larger attention weight is assigned to traffic participants in front of the ego vehicle and with longitudinal overlap with the ego vehicle compared to traffic participants that are already behind the ego vehicle. In GAT, this relative importance is automatically encoded as weights, and the weights are learned from successful and unsuccessful training processes.

[0004] The input of GAT usually includes a node feature matrix and an adjacency matrix, which contain the original information of each vehicle, such as speed, location, and intention. The output of GAT is the feature vector of the node, which contains both local environmental information and broader global environmental information by fusing the raw data from different sources. The output node vector mapping becomes the key information required for the unmanned vehicle to make driving decisions in the traffic environment in which it is located. After being processed by GAT, the node information has better information integration and abstraction capabilities to more effectively support the making of intelligent driving decisions.

[0005] When an autonomous vehicle makes a behavioral decision, the interaction between traffic participants is the decisive factor affecting the driving of the autonomous vehicle, so the interactive information carried by the edge cannot be ignored. However, in the process of GAT's feature aggregation and updating of the graph, the information carried by the edge in the graph is ignored and only the relationship between nodes is considered. This may cause the model to fail to fully utilize the edge features, thus affecting the ability to understand and predict vehicle behavior. Summary of the invention

[0006] In view of the defects of the prior art, the present invention designs a heterogeneous Edge Enhanced Graph Attention Network (EGAT) to update node features by aggregating information from the neighborhood, that is, to aggregate the information of traffic participants within the sensor range of the unmanned vehicle.

[0007] In order to achieve the above object, the present invention proposes a driverless vehicle behavior decision method for a mixed human-vehicle road environment, comprising the following steps:

[0008] (1) acquiring historical data of different types of traffic participants, wherein the traffic participants include motor vehicles, non-motor vehicles, and pedestrians; the historical data of the participants include the position, speed, and heading angle of the traffic participants within the past t time period;

[0009] (2) Establish a graph representation of the multi-agent interaction environment, including the node feature matrix, adjacency matrix, and supplementary adjacency matrix;

[0010] Using a gated recurrent unit (GRU) to encode the historical data of the different types of traffic participants to obtain dynamic features, and adding the dynamic features to a node feature vector to construct a node feature matrix;

[0011] Gaussian process regression (GPR) is used to represent the Gaussian velocity field to construct the adjacency matrix;

[0012] constructing a supplementary adjacency matrix using the edge feature matrix and the edge type matrix of the relationships between the different types of traffic participants;

[0013] (3) extracting interactive features of traffic scenes using the supplementary adjacency matrix and fusing it with the node feature matrix to obtain an updated node feature matrix;

[0014] (4) The adjacency matrix and the updated node feature matrix are input into the graph attention neural network to extract interaction features, and the behavioral decision is completed through the decision model.

[0015] Furthermore, the node feature matrix is ​​specifically:

[0016] GRU is used to encode the historical data of the traffic participants and extract the dynamic features of the traffic participants:

[0017]

[0018] Among them: GRU h stands for Gated Recurrent Unit GRU, represents the dynamic characteristics of traffic participant i at time t, represents the historical data of traffic participant i at time t;

[0019] Add this dynamic feature to the node feature vector to construct the node feature matrix;

[0020]

[0021] in: is the node feature vector of node i at time t, which contains the current speed information, location information, heading angle, acceleration, surrounding map information and dynamic characteristics of traffic participant i.

[0022] Furthermore, the adjacency matrix is ​​specifically:

[0023] Use GPR to establish the probability distribution of the vehicle velocity field and obtain a quantitative description of the uncertainty of vehicle motion:

[0024]

[0025] Where: A is the magnification factor, x i is the position of the ith traffic participant’s vehicle, x j is the position of the jth surrounding traffic participant, σ is the scale constant;

[0026] Take the i-th traffic participant and its surrounding traffic participant j at x i The relative speed of the position is used to describe the interaction relationship between vehicles, so the calculation method of the elements in the adjacency matrix is ​​as follows:

[0027] A t (i,j)=K(x i ,x j )K(x i ,x j )- 1 △V(x i ,x j )

[0028] ΔV(x i , x j )=V i -V j

[0029] Where: △V(x i ,x j ) is the distance between the ith traffic participant and its surrounding traffic participant j at x i Relative velocity of position, V i 、V j is the speed of the i-th and j-th traffic participants.

[0030] Furthermore, the supplementary adjacency matrix includes an edge feature matrix E f and edge type matrix E type ;

[0031]

[0032] in: and They represent the edge features and edge types from node j to node i respectively;

[0033] The edge features include the relative speed, relative position and the angle between the directions of the nodes, that is, The relative velocity can be expressed as v ij =(v jx -v ix ,v jy -v iy ), the relative position is represented by the Euclidean distance The orientation angle can be expressed as θ ij =atan2(y j -y i ,x j -x i ); x i ,y i is the position of node i, v ix 、v iy is the velocity of node i in the x and y directions;

[0034] The edge types include: motor vehicle-motor vehicle, pedestrian-pedestrian, non-motor vehicle-non-motor vehicle, motor vehicle-pedestrian, pedestrian-motor vehicle, motor vehicle-non-motor vehicle, non-motor vehicle-motor vehicle, pedestrian-non-motor vehicle and non-motor vehicle-pedestrian.

[0035] Furthermore, the step (3) is specifically as follows:

[0036] (3.1) Construct the node transformation matrix M κi and M e , where M e is the edge feature transformation matrix, M et is the edge type transformation matrix, and the feature dimension of the edge feature is d e , the characteristic dimension of the edge type is d type, the feature dimension of the target is d, and the transformation matrix dimension of the edge feature is d e ×d, the dimension of the edge type conversion matrix is ​​d type ×d;

[0037] (3.2) Perform spatial transformation on the edge features and edge types:

[0038]

[0039] in: and represent edge feature and edge type matrices respectively;

[0040] (3.3) The edge features and edge types after spatial transformation are concatenated together to form the edge feature matrix:

[0041]

[0042] in: is the edge feature after spatial transformation, is the edge type after spatial transformation, || represents the splicing operation;

[0043] (3.4) fusing the edge feature matrix with the node feature matrix to obtain a node feature matrix containing edge features;

[0044]

[0045] in: Represents the feature vector of the traffic participant j.

[0046] (3.5) Using the shared attention mechanism of a single-layer neural network, we can obtain the node feature matrix containing edge features. Get the attention coefficient:

[0047]

[0048] Where: α ij is the attention coefficient, is the transpose of the parameterized attention mechanism, represents the feature vector of traffic participant i, represents the feature matrix containing edge features, || represents a concatenation operation;

[0049] (3.6) Update the feature vector of traffic participant j according to the attention coefficient to obtain the updated node feature matrix;

[0050]

[0051] Where: σ is the sigmoid function, is the edge feature after spatial transformation, represents the feature vector of traffic participant j, || represents the concatenation operation, and W is the linear transformation matrix.

[0052] Furthermore, the step (4) is specifically as follows:

[0053] (4.1) Inputting the updated node feature matrix and adjacency matrix into the graph attention neural network to obtain the topological features of the graph;

[0054] (4.2) The topological features of the graph are input into the decision algorithm module to select the action with the greatest value in the action space as the optimal action.

[0055] Furthermore, the information in the action space is a combination of driving behaviors, as shown below:

[0056] a=[CL,GS,CR]

[0057] Among them: CL, GS, and CR represent lane change to the left lane, straight driving, and lane change to the right lane, respectively. These actions are based on semantic-level commands and are used to guide the movement of the vehicle.

[0058] Furthermore, the decision model is implemented by introducing a duel neural network based on DQN.

[0059] Beneficial effects of the present invention:

[0060] The present invention uses GRU to extract the future intentions and behaviors of traffic participants, and inputs them into the node feature vector as dynamic features to construct a node feature matrix. The edge features and edge types are projected into the same feature space, spliced ​​and fused with the node features for information enhancement, and an attention mechanism is introduced to score each node, assigning different importance to different nodes, that is, assigning different attention to different traffic participants, and obtaining an enhanced node feature matrix to better model and understand the interaction relationship between different types of traffic participants, solving the problem that GAT cannot process edge features in the interaction graph, helping to understand vehicle behavior and improving prediction capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 The present invention is a flowchart of a method for making decisions on driverless vehicle behavior in a mixed road environment with vehicles and people according to an embodiment of the present invention.

[0062] Figure 2 The figure is a flow chart of the construction process of the traffic scene interaction relationship diagram representation according to an embodiment of the present invention.

[0063] Figure 3 Schematic diagram of the types of interactive relationships among traffic participants according to an embodiment of the present invention.

[0064] Figure 4 Schematic diagram of the structure of a gated cycle unit according to an embodiment of the present invention.

[0065] Figure 5 Schematic diagram of the attention mechanism for adding edge features in an embodiment of the present invention.

[0066] Figure 6 These are average reward value curves of three DQN-based algorithms and rule-based algorithms according to the embodiments of the present invention.

[0067] Figure 7 This is a box plot of the reward value of the behavior decision algorithm according to an embodiment of the present invention. DETAILED DESCRIPTION

[0068] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0069] like Figure 1 As shown, the embodiment of the present invention provides a method for unmanned vehicle behavior decision-making in a mixed road environment with vehicles and people, and designs a heterogeneous edge-enhanced graph attention neural network to aggregate information from the neighborhood to update node features, so as to make behavior decisions for unmanned vehicles. It includes the following steps:

[0070] S101. Acquire historical data of different types of traffic participants, where the traffic participants include motor vehicles, non-motor vehicles, and pedestrians; the historical data of the participants include the position, speed, and heading angle of the traffic participants within the past t time period.

[0071] In the actual traffic environment, there are many types of traffic participants, including motor vehicles, non-motor vehicles and pedestrians. Human drivers can comprehensively consider the different characteristics of different types of traffic participants to make safe and reasonable driving behavior decisions. If unmanned vehicles want to achieve similar or even superior decision-making capabilities as humans, they must take into account different types of traffic participants in the traffic environment, rather than just a single type.

[0072] Therefore, it is necessary to obtain historical data of different types of traffic participants, which include the position, speed and heading angle (driving direction of vehicles and orientation of pedestrians) of traffic participants in the past t time.

[0073] S102. Establish a graph representation of the multi-agent interaction environment, including a node feature matrix, an adjacency matrix, and a supplementary adjacency matrix.

[0074] like Figure 2 As shown in the figure, GRU is used to encode the historical data of surrounding traffic participants and extract the dynamic features of traffic participants.

[0075]

[0076] Among them: GRU h represents the Gated Recurrent Unit GRU encoder, represents the dynamic features extracted from traffic participant i at time t, Represents the historical data of traffic participant i at time t, including the position, speed and heading angle of the traffic participant in the past t time.

[0077] Like a standard recurrent neural network, the GRU maintains a hidden state vector that is updated at each time step. The hidden state captures information about the sequence encountered, and the updated hidden state is calculated as a combination of the previous hidden state and the candidate hidden state, such as Figure 4 shown.

[0078]

[0079] h represents the hidden state of the unit, r represents the degree of forgetting of the hidden state, with a value of 0 representing “completely forgotten” and 1 representing “retained”, z represents the degree of update of the hidden state, and ο represents the element-by-element multiplication of the matrix.

[0080] This dynamic feature is input into the node feature vector to construct the node feature matrix.

[0081]

[0082] in: is the node feature vector of node i (traffic participant i) at time t, which contains the current speed information, location information, heading angle (vehicle's driving direction, pedestrian's direction), acceleration, surrounding map information and dynamic characteristics of traffic participant i.

[0083] In the process of constructing the adjacency matrix, Gaussian process regression (GPR) is used to construct the Gaussian velocity field. The application of the Gaussian velocity field allows the modeling of spatial and temporal interactions between the ego vehicle and other surrounding traffic participants within a specified spatial range. Compared with the standard method of using only relative position or velocity data to complete adjacency matrix modeling, the introduction of the Gaussian velocity field has significant advantages. GPR is a non-parametric regression method that allows the probability distribution of an unknown function to be modeled without assuming the specific form of the function in advance. When an unmanned vehicle is driving in a complex traffic environment, this unknown function represents the velocity field of the vehicle, which varies in time and space. GPR can capture the interaction effects in space and time, not just relative position or velocity data. This allows it to better adapt to the changing traffic environment, such as the cooperative behavior and traffic flow between vehicles, so that the interaction between vehicles can be more accurately quantified and analyzed.

[0084] Use GPR to establish the probability distribution of the velocity field and obtain a quantitative description of the uncertainty of the motion:

[0085]

[0086] Where: A is the magnification factor, x i is the position of the ith traffic participant’s vehicle, x j is the position of the jth surrounding traffic participant, σ is the scale constant, and K represents the probability distribution of the velocity field.

[0087] The correlation between two agents (people and cars in the scene) decays with distance. A smaller σ means faster decay, which reduces the correlation of agents farther away. Agents within the range of [0m, 6m] of the ego vehicle will affect each other. In order to take into account key interactive behaviors while saving computing resources, the embodiment of the present invention sets the scale constant to 4m.

[0088] Take the i-th traffic participant and its surrounding traffic participant j at x i The relative speed of the position is used to describe the interaction between vehicles, so the calculation method of the elements in the adjacency matrix is ​​as follows:

[0089] A t (i,j)=K(x i ,x j )K(x i ,x j ) -1 △V(x i ,x j )

[0090] ΔV(x i , x j )=V i -V j

[0091] Where: △V(x i ,x j ) is the distance between the ith traffic participant and its surrounding traffic participant j at x i Relative velocity of position, V i 、V j is the speed of the i-th and j-th traffic participants.

[0092] The relationship between traffic participants is represented by a supplementary adjacency matrix, which includes the edge feature matrix E f and edge type matrix E type .

[0093]

[0094] in: and They represent the edge features and edge types from node j to node i respectively.

[0095] The edge features include the relative speed, relative position and the angle between the directions of the nodes, that is, The relative velocity can be expressed as v ij =(v jx -v ix ,v jy -v iy ), the relative position is represented by the Euclidean distance The orientation angle can be expressed as θi j =atan2(y j -y i ,x j -x i ); x i ,y i is the position of node i, v ix 、v iy is the velocity of node i in the x and y directions.

[0096] There are differences in the types of interactions between different types of agents. In order to model and analyze this type of interaction, the previous graph with only one type of edge cannot meet the design needs. The type of edge needs to be reflected in the graph. Different types of edges, such as Figure 3 As shown, each line with an arrow represents an edge type, which includes 9 types in total: motor vehicle-motor vehicle, pedestrian-pedestrian, non-motor vehicle-non-motor vehicle, motor vehicle-pedestrian, pedestrian-motor vehicle, motor vehicle-non-motor vehicle, non-motor vehicle-motor vehicle, pedestrian-non-motor vehicle and non-motor vehicle-pedestrian.

[0097] S103, using the supplementary adjacency matrix to extract traffic scene interaction features, and integrating it with the node feature matrix to obtain an updated node feature matrix.

[0098] The edge features in the supplementary adjacency matrix include some relative measurements such as relative distance and relative speed between traffic participants. The edge type is a discrete quantity composed of one-hot encoding or other identification methods. The edge information and edge type are spatially transformed accordingly and merged with the node feature matrix to finally obtain the updated node feature matrix, corresponding to Figure 2 The relative measurement in is:

[0099] (1) Construct the node transformation matrix M κi and M e , where M e is the edge feature transformation matrix, M et is the edge type transformation matrix, and the feature dimension of the edge feature is d e , the characteristic dimension of the edge type is dtype , the feature dimension of the target is d, and the transformation matrix dimension of the edge feature is d e ×d, the dimension of the edge type conversion matrix is ​​d type ×d;

[0100] (2) Perform spatial transformation on the edge features and edge types:

[0101]

[0102] (3) The edge features and edge types after spatial transformation are concatenated together to form the edge feature matrix:

[0103]

[0104] in: is the edge feature after spatial transformation, is the edge type that has undergone spatial transformation, and || represents the splicing operation.

[0105] (4) The edge feature matrix is ​​merged with the node feature matrix to obtain a node feature matrix containing edge features:

[0106]

[0107] in: Represents the feature vector of the traffic participant j.

[0108] (5) Using the shared attention mechanism of a single-layer neural network to process the node feature matrix containing edge features Get the attention coefficient, such as Figure 5 shown.

[0109]

[0110] Where: α ij is the attention coefficient, is the transpose of the parameterized attention mechanism, represents the feature vector of traffic participant i, represents the node feature matrix containing edge features, and || represents a concatenation operation.

[0111] (6) Update the feature vector of traffic participant j according to the attention coefficient to obtain the updated node feature matrix.

[0112]

[0113] Where: σ is the sigmoid function, is the edge feature after spatial transformation, represents the feature vector of traffic participant j, || represents the concatenation operation, and W is the linear transformation matrix.

[0114] S104, inputting the adjacency matrix and the updated node feature matrix into the graph attention neural network to extract interaction features, and completing the behavior decision through the decision algorithm model.

[0115] (1) Input the updated node feature matrix and adjacency matrix into the graph attention neural network to obtain the topological features of the graph.

[0116] (2) The topological features of the graph are input into the decision model to select the action with the greatest value in the action space as the optimal action.

[0117] In order to better focus on the state with higher training value and its corresponding action, the decision model of the embodiment of the present invention is implemented by introducing a dueling neural network (DuelingDQN) based on DQN.

[0118] The information in the action space is a combination of driving behaviors, as shown below:

[0119] a=[CL,GS,CR]

[0120] Among them: CL, GS, and CR represent lane change to the left lane, straight driving, and lane change to the right lane, respectively. These actions are based on semantic-level commands and are used to guide the movement of the vehicle.

[0121] The reward function consists of rewards and penalties: average speed and intention rewards, lane change and collision penalties. The reward function is expressed as:

[0122] R=ω1R S +ω2P LC +ω3P C

[0123] Where: i Represents the weight of each item in the reward function.

[0124] R S It is a speed bonus to encourage driverless cars to drive as close to the maximum speed limit as possible to improve the efficiency of the entire traffic system. The calculation formula is:

[0125]

[0126] Where: v SV and v max They represent the speed of the driverless car and the speed limit on the current road respectively.

[0127] P LC The term represents the algorithm's hope that the driverless vehicle can complete the driving task with the least number of lane changes, that is, the lane change penalty term:

[0128] P LC=n LC

[0129] Where: n LC is the number of lane changes of the autonomous vehicle in the current time step.

[0130] P C Collision penalties to ensure system safety:

[0131] P C =n C

[0132] Where: n C is the number of collisions in the current time step. In order to ensure strict punishment for collision behavior, ω3 is set to -100 to ensure that the reward value is negative after a collision.

[0133] In order to further verify the advantages of the embodiments of the present invention, the following comparative analysis is performed using a rule-based method (Rule-Based), a graph convolutional neural network combined with a DQN method (GCN-DQN), an EGAT-based DQN fusion method, and an EGAT-DuelingDQN method (embodiment scheme of the present invention). Among them, GCN represents a graph convolutional neural network, and EGAT represents a graph attention neural network using an enhanced feature matrix in the embodiment of the present invention.

[0134] The training parameter details are shown in Table 1.

[0135] Table 1

[0136]

[0137] After completing all training work, Figure 6 The training process of each driving behavior decision algorithm is shown, with the x-axis representing the number of training rounds and the y-axis representing the average reward value obtained per round during training. The results show that both EGAT-DQN and EGAT-DuelingDQN converged to similar performance levels, with the reward value converging to around 3300. GCN-DQN also achieved convergence, but the average reward was slightly lower than the previous two algorithms. In contrast, the average reward curve of the rule-based method fluctuated continuously, with no clear convergence point, and it was still oscillating until the end of training.

[0138] The results show that introducing GCN and EGAT in the decision-making process can effectively consider the interactions between various types of traffic participants on the road, so that the decision algorithm can converge and have a higher reward value. In addition, compared with GCN-DQN, EGAT-DQN obtains higher rewards. Based on the above comparative analysis, the decision-making method integrating EGAT can more comprehensively consider the interactions between traffic participants and make more intelligent decisions.

[0139] In order to compare the robustness of rule-based methods with other GCN-DQN-based methods. Figure 7 Box plots showing the rewards of the four algorithms. The upper horizontal line (A) represents the maximum value, and the lower horizontal line (E) represents the minimum value. The middle line inside the box (C) is the median, the line above the box (B) is the 75th percentile, and the line below the box (D) is the 25th percentile. The median of the rule-based method is lower than that of the other three algorithms, indicating that the rewards of rule-based methods are mostly lower. In addition, the median of the GCN-based method is also lower than that of the two EGAT-based algorithms.

[0140] The range of the box represents the interquartile range (IQR), which is the range of the middle 50% of the data. The smaller the IQR, the smaller the variability of the data, which can be used as an indicator of robustness. In this example, the interquartile range of the three algorithms based on reinforcement learning is smaller than that of the rule-based method, indicating that the rewards based on GCN-DQN are more concentrated on the median and have stronger robustness. In addition, the median of DuelingDQN is higher than that of the DQN method by using EGAT to process the interaction relationship graph and obtain the interaction features as the input of the reinforcement learning module. The optimized DuelingDQN method in this paper shows better decision-making ability.

[0141] Table 2 shows the average test reward during the test phase. Replacing GCN with EGAT can increase the reward by about 6%, indicating that EGAT can extract and understand the interaction between vehicles more effectively than GCN. The reason why GCN cannot achieve the same effect as EGAT may be that it lacks the processing of edge information. In addition, the results show that DuelingDQN performs best in the test. It can be concluded that the combination of EGAT and DuelingDQN has a good effect in the interaction scenario.

[0142] Table 2

[0143]

[0144] After comparing multiple algorithms, it is proved that the EGAT-DuelingDQN method has the best decision-making effect, and the average reward value is 6% higher than that of the GCN-DQN method. This decision-making method can achieve safer and more reasonable behavioral decisions in interactive scenarios.

[0145] The specific embodiments of the present invention are described above in conjunction with the accompanying drawings. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A driverless vehicle behavior decision method for a mixed human-vehicle road environment, characterized in that: The steps include: (1) acquiring historical data of different types of traffic participants, wherein the traffic participants include motor vehicles, non-motor vehicles, and pedestrians; the historical data of the participants include the position, speed, and heading angle of the traffic participants within the past t time period; (2) Establish a graph representation of the multi-agent interaction environment, including the node feature matrix, adjacency matrix, and supplementary adjacency matrix; Using a gated recurrent unit (GRU) to encode the historical data of the different types of traffic participants to obtain dynamic features, and adding the dynamic features to a node feature vector to construct a node feature matrix; Gaussian process regression (GPR) is used to represent the Gaussian velocity field to construct the adjacency matrix; constructing a supplementary adjacency matrix using the edge feature matrix and the edge type matrix of the relationships between the different types of traffic participants; (3) extracting interactive features of traffic scenes using the supplementary adjacency matrix and fusing it with the node feature matrix to obtain an updated node feature matrix; (4) The adjacency matrix and the updated node feature matrix are input into the graph attention neural network to extract interaction features, and the behavioral decision is completed through the decision model.

2. The method for optimizing the behavior decision model of an unmanned vehicle according to claim 1, characterized in that: The node feature matrix is ​​specifically: GRU is used to encode the historical data of the traffic participants and extract the dynamic features of the traffic participants: Among them: GRU h stands for Gated Recurrent Unit GRU, represents the dynamic characteristics of traffic participant i at time t, represents the historical data of traffic participant i at time t; Add this dynamic feature to the node feature vector to construct the node feature matrix; in: i∈[1,n] is the node feature vector of node i at time t, which contains the current speed information, location information, heading angle, acceleration, surrounding map information and dynamic characteristics of traffic participant i.

3. The method for optimizing the behavior decision model of an unmanned vehicle according to claim 1, characterized in that: The adjacency matrix is ​​specifically: Use GPR to establish the probability distribution of the vehicle velocity field and obtain a quantitative description of the uncertainty of vehicle motion: Where: A is the magnification factor, x i is the position of the ith traffic participant’s vehicle, x j is the position of the jth surrounding traffic participant, σ is the scale constant; Take the i-th traffic participant and its surrounding traffic participant j at x i The relative speed of the position is used to describe the interaction relationship between vehicles, so the calculation method of the elements in the adjacency matrix is ​​as follows: A t (i,j)=K(x i ,x j )K(x i ,x j ) -1 ΔV(x i ,x j ) Δv(x i ,x j )=V i -V j Where: ΔV(x i ,x j ) is the distance between the ith traffic participant and its surrounding traffic participant j at x i Relative velocity of position, V i 、V j is the speed of the i-th and j-th traffic participants.

4. The method for optimizing the behavior decision model of an unmanned vehicle according to claim 1, characterized in that: The supplementary adjacency matrix contains the edge feature matrix E f and edge type matrix E type ; in: and They represent the edge features and edge types from node j to node i respectively; the edge features include the relative speed, relative position and the angle between the directions of the nodes, that is, The relative velocity can be expressed as v ij =(v jx -v ix ,v jy -v iy ), the relative position is represented by the Euclidean distance The orientation angle can be expressed as θ ij =atan2(y j -y i ,x j -x i ); x i ,y i is the position of node i, v ix 、v iy is the velocity of node i in the x and y directions; The edge types include: motor vehicle-motor vehicle, pedestrian-pedestrian, non-motor vehicle-non-motor vehicle, motor vehicle-pedestrian, pedestrian-motor vehicle, motor vehicle-non-motor vehicle, non-motor vehicle-motor vehicle, pedestrian-non-motor vehicle and non-motor vehicle-pedestrian.

5. The method for optimizing the behavior decision model of an unmanned vehicle according to claim 1, characterized in that: The step (3) is specifically: (3.1) Construct the node transformation matrix M κi and M e , where M e is the edge feature transformation matrix, M et is the edge type transformation matrix, and the feature dimension of the edge feature is d e , the characteristic dimension of the edge type is d type , the feature dimension of the target is d, and the transformation matrix dimension of the edge feature is d e ×d, the dimension of the edge type conversion matrix is ​​d type ×d; (3.2) Perform spatial transformation on the edge features and edge types: in: and represent edge feature and edge type matrices respectively; (3.3) The edge features and edge types after spatial transformation are concatenated together to form the edge feature matrix: in: is the edge feature after spatial transformation, is the edge type after spatial transformation, || represents the splicing operation; (3.4) fusing the edge feature matrix with the node feature matrix to obtain a node feature matrix containing edge features; in: Represents the feature vector of the traffic participant j. (3.5) Using the shared attention mechanism of a single-layer neural network, we can obtain the node feature matrix containing edge features. Get the attention coefficient: Where: α ij is the attention coefficient, is the transpose of the parameterized attention mechanism, represents the feature vector of traffic participant i, represents the feature matrix containing edge features, || represents a concatenation operation; (3.6) Update the feature vector of traffic participant j according to the attention coefficient to obtain the updated node feature matrix; Where: σ is the sigmoid function, is the edge feature after spatial transformation, represents the feature vector of traffic participant j, || represents the concatenation operation, and W is the linear transformation matrix.

6. The method for optimizing the behavior decision model of an unmanned vehicle according to claim 1, characterized in that: The step (4) is specifically: (4.1) Inputting the updated node feature matrix and adjacency matrix into the graph attention neural network to obtain the topological features of the graph; (4.2) The topological features of the graph are input into the decision algorithm module to select the action with the greatest value in the action space as the optimal action.

7. The method for optimizing the behavior decision model of an unmanned vehicle according to claim 6, characterized in that: The information in the action space is a combination of driving behaviors, as shown below: a=[CL,GS,CR] Among them: CL, GS, and CR represent lane change to the left lane, straight driving, and lane change to the right lane, respectively. These actions are based on semantic-level commands and are used to guide the movement of the vehicle.

8. The method for optimizing the behavior decision model of an unmanned vehicle according to claim 1, characterized in that: The decision model is implemented by introducing a duel neural network based on DQN.

Images