Vehicle trajectory prediction method and device, electronic equipment and storage medium

Abstract

The invention provides a vehicle trajectory prediction method and device, electronic equipment and a storage medium, and belongs to the technical field of artificial intelligence. Local trajectory features of each vehicle node are extracted from historical moving trajectory data of each vehicle node; performing global feature extraction in combination with the historical moving trajectory data corresponding to each vehicle node to obtain global trajectory features, and performing spatial feature extraction on the motif matrix, the corresponding local trajectory features and the global trajectory features for each vehicle node to obtain target spatial features of the vehicle node; performing time feature extraction on the motif matrix and the global trajectory feature to obtain a target time feature, and combining the historical moving trajectory data, the local trajectory feature, the target spatial feature and the target time feature corresponding to each vehicle node through a preset graph neural network to obtain a target time feature; and predicting the predicted moving trajectory of the target vehicle node in the traffic network graph structure. The accuracy of vehicle trajectory prediction can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a vehicle trajectory prediction method, device, electronic device, and storage medium. Background Art

[0002] In intelligent transportation systems, vehicle trajectory prediction technology is often used to optimize driving routes and manage traffic flow, which can improve road utilization while ensuring driving safety. In related technologies, vehicle trajectory prediction is performed only using road condition images collected by the vehicle itself, while ignoring the mutual influence between different vehicles, resulting in reduced accuracy of vehicle trajectory prediction. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to propose a vehicle trajectory prediction method, device, electronic device, and storage medium, aiming to improve the accuracy of vehicle trajectory prediction.

[0004] To achieve the above object, a first aspect of an embodiment of the present application proposes a vehicle trajectory prediction method, the method comprising:

[0005] Acquire a traffic network graph structure; wherein the traffic network graph structure includes a plurality of vehicle nodes and interaction relationships between the plurality of vehicle nodes;

[0006] Determine a motif matrix according to the vehicle nodes and the interaction relationship;

[0007] Obtaining historical running trajectory data corresponding to each vehicle node, extracting local trajectory features of each vehicle node from the historical running trajectory data of each vehicle node respectively, and performing global feature extraction by combining the historical running trajectory data corresponding to each vehicle node to obtain global trajectory features;

[0008] For each vehicle node, extracting spatial features from the motif matrix, the corresponding local trajectory features and the global trajectory features to obtain target spatial features of the vehicle node;

[0009] Performing time feature extraction on the motif matrix and the global trajectory feature to obtain target time features;

[0010] The predicted operation trajectory of the target vehicle node in the traffic network graph structure is predicted by combining the historical operation trajectory data, the local trajectory features, the target space features and the target time features corresponding to each vehicle node through a preset graph neural network.

[0011] In some embodiments, for each vehicle node, performing spatial feature extraction on the motif matrix, the corresponding local trajectory features, and the global trajectory features to obtain the target spatial features of the vehicle node includes:

[0012] Adding the elements of each row in the motif matrix to obtain a plurality of spatial motif vectors;

[0013] Performing feature extraction on each of the spatial motif vectors respectively to obtain a spatial motif feature of each of the spatial motif vectors;

[0014] For each of the vehicle nodes, the local trajectory feature and the global trajectory feature are used as query vectors, and each of the spatial motif features and the query vector are respectively subjected to multi-head attention calculation to obtain multiple initial attention features;

[0015] Performing feature fusion on all the initial attention features, the largest initial attention feature, the smallest initial attention feature, and the initial attention feature representing the median to obtain a target attention feature;

[0016] The target space feature of the vehicle node is determined according to the target attention feature and the local trajectory feature.

[0017] In some embodiments, extracting time features from the motif matrix and the global trajectory features to obtain target time features includes:

[0018] Adding the elements of each column in the motif matrix to obtain a plurality of time motif vectors;

[0019] Performing feature extraction on each of the time motif vectors respectively to obtain initial time motif features of each of the time motif vectors;

[0020] Performing feature fusion on all the initial time motif features, the largest initial time motif feature, the smallest initial time motif feature, and the initial time motif feature representing the median to obtain a target time motif feature;

[0021] Performing weight calculation according to the global trajectory features and the target time motif features to obtain the weight of each time motif vector;

[0022] The target time feature is determined according to each of the time motif vectors and the weight corresponding to the time motif vector.

[0023] In some embodiments, the preset graph neural network includes multiple network layers, and the preset graph neural network combines the historical running trajectory data, the local trajectory features, the target spatial features, and the target time features corresponding to each vehicle node to predict the predicted running trajectory of the target vehicle node in the traffic network graph structure, including:

[0024] For each of the vehicle nodes, extract the historical running trajectory data, the local trajectory features, and the target space features of the vehicle node through the first network layer to obtain the node features output by the first network layer;

[0025] Determine a first edge feature between the vehicle node and any vehicle node according to the historical running track data of the vehicle node, the historical running track data of any vehicle node other than the vehicle node, and the target time feature;

[0026] Determine a second edge feature of the vehicle node according to the historical running trajectory data of the vehicle node and the target time feature;

[0027] For each of the vehicle nodes, extracting, through the current network layer, the node features of all vehicle nodes output by the previous network layer, the first edge features between the vehicle node and each vehicle node other than the vehicle node, and the second edge features of all vehicle nodes, to obtain the node features of the vehicle node output by the current network layer;

[0028] Until the node features of each of the vehicle nodes output by the last network layer are obtained, the node features of each of the vehicle nodes are aggregated to obtain target aggregate features;

[0029] The predicted running trajectory of the target vehicle node in the traffic network graph structure is predicted according to the target aggregation feature.

[0030] In some embodiments, extracting, through the current network layer, the node features of all vehicle nodes output by the previous network layer, the first edge features between the vehicle node and each vehicle node other than the vehicle node, and the second edge features of all vehicle nodes to obtain the node features of the vehicle node output by the current network layer includes:

[0031] Determining a first aggregation weight according to node features of all vehicle nodes, first edge features between the vehicle node and each vehicle node other than the vehicle node, and second edge features of the vehicle node;

[0032] Determining a second aggregation weight according to the node features of all vehicle nodes and the second edge features of all vehicle nodes;

[0033] Performing feature weighting on the node feature of the vehicle node according to the second aggregation weight to obtain a first candidate feature;

[0034] Performing feature weighting on the node features of another vehicle node other than the vehicle node according to the first aggregation weight to obtain a second candidate feature;

[0035] The first candidate feature and the second candidate feature are subjected to feature fusion to obtain a node feature of the vehicle node.

[0036] In some embodiments, the preset graph neural network is trained according to the following steps:

[0037] Acquire a sample historical running trajectory, a sample local trajectory, a sample spatial feature, a sample time feature, real trajectory data, and real lane data; the real trajectory data has a first probability distribution;

[0038] Performing feature aggregation on the sample historical running trajectory, the sample local trajectory, the sample spatial feature and the sample time feature through a preset graph network to obtain a sample aggregation feature;

[0039] Performing trajectory prediction on the sample aggregation features to obtain predicted trajectory data; the predicted trajectory data has a second probability distribution;

[0040] Determining predicted lane data according to the sample local trajectory, the sample aggregation feature and the real lane data;

[0041] determining a target loss according to the actual trajectory data, the predicted trajectory data, the actual lane data, the predicted lane data, the first probability distribution, and the second probability distribution;

[0042] The model parameters of the graph network are updated according to the target loss to obtain the graph neural network.

[0043] In some embodiments, determining the target loss according to the actual trajectory data, the predicted trajectory data, the actual lane data, the predicted lane data, the first probability distribution, and the second probability distribution includes:

[0044] Performing loss calculation according to the actual trajectory data and the predicted trajectory data to obtain a first loss;

[0045] Performing loss calculation according to the real lane data and the predicted lane data to obtain a second loss;

[0046] Calculate the loss according to the first probability distribution and the second probability distribution to obtain a third loss;

[0047] The first loss, the second loss and the third loss are weighted to obtain the target loss.

[0048] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a vehicle trajectory prediction device, the device comprising:

[0049] An acquisition module, used to acquire a traffic network graph structure; wherein the traffic network graph structure includes a plurality of vehicle nodes and interaction relationships between the plurality of vehicle nodes;

[0050] A determination module, used to determine a motif matrix according to the vehicle node and the interaction relationship;

[0051] A first feature extraction module is used to obtain historical running trajectory data corresponding to each vehicle node, extract local trajectory features of each vehicle node from the historical running trajectory data of each vehicle node, and perform global feature extraction on the historical running trajectory data corresponding to each vehicle node to obtain global trajectory features;

[0052] A second feature extraction module is used to extract spatial features from the motif matrix, the corresponding local trajectory features and the global trajectory features for each vehicle node to obtain a target spatial feature of the vehicle node;

[0053] A third feature extraction module is used to extract time features from the motif matrix and the global trajectory features to obtain target time features;

[0054] The prediction module is used to predict the predicted operation trajectory of the target vehicle node in the traffic network graph structure by combining the historical operation trajectory data, the local trajectory features, the target space features and the target time features corresponding to each vehicle node through a preset graph neural network.

[0055] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the vehicle trajectory prediction method of the above-mentioned first aspect is implemented.

[0056] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the vehicle trajectory prediction method of the above-mentioned first aspect is implemented.

[0057] The vehicle trajectory prediction method, vehicle trajectory prediction device, electronic device, and computer-readable storage medium proposed in the embodiment of the present application, by comprehensively considering the mutual influence between different vehicles, model the interaction relationship between vehicles as a graph, and obtain a traffic network graph structure to express the interaction relationship between different vehicles based on the traffic network graph structure. In a complex traffic network graph structure, in order to better understand the interaction process between vehicles, a motif matrix is ​​determined based on vehicle nodes and interaction relationships to achieve high-order connectivity in a complex graph structure based on the motif matrix. In order to predict the trajectory of a vehicle in the future, the historical operation trajectory data corresponding to each vehicle node is obtained. The local trajectory features of each vehicle node are extracted from the historical operation trajectory data of each vehicle node respectively, so as to extract the detail features that play a key role in trajectory prediction from the historical operation trajectory data, and ignore noise and irrelevant features. Taking into account the influence of the trajectory of other vehicle nodes on the trajectory of the current vehicle node, the historical operation trajectory data corresponding to each vehicle node are combined to perform global feature extraction to obtain global trajectory features to obtain the overall trajectory attributes of the traffic network graph structure. In order to dynamically capture the neighbors that interact with the vehicle node, for each vehicle node, the motif matrix, the local trajectory features of the vehicle node, and the global trajectory features are extracted for spatial features to obtain the target spatial features of the vehicle node. In order to dynamically capture the probability of the vehicle node interacting with each neighbor, the motif matrix and the global trajectory features are extracted for temporal features to obtain the target temporal features. The interaction information between vehicles is extracted through the spatial dimension and the temporal dimension to realize the dynamic modeling of the interaction process according to the target spatial features and the target temporal features. The predicted running trajectory of the target vehicle node is predicted by combining the historical running trajectory data, local trajectory features, target spatial features, and target temporal features of each vehicle node through the preset graph neural network. By fusing the interaction feature information and trajectory information between vehicle nodes, the comprehensive expression of feature representation can be realized, the feature expression ability is enhanced, and the accuracy of vehicle trajectory prediction is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 is a flow chart of a vehicle trajectory prediction method provided by an embodiment of the present application;

[0059] Figure 2 is a schematic diagram of a traffic network graph structure provided in an embodiment of the present application;

[0060] Figure 3 yes Figure 1 Flow chart of step S140 in FIG.

[0061] Figure 4 yes Figure 1 Flow chart of step S150 in FIG.

[0062] Figure 5 yes Figure 1Flow chart of step S160 in FIG.

[0063] Figure 6 yes Figure 5 Flow chart of step S540 in FIG.

[0064] Figure 7 It is a flowchart of the training process of the graph neural network provided in the embodiment of the present application;

[0065] Figure 8 yes Figure 7 Flowchart of step S750 in FIG.

[0066] Fig. 9 is a schematic diagram of the structure of a vehicle trajectory prediction device provided in an embodiment of the present application;

[0067] Fig.10 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0069] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0070] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0071] Vehicle trajectory prediction refers to predicting the future trajectory of a vehicle based on its historical trajectory and related information. It can be applied to fields such as autonomous driving, safety planning, and traffic flow control. From a micro perspective, vehicle trajectory prediction helps to quickly make safe and accurate driving strategies, while optimizing driving routes and further ensuring driving safety. From a macro perspective, vehicle trajectory prediction helps to control the overall traffic flow, improve the smoothness of road traffic, and thus reduce the incidence of traffic accidents.

[0072] In intelligent transportation systems, vehicle trajectory prediction technology is often used to optimize driving routes and manage traffic flow, which can improve road utilization while ensuring driving safety. In related technologies, vehicle trajectory prediction is performed only using road condition images collected by the vehicle itself, while ignoring the mutual influence between different vehicles, resulting in reduced accuracy of vehicle trajectory prediction.

[0073] Based on this, the embodiments of the present application provide a vehicle trajectory prediction method, a vehicle trajectory prediction device, an electronic device and a computer-readable storage medium, aiming to improve the accuracy of vehicle trajectory prediction.

[0074] The vehicle trajectory prediction method, vehicle trajectory prediction device, electronic device and computer-readable storage medium provided in the embodiments of the present application are specifically described through the following embodiments. First, the vehicle trajectory prediction method in the embodiments of the present application is described.

[0075] The vehicle trajectory prediction method provided in the embodiment of the present application relates to the field of artificial intelligence technology. The vehicle trajectory prediction method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the vehicle trajectory prediction method, etc., but is not limited to the above forms.

[0076] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0077] Figure 1is an optional flowchart of a vehicle trajectory prediction method provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps S110 to S160.

[0078] Step S110, obtaining a traffic network graph structure; wherein the traffic network graph structure includes a plurality of vehicle nodes and interaction relationships between the plurality of vehicle nodes;

[0079] Step S120, determining a motif matrix according to vehicle nodes and interaction relationships;

[0080] Step S130, obtaining historical running trajectory data corresponding to each vehicle node, extracting local trajectory features of each vehicle node from the historical running trajectory data of each vehicle node, and performing global feature extraction by combining the historical running trajectory data corresponding to each vehicle node to obtain global trajectory features;

[0081] Step S140, for each vehicle node, extracting spatial features from the motif matrix, the corresponding local trajectory features and the global trajectory features to obtain the target spatial features of the vehicle node;

[0082] Step S150, extracting time features from the motif matrix and the global trajectory features to obtain target time features;

[0083] Step S160, predicting the predicted running trajectory of the target vehicle node in the traffic network graph structure by combining the historical running trajectory data, local trajectory features, target space features and target time features corresponding to each vehicle node through a preset graph neural network.

[0084] In step S110 of some embodiments, vehicles can move at different times and spaces, so that the traffic system has traffic flow characteristics. In order to accurately model the traffic system, the vehicles in the traffic system are defined as nodes, and the interaction relationship between vehicles is defined as edges. The traffic system is abstracted as a structure composed of nodes and edges to obtain a traffic network graph structure. Among them, the traffic network graph structure includes multiple vehicle nodes and the interaction relationship between multiple vehicle nodes, and the vehicle nodes include target vehicle nodes. The interaction relationship refers to the way in which vehicles traveling on the road interact and influence each other. For example, the behavior of vehicles following a convoy, the lane change and overtaking behavior of vehicles, the cooperative communication behavior between vehicles, and the driving impact of the front vehicle on the rear vehicle due to acceleration and deceleration, etc.

[0085] For a traffic scene with N vehicles, an unweighted undirected graph with N vehicle nodes can be defined, and the traffic network graph structure is G = (V, E), where V is the node set and E is the edge set. The connection pattern (interaction relationship) between the N vehicle nodes is encoded. If there is an interaction relationship between two vehicle nodes, the connection pattern is encoded as 1, otherwise the connection pattern is encoded as 0, and the adjacency matrix corresponding to the traffic network graph structure is obtained. The adjacency matrix is ​​a 0-1 matrix of size N×N, and the element in the i-th row and j-th column represents the connection pattern between the i-th vehicle node and the j-th vehicle node.

[0086] In step S120 of some embodiments, the interaction relationship between vehicles shows different characteristics and complexity under different traffic flow densities. As the traffic flow density increases, the potential conflicts between vehicles increase, making the interaction relationship between vehicles more complex. The embodiment of the present application considers the impact of different traffic flow densities on the interaction between vehicle nodes, and transforms the dynamic selection of neighboring vehicle nodes participating in the interaction by vehicle nodes into separating these neighboring vehicle nodes from other vehicle nodes. This segmentation process can be regarded as a special graph clustering, using a clustering framework called a motif network to describe this interaction in a higher-order mode. The motif network is defined as:

[0087] M(B,A)={(set(v),set[X A (v)])|v∈V,A v =B},

[0088] Where M represents the motif matrix; v represents an ordered vehicle tuple, including N vehicle nodes; set represents an operator that converts an ordered tuple into an unordered tuple, set{(v1,v2,...,v N )}={v1,v2,...,v N},v i represents the i-th vehicle node; X A is a selection function used to select k neighbor vehicle nodes that interact with the vehicle node, k≤N; B is the adjacency matrix between the vehicle node and the k neighbor vehicle nodes, and B is a 0-1 matrix of size k×k.

[0089] like Figure 2 As shown in the figure, the traffic network graph structure includes a vehicle node set and an edge set. The vehicle node set includes six different vehicle nodes, a, b, c, d, e and f. The edge set includes edges connecting vehicle nodes. The edges are used to represent the interaction relationship between vehicle nodes. The adjacency matrix is ​​determined according to the edges of vehicle nodes as follows:

[0090]

[0091] Where A represents the adjacency matrix of the traffic network graph structure.

[0092] The number of neighboring vehicle nodes that interact with the vehicle node is selected as 2, then set(X A (v)) is represented as {a,c,d}, {a,b,c}, {b,c,e}, {c,d,e}, {b,e,f}, then the motif matrix is ​​represented as:

[0093]

[0094] The elements of the motif matrix represent the number of repeated edges between two vehicle nodes in all triangles. For example, edge ac appears in two triangles, so M 13 =M 31 =2.

[0095] Obtain the adjacency matrix of the traffic network graph structure of multiple consecutive historical time steps, and select the traffic flow density under the corresponding historical time step, determine the corresponding sub-model matrix according to the adjacency matrix and traffic flow density of the corresponding historical time step, arrange multiple sub-model matrices of the same traffic flow density into a row according to the order of the historical time steps, and arrange multiple sub-model matrices of the same historical time step into a column according to the order of the traffic flow density to obtain the model matrix, and the sub-model matrix can be regarded as an element of the model matrix. The embodiment of the present application uses the range, i.e., the number, of neighboring vehicle nodes selected by the vehicle node to represent the traffic flow density. The model matrix is ​​expressed as:

[0096]

[0097] Among them, M s,t is the sub-motif matrix with historical time step t and traffic flow density s, representing the element in the s-th row and t-th column of the motif matrix, s = s1, ..., sS, t = t1, ..., tT.

[0098] The motif matrix can be used to achieve high-order connectivity of interaction relationships in complex graph network clustering, so as to dynamically and selectively model the interaction process in different scenarios, thereby improving the adaptability and accuracy of vehicle trajectory prediction.

[0099] In step S130 of some embodiments, the historical running trajectory data corresponding to each vehicle node is obtained. The historical running trajectory data is a multivariate time series, including the running trajectory of the vehicle node in multiple consecutive historical time steps. The running trajectory has four dimensions, and the running trajectory includes the horizontal coordinate, vertical coordinate, movement speed and movement direction of the vehicle node. The historical running trajectory data of the i-th vehicle node is expressed as is the running trajectory of the i-th vehicle node at the historical time step t. In order to capture the motion state characteristics of each vehicle node at different time steps, such as motion position, motion speed, motion direction, etc., the historical running trajectory data of each vehicle node is firstly extracted by Multi-Layer Perceptron (MLP), and then the extracted features are extracted by Long Short-Term Memory (LSTM) to obtain the local trajectory features of each vehicle node. The local trajectory features of the i-th vehicle node are expressed as In order to capture the overall motion state in the traffic scene and the dependency relationship between the motion states of each vehicle node, the historical running trajectory data corresponding to each vehicle node is combined, and the feature extraction of all historical running trajectory data is performed through the point network PointNet. The extracted features are extracted through the multi-layer perceptron to obtain the global trajectory feature. The global trajectory feature is expressed as

[0100] In order to more finely characterize the mutual influence between vehicles, the interaction information between vehicles is extracted from the spatial dimension and the temporal dimension, and modeled by using the spatiotemporal decomposition method. This modeling method not only considers the instant interaction between vehicles, but also incorporates the continuity of time and the relativity of space, thereby improving the accuracy of vehicle trajectory prediction. The dynamic modeling process of the interaction process in space and time is described in detail below.

[0101] See also Figure 3 In some embodiments, step S140 may include but is not limited to steps S310 to S350:

[0102] Step S310, adding the elements of each row in the motif matrix to obtain a plurality of spatial motif vectors;

[0103] Step S320, performing feature extraction on each spatial motif vector to obtain a spatial motif feature of each spatial motif vector;

[0104] Step S330, for each vehicle node, using the local trajectory feature and the global trajectory feature as a query vector, and performing multi-head attention calculation on each spatial motif feature and the query vector to obtain multiple initial attention features;

[0105] Step S340, performing feature fusion on all initial attention features, the maximum initial attention feature, the minimum initial attention feature, and the initial attention feature representing the median to obtain a target attention feature;

[0106] Step S350, determining the target space features of the vehicle node according to the target attention features and the local trajectory features.

[0107] In step S310 of some embodiments, dynamic modeling of spatial interaction relationships refers to dynamically selecting neighbors that participate in interactions with the target vehicle node under different traffic flow densities. In the modeling process, the elements of each row in the motif matrix are added to remove the time dimension of the motif matrix to obtain a motif matrix in the spatial dimension. The motif matrix in the spatial dimension includes spatial motif vectors under multiple traffic flow densities. It should be noted that the elements of the motif matrix are sub-motif matrices, and matrix addition operations are performed on the sub-motif matrices of each row to obtain multiple spatial motif vectors. The spatial motif vector is the sum of a row of sub-motif matrices. The spatial motif vector can be defined as:

[0108]

[0109] in, Represents the spatial motif vector with traffic flow density s.

[0110] In step S320 of some embodiments, for each spatial motif vector, feature extraction is performed on the spatial motif vector through a spatial pyramid pooling network (SPP) to extract the interaction relationship of the spatial dimension to obtain a first spatial feature map, a second spatial feature map is extracted from the first spatial feature map through a fully connected layer (FC), and the second spatial feature map is layer normalized through layer normalization (LN) to obtain a spatial motif feature. The spatial motif feature is represented as

[0111] In step S330 of some embodiments, for each vehicle node, the local trajectory features and global trajectory features of the vehicle node are concatenated to obtain a query vector, and the spatial motif features are used as key vectors and value vectors respectively. The scaled dot product attention is used to perform multi-head self-attention calculation based on the query vector, key vector and value vector to spatially fuse the trajectory features of the vehicle node and the features of the motif matrix to obtain the initial attention features corresponding to the spatial motif features. The specific process of multi-head self-attention calculation is as follows:

[0112] β i,s =MultiHead(Q i ,K s ,V s )=(head1;...;head h )W O ,

[0113]

[0114] Where i represents the i-th vehicle node; operator; represents the splicing operation; β i,s represents the initial attention feature of the i-th vehicle node under the traffic flow density s; MultiHead represents multi-head self-attention calculation; Attention represents attention calculation; head k represents the feature of the attention calculation output by the kth attention head; Q i represents the query vector of the i-th vehicle node; K s 、V s denote the key vector and value vector respectively; and W O They represent the weight matrix of the query vector, the weight matrix of the key vector, the weight matrix of the value vector, and the weight matrix of the features output by multiple attention heads respectively.

[0115] In step S340 of some embodiments, the modulus of each initial attention feature is calculated, and the initial attention feature corresponding to the maximum modulus is used as the maximum initial attention feature, the initial attention feature corresponding to the minimum modulus is used as the minimum initial attention feature, and the initial attention feature with the median modulus is used as the initial attention feature representing the median. In order to reduce information loss, the initial attention features under all traffic flow densities, the maximum initial attention features, the minimum initial attention features, and the initial attention features representing the median are feature fused, and the fused features are feature extracted through a multilayer perceptron to obtain the target attention feature. The target attention feature is expressed as:

[0116]

[0117] in, is the target attention feature of the i-th vehicle node; β i,s is the initial attention feature of the i-th vehicle node under traffic flow density s; β i,max , β i,min and β i,med are the largest initial attention feature, the smallest initial attention feature, and the median initial attention feature among all the initial attention features.

[0118] In step S350 of some embodiments, the target attention feature and the local trajectory feature are multiplied element by element to obtain the target space feature of the vehicle node. The target space feature of the vehicle node i is expressed as

[0119] Through the above steps S310 to S350, the interactive relationship in the spatial dimension can be dynamically modeled, so that the vehicle node can adaptively select the neighboring vehicle nodes with which to interact under different traffic flow densities, and realize multi-scale neighbor selection, so that the vehicle node can adaptively select the optimal range of neighbors participating in the interaction at each moment, thereby improving the accuracy of vehicle trajectory prediction.

[0120] See also Figure 4 In some embodiments, step S150 may include but is not limited to steps S410 to S450:

[0121] Step S410, adding the elements of each column in the motif matrix to obtain a plurality of time motif vectors;

[0122] Step S420, performing feature extraction on each time motif vector respectively to obtain an initial time motif feature of each time motif vector;

[0123] Step S430, performing feature fusion on all initial time motif features, the maximum initial time motif feature, the minimum initial time motif feature, and the initial time motif feature representing the median to obtain a target time motif feature;

[0124] Step S440, performing weight calculation according to the global trajectory features and the target time motif features to obtain the weight of each time motif vector;

[0125] Step S450: determining the target time feature according to each time motif vector and the weight corresponding to the time motif vector.

[0126] In step S410 of some embodiments, the dynamic modeling of the interaction relationship in time refers to dynamically generating the interaction order between the vehicle node and each neighbor participating in the interaction under a specific traffic flow density, that is, the size of the interaction probability between the vehicle node and each neighbor. The elements of the motif matrix are sub-motif matrices. In the time dimension, the sub-motif matrices of each column in the motif matrix are subjected to matrix addition operations to obtain multiple time motif vectors as follows: The time motif vector is the sum of a column of sub-motif matrices. The time motif vector can be defined as:

[0127]

[0128] in, represents the temporal motif vector at time t.

[0129] In step S420 of some embodiments, for each time motif vector, feature extraction is performed on the time motif vector through a spatial pyramid pooling network to extract the interaction relationship of the time dimension to obtain a first time feature map, a second time feature map is extracted from the first time feature map through a fully connected layer, and the second time feature map is layer-normalized through layer normalization to obtain an initial time motif feature. The initial time motif feature is expressed as

[0130] In step S430 of some embodiments, the modulus of each initial time motif feature is calculated respectively, and the initial time motif feature corresponding to the maximum modulus is used as the maximum initial time motif feature, the initial time motif feature corresponding to the minimum modulus is used as the minimum initial time motif feature, and the initial time motif feature with the median modulus is used as the initial time motif feature representing the median. In order to reduce the information loss of the time dimension, all initial time motif features are The largest initial time motif feature Minimum initial time pattern feature and the initial time pattern features representing the median Perform feature splicing and use a multi-layer perceptron to extract the spliced ​​features Extract the target time motif features.

[0131] In step S440 of some embodiments, the global trajectory features and the target time motif are concatenated and input into a reweighting network for weight calculation to obtain a vector of each time motif. The weight α t .

[0132] In step S450 of some embodiments, each time motif vector and the weight corresponding to the time motif vector are weighted and calculated to obtain the target time feature. The target time feature is expressed as:

[0133] Through the above steps S410 to S450, the interaction relationship in the time dimension can be dynamically modeled, so that the vehicle node can adaptively determine the interaction order of neighbor nodes under different traffic flow densities, thereby ensuring the accuracy of trajectory prediction.

[0134] The embodiment of the present application selects and sorts the neighbors that interact with the vehicle at different densities from the perspective of time and space, so that under different traffic densities, the vehicle can intelligently select the neighbor vehicle that is most suitable for participating in the interaction, which not only improves the pertinence of the interaction, but also ensures the accuracy of trajectory prediction. The embodiment of the present application can flexibly adjust the set of vehicles participating in the interaction according to the specific situation, thereby providing a more accurate and reliable trajectory prediction service for the vehicle.

[0135] The historical operation trajectory data, local trajectory features, target space features and target time features corresponding to each vehicle node are input into the preset graph attention network (GAT). The interaction features of multiple vehicles are fused through the graph neural network to predict the operation trajectory of the target vehicle node in the traffic network graph structure in the future and obtain the predicted operation trajectory.

[0136] See also Figure 5 In some embodiments, the preset graph neural network includes multiple network layers, and the network layer includes a fully connected layer and a layer normalization layer connected in series in sequence. Step S160 may include but is not limited to steps S510 to S560:

[0137] Step S510, for each vehicle node, extract the historical running trajectory data, local trajectory features, and target space features of the vehicle node through the first network layer to obtain the node features output by the first network layer;

[0138] Step S520, determining a first edge feature between the vehicle node and any vehicle node according to the historical running trajectory data of the vehicle node, the historical running trajectory data of any vehicle node other than the vehicle node, and the target time feature;

[0139] Step S530, determining the second edge feature of the vehicle node according to the historical running trajectory data of the vehicle node and the target time feature;

[0140] Step S540, for each vehicle node, extracting the node features of all vehicle nodes output by the previous network layer, the first edge features between the vehicle node and each vehicle node other than the vehicle node, and the second edge features of all vehicle nodes through the current network layer, to obtain the node features of the vehicle node output by the current network layer;

[0141] Step S550, until the node features of each vehicle node output by the last network layer are obtained, the node features of each vehicle node are aggregated to obtain target aggregate features;

[0142] Step S560, predicting the predicted running trajectory of the target vehicle node in the traffic network graph structure according to the target aggregation feature.

[0143] In step S510 of some embodiments, for vehicle node i, the historical running trajectory data X of vehicle node i is i , local trajectory features and the target space feature φ Si Input to the first network layer for feature extraction to obtain the node features output by the first network layer

[0144] In step S520 of some embodiments, the historical running trajectory data of vehicle node i is The running trajectory of the historical time step in the same time step is compared with the historical running trajectory data of any vehicle node j except the vehicle node Subtract the running trajectory of the corresponding historical time step from the target time feature to obtain the trajectory deviation of the historical time step. After adding the trajectory deviations of each historical time step, the result is compared with the element of the i-th row and j-th column in the target time feature. The first edge feature between vehicle node i and any vehicle node j is obtained by concatenation. The first edge feature between vehicle node i and vehicle node j is expressed as:

[0145]

[0146] In step S530 of some embodiments, the trajectory deviation of the historical time step is determined to be zero according to the historical running trajectory data of the vehicle node i, and the trajectory deviations of each historical time step are added and concatenated with the element of the i-th row and i-th column in the target time feature to obtain the second edge feature of the vehicle node. The second edge feature of the vehicle node i is expressed as:

[0147]

[0148] In step S540 of some embodiments, for vehicle node i, the node features of all vehicle nodes output from the previous network layer l-1 through the current network layer l are The first edge feature between the vehicle node and each vehicle node other than the vehicle node and the second edge features of all vehicle nodes Extract the node features of vehicle node i and obtain the node features of vehicle node i output by the current network layer

[0149] In step S550 of some embodiments, the node features of the vehicle node i output by the next network layer are obtained with reference to step S540, until the node features of each vehicle node output by the last network layer are obtained, the node features of each vehicle node are aggregated, and the aggregated features are layer-normalized to obtain the target aggregated features. The target aggregated features are expressed as:

[0150]

[0151] In step S560 of some embodiments, the target aggregate features are feature decoded by a decoder to obtain a predicted running trajectory of the target vehicle node i in the traffic network graph structure.

[0152] The above steps S510 to S560 enhance the feature expression capability by fusing the interactive feature information and trajectory information of the vehicle nodes in the time dimension and the space dimension, thereby improving the accuracy of the vehicle trajectory prediction.

[0153] See also Figure 6 In some embodiments, step S540 may include but is not limited to steps S610 to S650:

[0154] Step S610, determining a first aggregation weight according to the node features of all vehicle nodes, the first edge features between the vehicle node and each vehicle node other than the vehicle node, and the second edge features of the vehicle node;

[0155] Step S620, determining a second aggregation weight according to the node features of all vehicle nodes and the second edge features of all vehicle nodes;

[0156] Step S630, performing feature weighting on the node features of the vehicle node according to the second aggregation weight to obtain a first candidate feature;

[0157] Step S640, performing feature weighting on node features of another vehicle node other than the vehicle node according to the first aggregation weight to obtain a second candidate feature;

[0158] Step S650: performing feature fusion on the first candidate feature and the second candidate feature to obtain a node feature of the vehicle node.

[0159] In step S610 of some embodiments, the node characteristics of all vehicle nodes output by the previous network layer l-1 are The first edge feature between the vehicle node i and each vehicle node j other than the vehicle node and the second edge feature of the vehicle node i are weighted to obtain the first aggregation weight between the vehicle node and each vehicle node other than the vehicle node. The calculation method of the first aggregation weight is as follows:

[0160]

[0161] Among them, a′ is a learnable parameter, dim(a′)=2dim(v i )+dim(e i,j ), dim represents the feature dimension; represents the first aggregation weight of vehicle node i and vehicle node j in the current network layer l; if j is not equal to i, e i,j Represents the first edge feature between vehicle node i and vehicle node j; if j is equal to i, e i,j Represents the second edge feature of vehicle node i.

[0162] In step S620 of some embodiments, the node characteristics of all vehicle nodes output by the previous network layer l-1 are The second edge features of all vehicle nodes are weighted and calculated to obtain the second aggregation weight of the vehicle node. The calculation method of the second aggregation weight is as follows:

[0163]

[0164] in, represents the second aggregation weight of vehicle node i in the current network layer l; e i,i Represents the second edge feature of vehicle node i.

[0165] In step S630 of some embodiments, the second aggregation weight is multiplied by the node feature of the vehicle node output by the previous network layer to obtain a first candidate feature. The first candidate feature is represented as:

[0166] In step S640 of some embodiments, the first aggregation weight is multiplied by the node feature of another vehicle node other than the vehicle node output by the previous network layer, and all multiplication results are added to obtain a second candidate feature. The second candidate feature is expressed as:

[0167]

[0168] In step S650 of some embodiments, the first candidate feature and the second candidate feature are added, and the added features are extracted through the fully connected layer and the layer normalization layer of the current network layer to obtain the node feature of the vehicle node i output by the current network layer l.

[0169] Through the above steps S610 to S650, the node features output by the vehicle node in the current network layer can be obtained, so as to predict the trajectory of the vehicle node based on the node features.

[0170] See also Figure 7 In some embodiments, the training process of the preset graph neural network may include but is not limited to steps S710 to S760:

[0171] Step S710, obtaining a sample historical running trajectory, a sample local trajectory, a sample spatial feature, a sample time feature, real trajectory data, and real lane data; the real trajectory data has a first probability distribution;

[0172] Step S720, performing feature aggregation on the sample historical running trajectory, the sample local trajectory, the sample spatial feature and the sample time feature through a preset graph network to obtain a sample aggregation feature;

[0173] Step S730, performing trajectory prediction on the sample aggregation features to obtain predicted trajectory data; the predicted trajectory data has a second probability distribution;

[0174] Step S740, determining predicted lane data according to the sample local trajectory, the sample aggregation feature and the actual lane data;

[0175] Step S750, determining a target loss according to the actual trajectory data, the predicted trajectory data, the actual lane data, the predicted lane data, the first probability distribution, and the second probability distribution;

[0176] Step S760, update the model parameters of the graph network according to the target loss to obtain a graph neural network.

[0177] In step S710 of some embodiments, a sample traffic graph structure for training the network and a sample historical running track of each vehicle node in the sample traffic graph structure at a historical time are obtained. Referring to steps S120 to S150, a motif matrix is ​​determined according to the sample traffic graph structure, and a sample local track of each vehicle node is extracted from the sample historical running track of each vehicle node, and a global feature extraction is performed on the sample historical running track corresponding to each vehicle node to obtain a sample global track. For each vehicle node, a spatial feature extraction is performed on the motif matrix, the corresponding sample local track and the sample global track to obtain a sample spatial feature of the vehicle node, and a temporal feature extraction is performed on the motif matrix and the sample global track to obtain a sample temporal feature. The real track data and the real lane data of the target vehicle node are obtained, the real track data is the real running track of the target vehicle node in the future time, and the real lane data is the coordinate position of the real lane. The real track data has a first probability distribution, and the first probability distribution is the real probability that the target vehicle node appears at a specific position, at a specific movement speed, and in a specific movement direction in the future time.

[0178] In step S720 of some embodiments, referring to steps S510 to S550, the sample historical operation trajectories, sample local trajectories, sample spatial features and sample time features of all vehicle nodes are input into a preset graph network for feature aggregation to obtain sample aggregation features.

[0179] In step S730 of some embodiments, trajectory prediction is performed on the sample aggregation features to obtain predicted trajectory data of the target vehicle node, which is the predicted running trajectory of the target vehicle node at a future time. The predicted trajectory data has a second probability distribution, which is the predicted probability that the target vehicle node will appear at a specific location, at a specific movement speed, and in a specific movement direction at a future time. The second probability distribution is expressed as:

[0180] p(Y|φ i ,X,Z)p(φi |X,Z),

[0181]

[0182] Among them, X represents the historical running trajectory of the sample; Y represents the predicted trajectory data; φ i Represents the potential intention of vehicle node i, including sample space features φ Si and sample time characteristic φ ti , there is an intersection between the sample space features and the sample time features; Z represents the local trajectory of the sample.

[0183] The sum of the second probability distribution of the predicted trajectory data of all vehicle nodes is expressed as:

[0184] p(Y|X,Z)=∑ i p(Y|φ i ,X,Z)p(φ i |X,Z)=Σ i p D p θ .

[0185] In step S740 of some embodiments, the real lane data includes first lane data at historical time and second lane data at future time, the first lane data is the coordinate position of the real lane at historical time, and the second lane data is the coordinate position of the real lane at future time.

[0186] The initial lane features are extracted from the first lane data through a multi-layer perceptron, and the target lane features are extracted from the initial lane features through a long short-term memory network, and the sample local trajectory, sample aggregation features and target lane features are spliced. After the spliced ​​features are extracted through a multi-layer perceptron, the extracted features are activated through a softmax function to obtain the predicted lane data. The predicted lane data is the coordinate position of the predicted lane at a future time.

[0187] In step S750 of some embodiments, the second lane data is one-hot encoded to obtain a real lane indication vector. The target loss is determined according to the real trajectory data, the predicted trajectory data, the real lane indication vector, the predicted lane data, the first probability distribution and the second probability distribution.

[0188] In step S760 of some embodiments, the target loss is minimized, and the model parameters of the graph network are updated to obtain a graph neural network.

[0189] Through the above steps S710 to S760, a graph neural network can be obtained to fuse the interactive features of different traffic flow densities and different times based on the graph neural network, and then accurately predict the predicted operating trajectory of each vehicle node.

[0190] See also Figure 8 In some embodiments, step S750 may include but is not limited to steps S810 to S840:

[0191] Step S810, performing loss calculation according to the actual trajectory data and the predicted trajectory data to obtain a first loss;

[0192] Step S820, calculating the loss according to the real lane data and the predicted lane data to obtain a second loss;

[0193] Step S830, calculating the loss according to the first probability distribution and the second probability distribution to obtain a third loss;

[0194] Step S840, weighting the first loss, the second loss, and the third loss to obtain a target loss.

[0195] In step S810 of some embodiments, the L2 norm of the real trajectory data and the predicted trajectory data is calculated to obtain a first loss.

[0196] In step S820 of some embodiments, a binary cross entropy loss function (BCE) is used to calculate the loss of the real lane indication vector and the predicted lane data to obtain a second loss. The second loss is expressed as:

[0197]

[0198] Among them, L SEL is the second loss; BCE represents the binary cross entropy loss function; g m is the real lane indication vector; They represent sample local trajectories, sample aggregation features, and target lane features respectively.

[0199] In step S830 of some embodiments, a KL (Kullback-Leibler) divergence between the first probability distribution and the second probability distribution is calculated to obtain a third loss.

[0200] In step S840 of some embodiments, the first loss, the second loss, and the third loss are weighted to obtain a target loss. The target loss is calculated as follows:

[0201] loss = L DIS +λ1L SEL +λ2L KLD ,

[0202] Among them, loss represents the target loss; L DIS Indicates the first loss; L SEL Indicates the second loss; LKLD represents the third loss; λ1 and λ2 both represent weight parameters, and λ2 increases with the increase of training rounds.

[0203] Through the above steps S810 to S840, the target loss can be obtained to train the graph neural network based on the target loss.

[0204] See also Fig. 9 The embodiment of the present application further provides a vehicle trajectory prediction device, which can implement the above-mentioned vehicle trajectory prediction method, and the vehicle trajectory prediction device includes:

[0205] The acquisition module 910 is used to acquire a traffic network graph structure; wherein the traffic network graph structure includes a plurality of vehicle nodes and interaction relationships between the plurality of vehicle nodes;

[0206] A determination module 920, configured to determine a motif matrix according to vehicle nodes and interaction relationships;

[0207] A first feature extraction module 930 is used to obtain historical running trajectory data corresponding to each vehicle node, extract local trajectory features of each vehicle node from the historical running trajectory data of each vehicle node, and perform global feature extraction on the historical running trajectory data corresponding to each vehicle node to obtain global trajectory features;

[0208] The second feature extraction module 940 is used to extract spatial features from the motif matrix, the corresponding local trajectory features and the global trajectory features for each vehicle node, so as to obtain the target spatial features of the vehicle node;

[0209] The third feature extraction module 950 is used to extract time features from the motif matrix and the global trajectory features to obtain target time features;

[0210] The prediction module 960 is used to predict the predicted running trajectory of the target vehicle node in the traffic network graph structure by combining the historical running trajectory data, local trajectory features, target space features and target time features corresponding to each vehicle node through a preset graph neural network.

[0211] The specific implementation of the vehicle trajectory prediction device is basically the same as the specific implementation of the above-mentioned vehicle trajectory prediction method, and will not be repeated here.

[0212] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned vehicle trajectory prediction method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0213] See also Fig.10 , Fig.10The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0214] The processor 1010 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0215] The memory 1020 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1020 can store an operating system and other applications. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1020, and the processor 1010 calls and executes the vehicle trajectory prediction method of the embodiment of this application;

[0216] Input / output interface 1030, used to implement information input and output;

[0217] Communication interface 1040, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0218] Bus 1050 , which transmits information between various components of the device (e.g., processor 1010 , memory 1020 , input / output interface 1030 , and communication interface 1040 );

[0219] The processor 1010 , the memory 1020 , the input / output interface 1030 , and the communication interface 1040 are connected to each other in communication within the device via a bus 1050 .

[0220] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned vehicle trajectory prediction method is implemented.

[0221] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0222] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0223] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0224] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0225] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0226] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0227] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0228] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0229] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0230] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0231] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0232] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A vehicle trajectory prediction method, characterized in that: The method comprises: Acquire a traffic network graph structure; wherein the traffic network graph structure includes a plurality of vehicle nodes and interaction relationships between the plurality of vehicle nodes; Determine a motif matrix according to the vehicle nodes and the interaction relationship; Obtaining historical running trajectory data corresponding to each vehicle node, extracting local trajectory features of each vehicle node from the historical running trajectory data of each vehicle node respectively, and performing global feature extraction by combining the historical running trajectory data corresponding to each vehicle node to obtain global trajectory features; For each vehicle node, extracting spatial features from the motif matrix, the corresponding local trajectory features and the global trajectory features to obtain target spatial features of the vehicle node; Performing time feature extraction on the motif matrix and the global trajectory feature to obtain target time features; The predicted running trajectory of the target vehicle node in the traffic network graph structure is predicted by combining the historical running trajectory data, the local trajectory features, the target space features and the target time features corresponding to each vehicle node through a preset graph neural network.

2. The method according to claim 1, characterized in that For each vehicle node, performing spatial feature extraction on the motif matrix, the corresponding local trajectory features and the global trajectory features to obtain the target spatial features of the vehicle node includes: Adding the elements of each row in the motif matrix to obtain a plurality of spatial motif vectors; Performing feature extraction on each of the spatial motif vectors respectively to obtain a spatial motif feature of each of the spatial motif vectors; For each of the vehicle nodes, the local trajectory feature and the global trajectory feature are used as query vectors, and each of the spatial motif features and the query vector are respectively subjected to multi-head attention calculation to obtain multiple initial attention features; Performing feature fusion on all the initial attention features, the largest initial attention feature, the smallest initial attention feature, and the initial attention feature representing the median to obtain a target attention feature; The target space feature of the vehicle node is determined according to the target attention feature and the local trajectory feature.

3. The method according to claim 1, characterized in that The extracting time features from the motif matrix and the global trajectory features to obtain target time features includes: Adding the elements of each column in the motif matrix to obtain a plurality of time motif vectors; Performing feature extraction on each of the time motif vectors respectively to obtain initial time motif features of each of the time motif vectors; Performing feature fusion on all the initial time motif features, the largest initial time motif feature, the smallest initial time motif feature, and the initial time motif feature representing the median to obtain a target time motif feature; Performing weight calculation according to the global trajectory features and the target time motif features to obtain the weight of each time motif vector; The target time feature is determined according to each of the time motif vectors and the weight corresponding to the time motif vector.

4. The method according to claim 1, characterized in that: The preset graph neural network includes a plurality of network layers, and the preset graph neural network combines the historical running trajectory data, the local trajectory features, the target space features, and the target time features corresponding to each vehicle node to predict the predicted running trajectory of the target vehicle node in the traffic network graph structure, including: For each of the vehicle nodes, extract the historical running trajectory data, the local trajectory features, and the target space features of the vehicle node through the first network layer to obtain the node features output by the first network layer; Determine a first edge feature between the vehicle node and any vehicle node according to the historical running track data of the vehicle node, the historical running track data of any vehicle node other than the vehicle node, and the target time feature; Determine a second edge feature of the vehicle node according to the historical running trajectory data of the vehicle node and the target time feature; For each of the vehicle nodes, extracting, through the current network layer, the node features of all vehicle nodes output by the previous network layer, the first edge features between the vehicle node and each vehicle node other than the vehicle node, and the second edge features of all vehicle nodes, to obtain the node features of the vehicle node output by the current network layer; Until the node features of each of the vehicle nodes output by the last network layer are obtained, the node features of each of the vehicle nodes are aggregated to obtain target aggregate features; The predicted running trajectory of the target vehicle node in the traffic network graph structure is predicted according to the target aggregation feature.

5. The method according to claim 4, characterized in that The extracting, through the current network layer, the node features of all vehicle nodes output by the previous network layer, the first edge features between the vehicle node and each vehicle node other than the vehicle node, and the second edge features of all vehicle nodes to obtain the node features of the vehicle node output by the current network layer includes: Determining a first aggregation weight according to node features of all vehicle nodes, first edge features between the vehicle node and each vehicle node other than the vehicle node, and second edge features of the vehicle node; Determining a second aggregation weight according to the node features of all vehicle nodes and the second edge features of all vehicle nodes; Performing feature weighting on the node feature of the vehicle node according to the second aggregation weight to obtain a first candidate feature; Performing feature weighting on the node features of another vehicle node other than the vehicle node according to the first aggregation weight to obtain a second candidate feature; The first candidate feature and the second candidate feature are subjected to feature fusion to obtain a node feature of the vehicle node.

6. The method according to claim 4, characterized in that The preset graph neural network is trained according to the following steps: Acquire a sample historical running trajectory, a sample local trajectory, a sample spatial feature, a sample time feature, real trajectory data, and real lane data; the real trajectory data has a first probability distribution; Performing feature aggregation on the sample historical running trajectory, the sample local trajectory, the sample spatial feature and the sample time feature through a preset graph network to obtain a sample aggregation feature; Performing trajectory prediction on the sample aggregation features to obtain predicted trajectory data; the predicted trajectory data has a second probability distribution; Determining predicted lane data according to the sample local trajectory, the sample aggregation feature and the real lane data; determining a target loss according to the actual trajectory data, the predicted trajectory data, the actual lane data, the predicted lane data, the first probability distribution, and the second probability distribution; The model parameters of the graph network are updated according to the target loss to obtain the graph neural network.

7. The method according to claim 6, characterized in that The determining the target loss according to the actual trajectory data, the predicted trajectory data, the actual lane data, the predicted lane data, the first probability distribution, and the second probability distribution includes: Performing loss calculation according to the actual trajectory data and the predicted trajectory data to obtain a first loss; Performing loss calculation according to the real lane data and the predicted lane data to obtain a second loss; Calculate the loss according to the first probability distribution and the second probability distribution to obtain a third loss; The first loss, the second loss and the third loss are weighted to obtain the target loss.

8. A vehicle trajectory prediction device, characterized in that: The device comprises: An acquisition module, used to acquire a traffic network graph structure; wherein the traffic network graph structure includes a plurality of vehicle nodes and interaction relationships between the plurality of vehicle nodes; A determination module, used to determine a motif matrix according to the vehicle node and the interaction relationship; A first feature extraction module is used to obtain historical running trajectory data corresponding to each vehicle node, extract local trajectory features of each vehicle node from the historical running trajectory data of each vehicle node, and perform global feature extraction on the historical running trajectory data corresponding to each vehicle node to obtain global trajectory features; A second feature extraction module is used to extract spatial features from the motif matrix, the corresponding local trajectory features and the global trajectory features for each vehicle node to obtain a target spatial feature of the vehicle node; A third feature extraction module is used to extract time features from the motif matrix and the global trajectory features to obtain target time features; The prediction module is used to predict the predicted operation trajectory of the target vehicle node in the traffic network graph structure by combining the historical operation trajectory data, the local trajectory features, the target space features and the target time features corresponding to each vehicle node through a preset graph neural network.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images