Vehicle track prediction method and system, terminal equipment and medium

Abstract

The invention belongs to the technical field of trajectory prediction, and provides a vehicle trajectory prediction method and system, terminal equipment and a medium, and the method comprises the steps: obtaining the vehicle information of a vehicle, the vehicle information of other vehicles, and the driving environment information; obtaining interaction information between the vehicle and other vehicles, performing time sequence coding on the interaction information and the vehicle information of other vehicles to obtain time interaction characteristics, and obtaining time interaction dynamic characteristics of the time interaction characteristics based on a time self-attention mechanism; according to the vehicle information of other vehicles and the driving environment information, extracting spatial interaction features, and obtaining spatial interaction dynamic features of the spatial interaction features based on a spatial interaction attention mechanism; and fusing the time interaction dynamic features and the space interaction dynamic features to obtain fused features, decoding the fused features to obtain driving intentions of other vehicles, and obtaining predicted trajectories of other vehicles according to the driving intentions. The accuracy of vehicle trajectory prediction can be improved.

Description

Technical Field

[0001] This invention belongs to the field of trajectory prediction technology, specifically relating to a vehicle trajectory prediction method, system, terminal equipment, and medium. Background Technology

[0002] With the rapid development of deep learning and artificial intelligence technologies, the implementation of autonomous driving technology has gradually become possible. In autonomous driving systems, the trajectory prediction module is responsible for receiving upstream perception information and accurately predicting the future trajectory or intention of surrounding targets, playing a crucial role in subsequent decision-making and planning modules. However, existing vehicle trajectory prediction technologies suffer from incomplete consideration of the interactions between the target vehicle and surrounding vehicles, significantly impacting the accuracy of trajectory prediction. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a vehicle trajectory prediction method, system, terminal equipment and medium to improve the accuracy of vehicle trajectory prediction.

[0004] In a first aspect, the present invention provides a vehicle trajectory prediction method, the method comprising the following steps:

[0005] The system acquires the first vehicle information of its own vehicle, the second vehicle information of other vehicles within a specified range of its own vehicle, and driving environment information; both the first and second vehicle information include vehicle location, vehicle speed, and acquisition time.

[0006] The difference between the first vehicle information and the second vehicle information is calculated and used as the interaction information between the vehicle and other vehicles. The interaction information and the second vehicle information are temporally encoded to obtain temporal interaction features that describe the temporal dependency between the interaction information and the second vehicle information. The temporal interaction dynamic features are obtained based on the temporal self-attention mechanism. The temporal interaction dynamic features are used to predict the driving intentions of other vehicles.

[0007] Based on the second vehicle information and driving environment information, spatial interaction features are extracted to describe the spatial interaction information of other vehicles, and spatial interaction dynamic features are obtained based on the spatial interaction attention mechanism; the spatial interaction dynamic features are used to predict the driving intentions of other vehicles.

[0008] The temporal and spatial interactive dynamic features are fused to obtain fused features. The fused features are then decoded to obtain the driving intentions of other vehicles, and the predicted trajectories of other vehicles are obtained based on the driving intentions.

[0009] Optional driving environment information includes lane lines, curbs, drivable areas, roadblocks, traffic signs, and traffic lights.

[0010] Optionally, the interaction information and the second vehicle information are time-series encoded separately to obtain temporal interaction features describing the temporal dependency between the interaction information and the second vehicle information, including:

[0011] Through calculation formula

[0012] H = concat(H) S H I )

[0013]

[0014]

[0015] The temporal interaction feature H is obtained; where H S H represents the state information feature. I Indicates the characteristics of time-based interactive information. This represents the hidden state vector obtained from the state information at time t using LSTM. This represents the hidden state vector containing the time interaction information at time t obtained through LSTM. This represents the hidden state vector obtained from the state information at time t-1 using LSTM. This represents the hidden state vector at time t-1 obtained through LSTM, where concat(·) represents concatenation, and LSTM(·) represents a Long Short Memory network used to capture temporal dependencies. Let t represent the embedding vector corresponding to the second vehicle information, where t = 1, 2, ..., T, and T represents the total time for trajectory prediction. f represents the embedding vector corresponding to the interaction information. c (·) indicates a fully connected S t Indicates the second vehicle information, I t The interaction information is represented by W1 and W2, which represent the parameter matrices of the Long Short-Term Memory network, and W3 and W4, which represent the trainable weight matrices.

[0016] Optionally, the temporal interaction dynamic features obtained based on the temporal self-attention mechanism include:

[0017] An attention mechanism matrix is ​​constructed based on temporal interaction features; the expression for the attention mechanism matrix is ​​as follows:

[0018]

[0019] Among them, Q H Represents the query matrix, K H V represents the bond matrix. H The value matrix is ​​represented by APReLU(·), and the adaptive activation matrix is ​​represented by APReLU(·). This represents the learnable weight matrix;

[0020] Through calculation formula We obtain self-attention; where d K K represents H The horizontal dimension of the middle;

[0021] The calculation formula is Mult1-Attention = concat[Attention1, Attention2, ..., Attention]. h ], thus obtaining the temporal interaction dynamic features Mult1, where Attention h This represents the attention output corresponding to each attention head, and h represents the number of heads in the multi-head attention.

[0022] Optionally, based on the second vehicle information and driving environment information, spatial interaction features for describing spatial interaction information of other vehicles are extracted, including:

[0023] Define the effective interaction space for other vehicles; the effective interaction space includes other vehicles and the driving environment within a preset range.

[0024] Extract vehicle information and driving environment information corresponding to the effective interaction space from the second vehicle information and driving environment information; and use the calculation formula.

[0025] g = concat(g) J g N )

[0026] g J =f c (J t W5)

[0027] g N =f c (N t W6)

[0028] We obtain the spatial interaction feature g; where g J g represents the embedding vector of vehicle information corresponding to the effective interaction space. N J represents the embedding vector of driving environment information corresponding to the effective interaction space. t N represents spatial interaction information between other vehicles and other vehicles within a preset range. t W5 and W6 represent the spatial interaction information between other vehicles and the driving environment within a preset range, and represent the trainable weight matrices.

[0029] Optionally, spatial interaction dynamic features obtained based on spatial interaction attention mechanisms include:

[0030] Through calculation formula

[0031] Mult2-SpaceAttention=concat[SpaceAttention1, SpaceAttention2, ..., SpaceAttention h ]

[0032]

[0033] We obtain the spatial interaction dynamic feature Mult2; where SpaceAttention represents spatial interaction attention, Q... g K represents the query matrix corresponding to spatial interaction attention. g V represents the key matrix corresponding to spatial interactive attention. g This represents the value matrix corresponding to spatial interaction attention. This represents the learnable weight matrix.

[0034] Optional driving intentions include left lane change, lane keeping, right lane change, deceleration, acceleration, and cruise control.

[0035] In a second aspect, the present invention provides a vehicle trajectory prediction system, comprising:

[0036] The information acquisition module is used to acquire the first vehicle information of its own vehicle, the second vehicle information of other vehicles within a specified range of its own vehicle, and driving environment information; both the first vehicle information and the second vehicle information include vehicle position, vehicle speed, and acquisition time.

[0037] The temporal interaction dynamic feature acquisition module is used to calculate the difference between the first vehicle information and the second vehicle information, and use the difference as the interaction information between the vehicle itself and other vehicles. The interaction information and the second vehicle information are temporally encoded to obtain temporal interaction features that describe the temporal dependency between the interaction information and the second vehicle information. The temporal interaction dynamic features are obtained based on the temporal self-attention mechanism. The temporal interaction dynamic features are used to predict the driving intentions of other vehicles.

[0038] The spatial interaction dynamic feature module is used to extract spatial interaction features that describe the spatial interaction information of other vehicles based on the second vehicle information and driving environment information, and to obtain the spatial interaction dynamic features of the spatial interaction features based on the spatial interaction attention mechanism; the spatial interaction dynamic features are used to predict the driving intentions of other vehicles.

[0039] The trajectory prediction module is used to fuse temporal and spatial interactive dynamic features to obtain fused features, decode the fused features to obtain the driving intentions of other vehicles, and obtain the predicted trajectories of other vehicles based on the driving intentions.

[0040] Thirdly, the present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described vehicle trajectory prediction method.

[0041] Fourthly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described vehicle trajectory prediction method.

[0042] The beneficial effects of this invention are:

[0043] The vehicle trajectory prediction method provided by this invention calculates the interaction information between the vehicle itself and other vehicles, performs temporal encoding on the interaction information to obtain temporal interaction features, utilizes a temporal self-attention mechanism to obtain temporal interaction dynamic features, then obtains spatial interaction dynamic features based on vehicle information of other vehicles, driving environment information, and a spatial interaction attention mechanism, and finally combines the temporal and spatial interaction dynamic features to predict the vehicle trajectory. This method fully considers the temporal dependence and spatial interaction between the vehicle itself and other vehicles within a specified range, thus improving the accuracy of vehicle trajectory prediction. In addition, it also considers the influence of the driving environment on other vehicles, further improving the accuracy of vehicle trajectory prediction. Attached Figure Description

[0044] Figure 1 This is a flowchart of a vehicle trajectory prediction method in one embodiment of this application;

[0045] Figure 2 This is a schematic diagram of the vehicle trajectory prediction system in one embodiment of this application;

[0046] Figure 3 This is a schematic diagram of the structure of a terminal device in one embodiment of this application. Detailed Implementation

[0047] To address the low accuracy of traditional vehicle trajectory prediction methods, this invention discloses a vehicle trajectory prediction method, system, terminal device, and medium. The method calculates the interaction information between the vehicle and other vehicles, performs temporal encoding on this interaction information to obtain temporal interaction features, utilizes a temporal self-attention mechanism to obtain temporal interaction dynamic features, then obtains spatial interaction dynamic features based on vehicle information of other vehicles, driving environment information, and a spatial interaction attention mechanism, and finally combines the temporal and spatial interaction dynamic features to predict the vehicle trajectory. This method fully considers the temporal dependence and spatial interaction between the vehicle and other vehicles within a specified range, improving the accuracy of vehicle trajectory prediction. Furthermore, it considers the influence of the driving environment on other vehicles, further enhancing the accuracy of vehicle trajectory prediction.

[0048] The vehicle trajectory prediction method provided by this invention will be described in detail below.

[0049] like Figure 1 As shown, the vehicle trajectory prediction method specifically includes the following steps:

[0050] Step 11: Obtain the first vehicle information of your own vehicle, the second vehicle information of other vehicles within a specified range of your own vehicle, and the driving environment information.

[0051] In embodiments of the present invention, the first vehicle information and the second vehicle information both include vehicle position, vehicle speed (linear speed and acceleration) and acquisition time, and the driving environment information includes lane lines, curbs, drivable areas, road obstacles, traffic signs and traffic lights.

[0052] For example, vehicle-mounted sensors (including but not limited to lidar, cameras, and millimeter-wave radar) installed on the vehicle can collect driving environment information and vehicle information within a longitudinal distance of 90m before and after the vehicle, including the vehicle's driving lane and the two adjacent lanes. After locating the vehicle's position using onboard GPS, the positions of other vehicles can be obtained through simple calculations combined with radar data. This is merely an example; in other embodiments of the invention, the size of the specified range can be adjusted according to actual needs.

[0053] Step 12: Calculate the difference between the first vehicle information and the second vehicle information, use the difference as the interaction information between the vehicle itself and other vehicles, perform temporal encoding on the interaction information and the second vehicle information respectively, obtain the temporal interaction features used to describe the temporal dependency between the interaction information and the second vehicle information, and obtain the temporal interaction dynamic features based on the temporal self-attention mechanism.

[0054] In embodiments of the present invention, the interaction information between the vehicle itself and other vehicles is defined as the lateral position difference Δx, the longitudinal position difference Δy, and the speed difference Δv between the vehicle itself and other vehicles. Wherein, Δx = x0 - x i Δy=y0-y i Δv=v0-v i (x0, y0) represents the position of the vehicle itself, (x i ,y i The vehicle's position is represented by ), where i = 1, ..., M, M represents the total number of other vehicles within the specified range, and v0 represents the vehicle's speed. i The speed of the i-th other vehicle.

[0055] The aforementioned time-interaction dynamic features are used to predict the driving intentions of other vehicles. This invention divides driving intentions into lateral driving intentions and longitudinal driving intentions. Lateral driving intentions include left lane change, lane keeping, and right lane change, while longitudinal driving intentions include deceleration, acceleration, and cruise control.

[0056] Temporal interactive dynamic features are a form of spatial interactive features extracted through an attention mechanism, and are also feature vectors.

[0057] Step 13: Based on the second vehicle information and driving environment information, extract spatial interaction features to describe the spatial interaction information of other vehicles, and obtain the spatial interaction dynamic features of the spatial interaction features based on the spatial interaction attention mechanism.

[0058] The aforementioned spatial interaction dynamics are also used to predict the driving intentions of other vehicles, as represented by this feature.

[0059] Step 14: Fuse the temporal interaction dynamic features and spatial interaction dynamic features to obtain fused features, decode the fused features to obtain the driving intentions of other vehicles, and obtain the predicted trajectories of other vehicles based on the driving intentions.

[0060] Specifically, based on the driving intention, multiple candidate predicted trajectories of other vehicles and the probability of each candidate predicted trajectory can be obtained. The candidate trajectory with the highest probability is taken as the predicted trajectory of other vehicles. Based on the predicted trajectories of other vehicles, guidance can be provided for the driving decisions of autonomous vehicles. For example, in one embodiment of the present invention, if it is predicted that a vehicle 50 meters ahead of the vehicle is about to change lanes to the left, the autonomous driving system of the vehicle can prepare to decelerate or avoid the vehicle based on the trajectory prediction result of that vehicle.

[0061] To further understand the vehicle trajectory prediction method provided by the present invention, the process of performing time-series encoding on the interaction information and the second vehicle information in step 12 to obtain the time interaction features used to describe the time dependency between the interaction information and the second vehicle information will be explained in detail below.

[0062] Specifically, firstly, through a fully connected layer, the interaction information and the second vehicle information are embedded into a high-dimensional space vector to form an embedding vector. This process can be represented as: in, The embedding vector corresponding to the second vehicle information is a high-dimensional vector generated by the state information and interaction information through a fully connected layer. It is used to transform the original input into the input of the LSTM, where t = 1, 2, ..., T, and T represents the total time for trajectory prediction. f represents the embedding vector corresponding to the interaction information. c (·) indicates a fully connected S t Indicates the second vehicle information, I t W3 and W4 represent the interactive information and the trainable weight matrices.

[0063] Then, the Long Short-Term Memory (LSTM) network is used to obtain the time dependency between the interaction information and the second vehicle information. This process can be represented as: Where W1 and W2 represent the parameter matrices of the Long Short-Term Memory network, This represents the hidden state vector obtained from the state information at time t using LSTM. This represents the hidden state vector containing the time interaction information at time t obtained through LSTM. This represents the hidden state vector obtained from the state information at time t-1 using LSTM. This represents the hidden state vector of time interaction information at time t-1 obtained through LSTM. LSTM(·) represents a Long Short Memory network used to capture time dependencies.

[0064] Finally, through the calculation formula

[0065] H = concat(H) S H I )

[0066]

[0067] The temporal interaction feature H is obtained; where H S H represents the state information feature. I It represents the characteristics of time-interactive information, and concat(·) means concatenation.

[0068] The process of obtaining the temporal interaction dynamic features based on the temporal self-attention mechanism in step 12 is explained in detail below, specifically from steps 12.1 to 12.3.

[0069] Step 12.1: Construct the attention mechanism matrix based on the temporal interaction features; the expression for the attention mechanism matrix is ​​as follows:

[0070]

[0071] Among them, Q H Represents the query matrix, K H V represents the bond matrix. H The value matrix is ​​represented by APReLU(·), and the adaptive activation matrix is ​​represented by APReLU(·). This represents the learnable weight matrix;

[0072] Step 12.2, using the calculation formula We obtain self-attention; where d K K represents H The dimension of the horizontal quantity. The dimension of the horizontal quantity changes from the initial input to Q. H ,K H V H The mapping is determined by an invariant parameter, K. H Dimensions and Q H V H The sameness comes from the same input mapped through different parameter matrices; specifically, it's used to compare Q. H and K H The similarity is used to calculate the final output.

[0073] Attention mechanisms can be used to capture long-term dependencies in temporal inputs.

[0074] Step 12.3, using the calculation formula

[0075] Mult1-Attention=concat[Attention1, Attention2, ..., Attention h ]

[0076] We obtain the temporal interaction dynamic features Mult1, where Attention h The value represents the attention output corresponding to each attention head, and h represents the number of heads in the multi-head attention. In some embodiments of the present invention, h = 8.

[0077] The following is a detailed explanation of the process in step 13 of extracting spatial interaction features to describe spatial interaction information of other vehicles based on the second vehicle information and driving environment information.

[0078] First, the effective interaction space for other vehicles is defined. In embodiments of the present invention, the effective interaction space includes vehicles and the driving environment within a preset range of other vehicles. For example, in one embodiment of the present invention, in order to predict the trajectory of a vehicle 60 meters in front of the vehicle itself, this vehicle is taken as the target vehicle. The effective interaction space is defined as the vehicles within a longitudinal distance of 30 meters before and after the target vehicle, and the driving environment information within the six directions of the target vehicle's driving direction: directly in front, left front, right front, directly behind, left rear, and right rear.

[0079] Then, vehicle information (vehicle information of vehicles in the six directions of the target vehicle's driving direction: directly in front, left front, right front, directly behind, left rear, and right rear) and driving environment information (driving environment information within the range) corresponding to the effective interaction space are extracted from the second vehicle information and driving environment information.

[0080] Finally, through the calculation formula

[0081] g = concat(g) J g N )

[0082] g J =f c (J t W5)

[0083] g N =f c (N t W6)

[0084] We obtain the spatial interaction feature g; where g J g represents the embedding vector of vehicle information corresponding to the effective interaction space. N J represents the embedding vector of driving environment information corresponding to the effective interaction space. t N represents spatial interaction information between other vehicles and other vehicles within a preset range. t This represents the spatial interaction information between other vehicles and the driving environment within a preset range. The interaction information between vehicles includes relative speed and relative acceleration; the interaction information between vehicles and the environment includes environmental information affecting vehicle trajectory, such as speed limit signs, lane markings, and traffic lights. W5 and W6 represent trainable weight matrices. Since spatial interactions do not have strong temporal correlation, more emphasis is placed on the spatial interaction information at the current moment. Therefore, only a fully connected layer is needed to generate the embedding vector, without requiring LSTM processing of the embedding vector.

[0085] The process of obtaining spatial interaction dynamic features based on the spatial interaction attention mechanism in step 13 is explained below.

[0086] Specifically, through calculation formula

[0087] Mult2-SpaceAttention=conCat[SpaceAttention1, SpaceAttention2, ..., SpaceAttention h ]

[0088]

[0089]

[0090] We obtain the spatial interaction dynamic feature Mult2; where SpaceAttention represents spatial interaction attention, Q... g K represents the query matrix corresponding to spatial interaction attention. g V represents the key matrix corresponding to spatial interactive attention. g This represents the value matrix corresponding to spatial interaction attention. This represents the learnable weight matrix.

[0091] The process of step 14 is described below, as shown in steps 14.1 to 14.3.

[0092] Step 14.1: Fuse the temporal interaction dynamic features and the spatial interaction dynamic features to obtain the fused features.

[0093] Specifically, attention mechanisms can be used to fuse temporal and spatial interactive dynamic features.

[0094] Step 14.2: Decode the fused features to obtain the driving intentions of other vehicles.

[0095] Specifically, this refers to the probability of each driving intention.

[0096] Step 14.3: Obtain the predicted trajectories of other vehicles based on the driving intention.

[0097] Specifically, the driving intention is decoded using LSTM to generate the probability of the trajectory corresponding to each intention, and the highest probability is selected as the final output.

[0098] As can be seen from the above steps, the vehicle trajectory prediction method provided by this invention calculates the interaction information between the vehicle itself and other vehicles, performs temporal encoding on the interaction information to obtain temporal interaction features, uses a temporal self-attention mechanism to obtain temporal interaction dynamic features, then obtains spatial interaction dynamic features based on vehicle information of other vehicles, driving environment information, and a spatial interaction attention mechanism, and finally combines the temporal and spatial interaction dynamic features to predict the vehicle trajectory. This fully considers the temporal dependence and spatial interaction between the vehicle itself and other vehicles within a specified range, thus improving the accuracy of vehicle trajectory prediction. In addition, it also considers the influence of the driving environment on other vehicles, further improving the accuracy of vehicle trajectory prediction.

[0099] The vehicle trajectory prediction system provided by this invention will be described below.

[0100] like Figure 2 As shown, the vehicle trajectory prediction system 200 includes:

[0101] The information acquisition module 201 is used to acquire the first vehicle information of its own vehicle, the second vehicle information of other vehicles within a specified range of its own vehicle, and driving environment information; the first vehicle information and the second vehicle information both include vehicle position, vehicle speed, and acquisition time.

[0102] The time-interaction dynamic feature acquisition module 202 is used to calculate the difference between the first vehicle information and the second vehicle information, use the difference as the interaction information between the vehicle itself and other vehicles, perform time-series encoding on the interaction information and the second vehicle information respectively to obtain time-interaction features that describe the time dependency between the interaction information and the second vehicle information, and acquire time-interaction dynamic features based on the time-interaction self-attention mechanism; the time-interaction dynamic features are used to predict the driving intentions of other vehicles.

[0103] The spatial interaction dynamic feature module 203 is used to extract spatial interaction features to describe the spatial interaction information of other vehicles based on the second vehicle information and driving environment information, and to obtain the spatial interaction dynamic features of the spatial interaction features based on the spatial interaction attention mechanism; the spatial interaction dynamic features are used to predict the driving intentions of other vehicles.

[0104] The trajectory prediction module 204 is used to fuse the temporal interaction dynamic features and the spatial interaction dynamic features to obtain fused features, decode the fused features to obtain the driving intentions of other vehicles, and obtain the predicted trajectories of other vehicles based on the driving intentions.

[0105] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. Their specific functions and technical effects can be found in the method embodiments section, and will not be repeated here. Those skilled in the art will understand that, for the sake of convenience and brevity, the division of the above-mentioned functional units and modules is only used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0106] like Figure 3 As shown, embodiments of the present invention provide a terminal device, such as... Figure 3 As shown, the terminal device D10 of this embodiment includes: at least one processor D100 ( Figure 3 The diagram shows only one processor, a memory D101, and a computer program D102 stored in the memory D101 and executable on the at least one processor D100, wherein the processor D100 executes the computer program D102 to implement the steps in any of the above method embodiments.

[0107] Specifically, when the processor D100 executes the computer program D102, it acquires first vehicle information of its own vehicle, second vehicle information of other vehicles within a specified range of its own vehicle, and driving environment information; it calculates the difference between the first vehicle information and the second vehicle information, uses the difference as the interaction information between its own vehicle and other vehicles, performs temporal encoding on the interaction information and the second vehicle information respectively to obtain temporal interaction features describing the time dependency between the interaction information and the second vehicle information, and obtains temporal interaction dynamic features based on a temporal self-attention mechanism; it extracts spatial interaction features describing the spatial interaction information of other vehicles based on the second vehicle information and the driving environment information, and obtains spatial interaction dynamic features based on a spatial interaction attention mechanism; it fuses the temporal interaction dynamic features and the spatial interaction dynamic features to obtain fused features, decodes the fused features to obtain the driving intentions of other vehicles, and obtains the predicted trajectories of other vehicles based on the driving intentions. Specifically, by calculating the interaction information between the vehicle itself and other vehicles and temporally encoding the interaction information, temporal interaction features are obtained. Temporal interaction dynamic features are then obtained using a temporal self-attention mechanism. Spatial interaction dynamic features are then obtained based on vehicle information of other vehicles, driving environment information, and a spatial interaction attention mechanism. Finally, vehicle trajectory prediction is performed by combining the temporal and spatial interaction dynamic features. This approach fully considers the temporal dependence and spatial interaction between the vehicle itself and other vehicles within a specified range, thus improving the accuracy of vehicle trajectory prediction. In addition, the influence of the driving environment on other vehicles is also considered, further enhancing the accuracy of vehicle trajectory prediction.

[0108] The processor D100 can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0109] In some embodiments, the memory D101 may be an internal storage unit of the terminal device D10, such as a hard disk or memory of the terminal device D10. In other embodiments, the memory D101 may be an external storage device of the terminal device D10, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the terminal device D10. Furthermore, the memory D101 may include both internal and external storage units of the terminal device D10. The memory D101 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory D101 can also be used to temporarily store data that has been output or will be output.

[0110] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0111] This application provides a computer program product that, when run on a terminal device, enables the terminal device to implement the steps described in the various method embodiments above.

[0112] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of protection of this application is limited to these examples; within the framework of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of one or more embodiments of this application as described above, which are not provided in detail for the sake of brevity.

[0113] One or more embodiments in this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of this application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments in this application should be included within the protection scope of this application.

Claims

1. A vehicle trajectory prediction method, characterized by, The method comprises the following steps: obtaining first vehicle information of a self vehicle, second vehicle information of other vehicles within a specified range of the self vehicle, and driving environment information; the first vehicle information and the second vehicle information both comprise vehicle positions, vehicle speeds, and acquisition times; calculating a difference between the first vehicle information and the second vehicle information, taking the difference as interaction information between the self vehicle and the other vehicles, respectively performing time sequence encoding on the interaction information and the second vehicle information to obtain time interaction features for describing time dependence between the interaction information and the second vehicle information, and obtaining time interaction dynamic features of the time interaction features based on a time self-attention mechanism; the time interaction dynamic features are used for predicting driving intentions of the other vehicles; extracting spatial interaction features for describing spatial interaction information of the other vehicles based on the second vehicle information and the driving environment information, and obtaining spatial interaction dynamic features of the spatial interaction features based on a spatial interaction attention mechanism; the spatial interaction dynamic features are used for predicting driving intentions of the other vehicles; fusing the time interaction dynamic features and the spatial interaction dynamic features to obtain fused features, decoding the fused features to obtain the driving intentions of the other vehicles, and obtaining predicted trajectories of the other vehicles based on the driving intentions.

2. The vehicle trajectory prediction method of claim 1, wherein, The driving environment information comprises lane lines, road edges, drivable areas, road barriers, traffic signs, and signal lights.

3. The vehicle trajectory prediction method of claim 1, wherein, The time sequence encoding on the interaction information and the second vehicle information respectively to obtain the time interaction features for describing the time dependence between the interaction information and the second vehicle information comprises: the time interaction dynamic features of the time interaction features based on the time self-attention mechanism comprise: H = concat(H S ,H I ) obtaining the time interaction feature H; wherein H S denotes the state information feature, H I denotes the time interaction information feature, denotes the hidden state vector of the state information at t moment obtained by LSTM, denotes the hidden state vector of the time interaction information at t moment obtained by LSTM, denotes the hidden state vector of the state information at t-1 moment obtained by LSTM, denotes the hidden state vector of the time interaction information at t-1 moment obtained by LSTM, concat(·) denotes concatenation, LSTM(·) denotes long short-term memory network, and is used to capture the time dependence, denotes the embedding vector corresponding to the second vehicle information, t = 1, 2,..., T, and T denotes the total time of trajectory prediction, denotes the embedding vector corresponding to the interaction information, f c (·) denotes full connection, S t denotes the second vehicle information, I t denotes the interaction information, W1, W2 denote parameter matrices of long short-term memory network, and W3, W4 denote trainable weight matrices.

4. The vehicle trajectory prediction method of claim 3, wherein, constructing an attention mechanism matrix according to the time interaction features; the expression of the attention mechanism matrix is as follows: the extraction of the spatial interaction features for describing the spatial interaction information of the other vehicles based on the second vehicle information and the driving environment information comprises: where Q H represents a query matrix, K h represents a key matrix, V H represents a value matrix, Appelu(·) represents an adaptive activation matrix, represents a learnable weight matrix; The self-attention Attention is obtained by the calculation formula wherein d K denotes the dimension of the horizontal vector in the matrix K. The time interaction dynamic feature Mult1 is obtained by a calculation formula Mult1-Attention=concat[Attention1,Attention2,...,Attention h ], wherein Attention h represents the attention output corresponding to each attention head, and h represents the number of heads of the multi-head attention.

5. The vehicle trajectory prediction method of claim 4, wherein, defining an effective interaction space of the other vehicles; the effective interaction space comprises vehicles within a preset range of the other vehicles and driving environments; extracting vehicle information and driving environment information corresponding to the effective interaction space from the second vehicle information and the driving environment information; the spatial interaction dynamic features of the spatial interaction features based on the spatial interaction attention mechanism comprise: the driving intentions comprise left lane changing, lane keeping, right lane changing, deceleration, acceleration, and constant speed cruising. g = concat(g J ,g N ) g J = f c (J t ; W5) g N = f c (vN t ; W6) obtaining the spatial interaction feature g; wherein g J an embedding vector representing vehicle information corresponding to the effective interaction space, g N an embedding vector representing driving environment information corresponding to the effective interaction space, J t representing spatial interaction information between the other vehicles and vehicles within a preset range of the other vehicles, N t representing spatial interaction information between the other vehicles and driving environments within a preset range of the other vehicles, W5, W6 represent trainable weight matrices.

6. The vehicle trajectory prediction method of claim 5, wherein, The method comprises the following steps: an information acquisition module is configured to obtain first vehicle information of a self vehicle, second vehicle information of other vehicles within a specified range of the self vehicle, and driving environment information; Mult2-SpaceAttention = concat[SpaceAttention1, SpaceAttention2,..., SpaceAttention h ] obtain the space interaction dynamic feature Mult2; wherein, SpaceAttention represents a space interaction attention, Q g Q represents a query matrix of the space interaction attention, g K represents a key matrix of the space interaction attention, g V represents a value matrix of the space interaction attention, W represents a learnable weight matrix.

7. The vehicle trajectory prediction method of claim 6, wherein, the first vehicle information and the second vehicle information both comprise vehicle positions, vehicle speeds, and acquisition times; 8. A vehicle trajectory prediction system, characterized in that, ​ ​ ​ The time interaction dynamic feature acquisition module is configured to calculate a difference between the first vehicle information and the second vehicle information, take the difference as interaction information between the ego vehicle and the other vehicle, time-encode the interaction information and the second vehicle information respectively, obtain time interaction features for describing time dependence between the interaction information and the second vehicle information, and acquire time interaction dynamic features of the time interaction features based on a time self-attention mechanism. The time interaction dynamic features are used to predict the driving intention of the other vehicle. The spatial interaction dynamic feature module is configured to extract spatial interaction features for describing spatial interaction information of the other vehicle according to the second vehicle information and the driving environment information, and acquire spatial interaction dynamic features of the spatial interaction features based on a spatial interaction attention mechanism. The spatial interaction dynamic features are used to predict the driving intention of the other vehicle. The trajectory prediction module is configured to fuse the time interaction dynamic features and the spatial interaction dynamic features to obtain fused features, decode the fused features to obtain the driving intention of the other vehicle, and acquire a predicted trajectory of the other vehicle according to the driving intention.

9. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the vehicle trajectory prediction method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1-9. The computer program is executed by the processor to implement the vehicle trajectory prediction method according to any one of claims 1 to 7.

Images