Trajectory prediction model training method and electronic equipment

Abstract

The embodiment of the invention is suitable for the technical field of automatic driving, and provides a trajectory prediction model training method and electronic equipment, and the method comprises the steps: obtaining training traffic features; the training traffic features include a first lane feature, first historical trajectory features of all targets, and target position features corresponding to target points expected to pass by by the first vehicle in a future duration; the first vehicle is a vehicle to be subjected to trajectory prediction in the targets; based on the target position feature, the first lane feature and the first historical trajectory feature, generating a first predicted trajectory of the first vehicle in the future duration; calculating target training loss according to the first prediction trajectory and the target point; and iteratively updating model parameters of the initial trajectory prediction model based on the target training loss to obtain a trajectory prediction model. By adopting the method, the accuracy of outputting the predicted trajectory by the trained trajectory prediction model can be improved.

Description

Technical Field

[0001] This application belongs to the field of autonomous driving technology, and in particular relates to a trajectory prediction model training method and electronic device. Background Technology

[0002] In the field of autonomous driving, to ensure that vehicles can drive safely and efficiently in complex traffic, they typically need to be able to infer their future driving trajectory based on the historical trajectories of their own vehicles, other vehicles, and pedestrians in the current traffic scenario, as well as information such as lane markings. Therefore, training a trajectory prediction model suitable for vehicles is particularly important.

[0003] Currently, trajectory prediction models are typically trained by taking the aforementioned historical trajectories, lane markings, and other information as input, and outputting a predicted trajectory. Then, the training loss is calculated based on the predicted trajectory and the actual trajectory to iterate the trajectory prediction model.

[0004] However, the historical trajectories in the above training methods are dynamic and uncertain factors, and the correlation between lane lines and vehicles is poor. Therefore, the predicted trajectories output by the trajectory prediction models trained in this way have low correlation with vehicles, low controllability, and low interpretability. In other words, the trajectory prediction accuracy of the trajectory prediction models trained in this way is low. Summary of the Invention

[0005] This application provides a trajectory prediction model training method and electronic device, which can solve the problem of low accuracy of trajectory prediction model output.

[0006] In a first aspect, embodiments of this application provide a trajectory prediction model training method, the method comprising:

[0007] Acquire training traffic features; the training traffic features include the first lane features, the first historical trajectory features of all targets, and the target location features corresponding to the target points that the first vehicle is expected to pass through in the future time period; the first vehicle is the vehicle in the target whose trajectory is to be predicted.

[0008] Based on the target location features, the first lane features, and the first historical trajectory features, a first predicted trajectory of the first vehicle in the future time period is generated.

[0009] Calculate the target training loss based on the first predicted trajectory and the target point;

[0010] The trajectory prediction model is obtained by iteratively updating the model parameters of the initial trajectory prediction model based on the target training loss.

[0011] Secondly, embodiments of this application provide a trajectory prediction model training apparatus, the apparatus comprising:

[0012] The acquisition module is used to acquire training traffic features; the training traffic features include the first lane features, the first historical trajectory features of all targets, and the target location features corresponding to the target points that the first vehicle is expected to pass through in the future time period; the first vehicle is the vehicle in the target whose trajectory is to be predicted.

[0013] The generation module is used to generate the first predicted trajectory of the first vehicle in the future time period based on the target location features, the first lane features, and the first historical trajectory features.

[0014] The calculation module is used to calculate the target training loss based on the first predicted trajectory and the target point;

[0015] The iteration module is used to iteratively update the model parameters of the initial trajectory prediction model based on the target training loss, so as to obtain the trajectory prediction model.

[0016] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect above.

[0017] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect above.

[0018] Fifthly, embodiments of this application provide a computer program product that, when run on an electronic device, causes the electronic device to execute the method described in the first aspect.

[0019] The beneficial effects of this application embodiment compared with the prior art are as follows: When acquiring training traffic features as training data for the model, it includes not only the first lane features of the road and the first historical trajectory features of the target, but also the target location features corresponding to the target points that the first vehicle is expected to pass through in the future. Here, the first vehicle is the vehicle whose trajectory is to be predicted within the target. Then, the electronic device can generate the first predicted trajectory of the first vehicle in the future based on the target location features, the first lane features, and the first historical trajectory features. Furthermore, the target training loss is calculated by combining the first predicted trajectory and the target points to iteratively update the model parameters of the initial trajectory prediction model, thus obtaining the trajectory prediction model. Based on this, by introducing the target points that the first vehicle needs to reach as training traffic features, the first predicted trajectory output by the trajectory prediction model during training can be strongly correlated with the target points, thereby increasing the relevance, controllability, and interpretability of the output first predicted trajectory. Moreover, when calculating the target training loss, it is calculated not only based on the first predicted trajectory but also based on the target points. Furthermore, by using the semantically information-rich "target point" as input and incorporating it into the calculation of the target training loss, the first predicted trajectory output by the initial trajectory prediction model during iteration can be correlated with the target point and thus influenced by the semantic information of the target point. Based on this, the prediction accuracy of the final generated trajectory prediction model can be improved. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating the implementation of a trajectory prediction model training method according to an embodiment of this application;

[0022] Figure 2 This is a schematic diagram illustrating an implementation method for generating training traffic features in a trajectory prediction model training method provided in an embodiment of this application;

[0023] Figure 3 This is a schematic diagram illustrating one implementation method for generating training traffic features in a trajectory prediction model training method provided in another embodiment of this application;

[0024] Figure 4 This is a schematic diagram illustrating an implementation method for generating a first predicted trajectory in a trajectory prediction model training method provided in an embodiment of this application;

[0025] Figure 5This is a schematic diagram illustrating an implementation method for generating third lane features and third historical trajectory features in a trajectory prediction model training method provided in an embodiment of this application;

[0026] Figure 6 This is a schematic diagram illustrating an implementation method for generating a second predicted trajectory in a trajectory prediction model training method provided in an embodiment of this application;

[0027] Figure 7 This is a schematic diagram illustrating an application scenario of trajectory prediction model training in a trajectory prediction model training method provided in an embodiment of this application;

[0028] Figure 8 This is a schematic diagram of the structure of a trajectory prediction model training device provided in one embodiment of this application;

[0029] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0030] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0031] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0032] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0033] In the field of autonomous driving, to ensure that vehicles can drive safely and efficiently in complex traffic, they typically need to be able to infer their future driving trajectory based on the historical trajectories of their own vehicles, other vehicles, and pedestrians in the current traffic scenario, as well as information such as lane markings. Therefore, training a trajectory prediction model suitable for vehicles is particularly important.

[0034] Currently, trajectory prediction models are typically trained by taking the aforementioned historical trajectories, lane markings, and other information as input, and outputting a predicted trajectory. Then, the training loss is calculated based on the predicted trajectory and the actual trajectory to iterate the trajectory prediction model.

[0035] However, the historical trajectories in the above training methods are dynamic and uncertain factors, and the correlation between lane lines and vehicles is poor. Therefore, the predicted trajectories output by the trajectory prediction models trained in this way have low correlation with vehicles, low controllability, and low interpretability. In other words, the trajectory prediction accuracy of the trajectory prediction models trained in this way is low.

[0036] Based on this, in order to improve the trajectory prediction accuracy of the trajectory prediction model, one embodiment of this application provides a trajectory prediction model training method. This method can be applied to electronic devices such as tablet computers, laptops, ultra-mobile personal computers (UMPCs), and netbooks. This embodiment of the application does not impose any restrictions on the specific type of electronic device.

[0037] Please see Figure 1 , Figure 1 The following is a flowchart illustrating the implementation of a trajectory prediction model training method according to an embodiment of this application. The method includes the following steps:

[0038] S101. Obtain training traffic features.

[0039] In one embodiment, the aforementioned training traffic features include not only the first lane features and the first historical trajectory features of all targets, but also the target location features corresponding to the target points that the first vehicle is expected to pass through in the future. The first vehicle is the vehicle among the targets whose trajectory is to be predicted.

[0040] In one embodiment, the first lane features mentioned above include, but are not limited to, lane features of the current driving lane of the first vehicle (e.g., one or more features such as left lane, middle lane, single lane, turning lane, U-turn lane, etc.) and lane line features of the current driving lane (e.g., lane line position, dashed line, solid line, etc.), and there is no limitation thereto.

[0041] The aforementioned targets include the first vehicle whose trajectory is to be predicted, as well as obstacles around the first vehicle. For example, obstacles such as pedestrians and other movable second vehicles are not limited to these.

[0042] It should be noted that in real-world scenarios, the first vehicle typically uses radar or camera sensors to collect the trajectories of surrounding obstacles. However, the acquisition range of radar and camera sensors is limited. Therefore, during training, when collecting training data, the distance between the obstacles and the first vehicle should be less than the acquisition range of the radar and camera sensors to ensure that the acquired training traffic features more closely resemble actual conditions.

[0043] In one embodiment, the aforementioned future duration can be set according to actual circumstances and is not limited thereto. Typically, in the field of trajectory prediction, the trajectory prediction model needs to output the predicted trajectory of the first vehicle within a future time period based on input features, and calculate the loss based on the preset actual trajectory of the first vehicle within the future time period. Therefore, the future time period can be the aforementioned future duration.

[0044] As an example, during training, the aforementioned real trajectory can be the pre-set driving trajectory of the first vehicle in the next 30 frames. It should be noted that if trajectory data is collected at a sampling frame rate of 10 frames per second during training, then the driving trajectory of the aforementioned 30 frames is the driving trajectory within 3 seconds.

[0045] In one embodiment, the target point is the location that the first vehicle is expected to pass through in the future. That is, the target point can be considered as a trajectory point in the actual trajectory. For example, it can be the end point of the actual trajectory or an intermediate trajectory point, without limitation.

[0046] As an example, the target point mentioned above can be the center location of the target lane that the first vehicle is expected to reach. Furthermore, this center location is at a predetermined distance in the direction of travel of the first vehicle, and its horizontal direction can coincide with the center of the target lane. For example, the predetermined distance could be 50 meters.

[0047] The target lane can be the same as or different from the current lane of the first vehicle; there is no limitation on this.

[0048] In one embodiment, when acquiring the aforementioned training traffic features, corresponding traffic data can be acquired first, and then the traffic data can be encoded to obtain the aforementioned training traffic features. For example, the aforementioned traffic data includes, but is not limited to, lane map data, historical trajectory data of all targets, and preset target points.

[0049] The lane map data may include, but is not limited to, the lane the vehicle is currently traveling in, the desired destination lane, the corresponding lane lines, and one or more lane types (single lane, turning lane, U-turn lane, etc.). For specific examples, please refer to the example illustration of the first lane feature mentioned above.

[0050] In one embodiment, traffic data can be encoded using a pre-defined encoding network. For example, an MLP (Multilayer Perceptron) or other autoencoders can be used; there is no limitation on this method.

[0051] However, it should be noted that the training traffic features generated in the above manner usually cannot retain more detailed information that affects the behavior of the first vehicle, so that the training traffic features have a low effect on predicting the first predicted trajectory during the subsequent training of the initial trajectory prediction model.

[0052] Therefore, in order to ensure that the final generated training traffic features retain as much detailed information as possible that has a greater impact on the behavior of the first vehicle while preserving the information of each subject, the electronic device can, according to, such as Figure 2 The steps S201-S203 shown generate training traffic features. Details are as follows:

[0053] S201. Obtain traffic data in traffic scenarios; traffic data includes lane map data, historical trajectory data of all targets, and preset target points.

[0054] S202. For any traffic data, use a fully connected network to encode the traffic data to obtain initial traffic features.

[0055] In one embodiment, the traffic data has already been explained above and will not be described again. The fully connected network can be the MLP network described above, which can encode the traffic data and extract the individual subject information during the encoding process to obtain initial traffic features.

[0056] In another embodiment, the initial traffic features obtained after processing by the MLP network are usually large in dimensionality. In this case, if subsequent model processing is performed directly based on the initial traffic features, the computational load of the model will increase.

[0057] Therefore, after obtaining the initial traffic features, dimensionality reduction can be performed on the initial traffic features to obtain the final initial traffic features. For example, a max pooling layer can be used for dimensionality reduction to obtain the final initial traffic features.

[0058] As an example, the generation of the above initial traffic features can be represented as follows:

[0059]

[0060] Among them, A p M represents the initial historical trajectory feature in the initial traffic features. p P represents the initial lane feature in the initial traffic features. p This represents the initial location feature in the initial traffic characteristics. This indicates max pooling layer processing, and MLP indicates MLP network.

[0061] S203. Use the first multi-layer attention network to perform attention calculation on the initial traffic features to obtain the training traffic features.

[0062] In one embodiment, the Multi Head Attention Network is an attention network that stacks attention mechanisms across multiple layers. When using a multi-head attention network to calculate attention for initial traffic features, each layer of the attention network can capture features and contextual information at the same level as the initial traffic features.

[0063] For example, more detailed information can be captured in lower-level attention networks, while more abstract and global contextual information can be captured in higher-level attention networks. Furthermore, each layer of attention network can further refine and combine features based on the output of the previous layer to form higher-level features.

[0064] Specifically, the attention mechanism in each layer of the attention network can generate corresponding query vectors, key-value vectors, and numerical vectors based on the input data, and generate higher-level features through a standard attention calculation process, which serve as input data for the next layer of the attention network. The attention calculation process used in this embodiment is a general process and will not be described in detail.

[0065] As an example, electronic devices can be based on, for example Figure 3 The steps S301-S303 shown generate traffic training features. Details are as follows:

[0066] S301. For any layer of the first attention network, perform nearest neighbor algorithm and location encoding processing on the historical traffic features output by the previous layer of the first attention network to obtain nearest neighbor features and location encoded features.

[0067] In one embodiment, the number of layers in the first multi-layer attention network can be set according to actual conditions and is not limited thereto. It should be noted that the historical traffic features of the first layer of the first attention network are the initial traffic features, and the output of the last layer of the first attention network is the traffic training features.

[0068] In one embodiment, the nearest neighbor processing can be considered as using the k-nearest neighbor algorithm to query the historical trajectory features corresponding to the multiple obstacles closest to the first vehicle from the historical trajectory features corresponding to each obstacle in the historical traffic features.

[0069] Understandably, when predicting trajectories, the nearest obstacles to the first vehicle typically have a significant impact on its movement. Therefore, nearest neighbor processing can be performed on historical traffic features to obtain nearest neighbor features.

[0070] Furthermore, the aforementioned location encoding process can be considered as encoding the location information corresponding to historical trajectories, lane lines, target points, etc. in historical traffic features, so as to generate location encoded features that can be understood by the model.

[0071] S302. The nearest neighbor feature is determined as the first numerical vector in the first multi-layer attention network, the concatenation feature of the location encoding feature and the nearest neighbor feature is determined as the first key value vector in the first multi-layer attention network, and the concatenation feature of the location encoding feature and the historical traffic feature is determined as the first query vector in the first multi-layer attention network.

[0072] S303. Attention is calculated based on the first numerical vector, the first key vector, and the first query vector to obtain the current traffic features output by the first attention network of the current layer; the current traffic features output by the first attention network of the last layer are the training traffic features.

[0073] In one embodiment, based on the above description of the multi-layer attention network, it can be seen that when the first multi-layer attention network generates traffic training features, it needs to first generate the first query vector, the first key value vector, and the first numerical vector corresponding to each layer of the first attention network.

[0074] Based on this, the electronic device can determine the nearest neighbor feature as the first numerical vector, the concatenation feature of the location encoding feature and the nearest neighbor feature as the first key value vector, and the concatenation feature of the location encoding feature and the historical traffic feature as the first query vector, so as to perform attention calculation and obtain the current traffic feature output by the first attention network of the current layer.

[0075] Understandably, when the current layer's first attention network is not the last layer's first attention network, the current traffic features will be used as input to the next layer's first attention network. That is, relative to the next layer's first attention network, the current traffic features will become historical traffic features, and the above steps S301-S303 will be repeated until the training traffic features are output.

[0076] As an example, the generation of the above-mentioned training traffic features can be represented as follows:

[0077]

[0078] value=κ(G j-1 ));

[0079] Here, MultiHeadAttn() represents a multi-layer attention network module, query represents the first query vector, key represents the first key-value vector, and value represents the first numerical vector. G j-1This indicates that the first attention network at layer j-1 outputs historical traffic features. The input to the first attention network at layer j can be the initial value G. 0 =[A p M p ,P p This refers to the initial traffic characteristics. Indicates to G j-1 Perform general sinusoidal position encoding. That is, generate position encoded features. κ() represents the historical trajectory features corresponding to the multiple obstacles closest to the first vehicle based on the k-nearest neighbor algorithm. That is, generate nearest neighbor features.

[0080] The generated training traffic features can be represented as: First historical trajectory feature A encoder First lane feature M encoder and target location features P encoder .

[0081] Based on the above generated example, it can be seen that by refining and combining the features in the initial traffic features through the first multi-layer attention network, the final training traffic features can not only retain their respective main information (i.e., retain lane information, historical trajectory information and target point location information respectively), but also retain detailed information that has a greater impact on the behavior of the first vehicle (i.e., the nearest neighbor features extracted by the nearest neighbor algorithm).

[0082] In one embodiment, the model structure of the trajectory prediction model to be trained can be any of the model structures such as CNN (Convolutional Neural Networks) and RNN (Recurrent Neural Networks). In this embodiment, the model structure of the trajectory prediction model is not limited.

[0083] In this embodiment, the trajectory prediction model described above can be structured as a Transformer model, which is a deep learning model architecture used for natural language processing and other sequence-to-sequence tasks. It should be noted that this model incorporates a self-attention mechanism to process features. For example, a multi-layer attention network is introduced.

[0084] S102. Based on the target location features, the first lane features, and the first historical trajectory features, generate the first predicted trajectory of the first vehicle in the future time period.

[0085] In one embodiment, when training the trajectory prediction model, an initial trajectory prediction model can be set first. During the training process, the initial trajectory prediction model performs forward propagation based on the input training traffic features and outputs a first predicted trajectory. Then, the model parameters are updated backward based on the first predicted trajectory and the target point until the iterative convergence condition is met, thus obtaining the trajectory prediction model.

[0086] In this case, when the model structure of the initial trajectory prediction model is different, the processing methods for target position features, first lane features, and first historical trajectory features during the forward propagation process are usually different. In this embodiment, in order to improve the accuracy of the first predicted trajectory, it is also possible to further process the following: Figure 4 Steps S401-S402, as shown, process the above features and output the first predicted trajectory. Details are as follows:

[0087] S401. A second multi-layer attention network is used to perform attention calculations on the target location features, the first lane features, and the first historical trajectory features to obtain the second lane features and the second historical trajectory features.

[0088] In one embodiment, the number of layers in the second multi-layer attention network may be the same as or different from the number of layers in the first multi-layer attention network, and this is not limited. Based on the above explanation of multi-layer attention networks, it is understood that when the second multi-layer attention network performs attention calculations on the aforementioned features, it also needs to determine the query vector, numerical vector, and key-value vector corresponding to each layer of the second attention network.

[0089] The purpose of generating the second lane feature and the second historical trajectory feature is to refine and combine the target location feature, the first lane feature, and the first historical trajectory feature multiple times to further form the second lane feature and the second historical trajectory feature containing more details, thereby improving the accuracy of the subsequently generated first predicted trajectory. That is, in the second multi-layer attention network, the useful information obtained from the previous layer can be retained in the next layer, so that the finally generated second lane feature and second historical trajectory feature can gather useful information from multiple layers.

[0090] However, it should be noted that since the traffic training features incorporate the target location, and in order to ensure that the output first predicted trajectory is related to the semantically information-rich "target point" and to adjust the first predicted trajectory based on the semantic information of the target point, the corresponding predicted trajectory and target location features can be incorporated into the generation of the query vector, numerical vector, and key-value vector of each layer of the second attention network.

[0091] Specifically, it can be based on, for example Figure 5 The process is carried out according to steps S501-S503 shown. Details are as follows:

[0092] S501. For any layer of the second attention network, generate the target query vector of the current layer of the second attention network based on the endpoint location features, target location features, the third lane features output by the previous layer of the second attention network, and the third historical trajectory features.

[0093] In one embodiment, the aforementioned endpoint location feature is the location feature corresponding to a preset trajectory endpoint in the training traffic features. During the training process, multiple preset trajectories that a vehicle may travel on can be set, each trajectory corresponding to a trajectory endpoint. These multiple preset trajectories can be used to represent various driving scenarios for the first vehicle over a future time period. It should be noted that among the multiple preset trajectories, one trajectory is the actual trajectory.

[0094] Specifically, the electronic device can cluster the x and y coordinates of multiple trajectory endpoints to obtain multiple cluster centers, and define them as the intended endpoints Inten∈R. K×2 Where K is the number of cluster centers. Then, the electronic device can sinusoidally encode the cluster centers and input them into the MLP network to obtain the endpoint location features.

[0095] For example, the endpoint location feature can be represented as follows:

[0096] Q Inten =MLP(PE(Inten));

[0097] Among them, Q Inten The endpoint location features are represented by PE(Inten), which indicates sinusoidal position encoding of the cluster centers; MLP indicates that the input is processed in the MLP.

[0098] In one embodiment, the aforementioned third lane feature and third historical trajectory feature correspond to the second lane feature and second historical trajectory feature, respectively, with the only difference being that the second lane feature and second historical trajectory feature are the outputs of the last layer of the second attention network in the second multi-layer attention network. The third lane feature and third historical trajectory feature, on the other hand, are the outputs of each of the other layers of the second attention network. That is, in this embodiment, the terms "first," "second," and "third," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0099] It is understandable that the second lane feature and the second historical trajectory feature are obtained by processing the first lane feature and the first historical trajectory feature using a second multi-layer attention network. Therefore, the third lane feature and the third historical trajectory feature can be considered as intermediate products in the above processing. That is, when the target position feature, the first lane feature, and the first historical trajectory feature are input into the second multi-layer attention network, each layer of the second attention network will output the third lane feature and the third historical trajectory feature, which will then be used as input to the next layer of the second attention network, until the last layer of the second attention network outputs the second lane feature and the second historical trajectory feature.

[0100] It should be noted that the reason why the output third lane feature and third historical trajectory feature are involved in the updating and processing in the multi-layer second attention network is that the first lane feature and first historical trajectory feature that have an impact on the first vehicle are gradually strengthened in the processing of the multi-layer second attention network, so as to obtain the second lane feature and second historical trajectory feature that have a greater impact on the first vehicle.

[0101] In one embodiment, to establish and strengthen the connection between the target location features, second lane features, and second historical trajectory features corresponding to the target point, and the first predicted trajectory, the electronic device may first generate a second query vector related to the target location features and a third query vector related to the first predicted trajectory, and determine both the second query vector and the third query vector as target query vectors for each layer of the second attention network to participate in the feature processing of each layer of the second attention network. This strengthens the association between the aforementioned multiple features and the first predicted trajectory.

[0102] For example, the second query vector mentioned above can be generated as follows:

[0103] The electronic device can generate a second query vector for the current layer's second attention network based on the end point location features, target location features, the third lane features output by the previous layer's second attention network, and the third historical trajectory features.

[0104] In one embodiment, the electronic device can determine the second query vector as a concatenated feature between the endpoint location feature, the target location feature, the third lane feature, and the third historical trajectory feature. However, the features generated by the concatenation method may have a large dimensionality.

[0105] Based on this, in this embodiment, the electronic device can also use a second multi-layer attention network to process and extract the feature information that needs attention as the second query vector.

[0106] Specifically, the electronic device can determine the third lane feature and the third historical trajectory feature output by the previous layer's second attention network as the fourth numerical vector of the current layer's second attention network. It can also determine the concatenated features of the third lane feature, the third historical trajectory feature, the endpoint position feature, and the target position feature output by the previous layer's second attention network as the fourth key-value vector and the fourth query vector of the current layer's second attention network. Then, attention calculation is performed based on the fourth numerical vector, the fourth key-value vector, and the fourth query vector to obtain the second query vector of the current layer's second attention network.

[0107] As an example, the above representation of generating the second query vector can be illustrated as follows:

[0108]

[0109] key=C j-1 +Q Inten +P encoder ,

[0110] value = C j-1 );

[0111] in, Let C represent the second query vector corresponding to the second attention network in the j-th layer. j-1 This represents the third lane feature and the third historical trajectory feature of layer j-1 (i.e., when layer j is the current layer, layer j-1 is the previous layer). Q Inten P represents the endpoint location feature. encoder This represents the target location features. Here, `value` represents a numerical vector, `key` represents a key-value vector, and `query` represents a query vector. In the above formula, these correspond to the fourth numerical vector, the fourth key-value vector, and the fourth query vector, respectively. `MultiHeadAttn` indicates that attention calculation is performed.

[0112] It should be noted that when j equals 1, the corresponding third lane feature and third historical trajectory feature of layer 0 are the first lane feature and first historical trajectory feature in the above traffic training features.

[0113] Furthermore, in order to enable each layer of the second attention network to focus on the main information of the third lane features and the third historical trajectory features during the iterative update process, the third lane features and the third historical trajectory features can be processed by MLP to obtain the final third lane features and the third historical trajectory features.

[0114] Specifically, the representation of the third lane feature and the third historical trajectory feature output by each layer of the second attention network can be as follows:

[0115]

[0116] Among them, C j The third lane feature and third historical trajectory feature represent the final values ​​of the second attention network at layer j. MLP represents the MLP network. This represents the initial characteristics of the third lane in the j-th layer. This represents the initial third historical trajectory feature of the j-th layer.

[0117] Based on the above formula, it can be seen that in the second multi-layer attention network, the second query vector of the current layer can be updated based on the output of the previous layer's second attention network, the target position features of the target points related to the first vehicle, and the endpoint position features of the preset trajectory endpoint. Therefore, when generating corresponding features based on the second query vector, the current layer's second attention network can include more detailed information that affects the first vehicle.

[0118] It should be noted that when generating the second query vector based on the attention calculation method, using the target location features and the endpoint location features as fixed quantities for attention calculation can ensure that each layer of the second attention network always focuses on information related to the target.

[0119] In one embodiment, similar to the second query vector, the third query vector can be generated as follows:

[0120] The electronic device can generate the second predicted trajectory of the first vehicle corresponding to the previous layer second attention network based on the third lane feature and the third historical trajectory feature output by the previous layer second attention network, and generate the third query vector of the current layer second attention network based on the first predicted point in the second predicted trajectory; the first predicted trajectory is the second predicted trajectory corresponding to the last layer second attention network.

[0121] In one embodiment, based on the above explanation that the third lane feature and the third historical trajectory feature are intermediate products in a multi-layered second attention network, it can be determined that the third lane feature and the third historical trajectory feature output by each layer of the second attention network are gradually establishing and strengthening a connection with the first predicted trajectory. Based on this, it can be considered that the electronic device can also predict the second predicted trajectory of the first vehicle based on the third lane feature and the third historical trajectory feature, that is, satisfying the prediction conditions of the second predicted trajectory.

[0122] Since the third lane features and the third historical trajectory features are intermediate products, it can be assumed that the second predicted trajectory output by each layer of the second attention network (except for the last layer of the second attention network) is also an intermediate product and not the final first predicted trajectory.

[0123] In one embodiment, the electronic device may output a second predicted trajectory based on the third lane features and the third historical trajectory features, or it may first output multiple third predicted trajectories and determine a second predicted trajectory from the multiple third predicted trajectories, without limitation.

[0124] In this embodiment, in order to improve the prediction accuracy of the second predicted trajectory, the electronic device can, according to, such as Figure 6 Steps S601-S603, as shown, generate the second predicted trajectory. Details are as follows:

[0125] S601. Based on the features of the third lane and the features of the third historical trajectory, predict multiple third predicted trajectories of the first vehicle.

[0126] In one embodiment, the purpose of generating multiple third predicted trajectories is to enable the third predicted trajectories to better cover various possible trajectory scenarios in the future. For example, braking, acceleration, straight driving, left turn, right turn, etc., so that the results can better describe the uncertainty of the third predicted trajectory.

[0127] In one embodiment, the electronic device can perform MLP network processing on the third lane features and the third historical trajectory features again to obtain the aforementioned third predicted trajectory.

[0128] The generation of multiple third-predicted trajectories can be represented as follows:

[0129]

[0130] in, Let C represent the Gaussian distribution of the multiple third predicted trajectories obtained based on the third lane features and third historical trajectory features output by the second attention network at layer j; T represents the T time points (i.e., each third predicted trajectory corresponds to trajectory points at T time points); C j This represents the third lane feature and the third historical trajectory feature output by the second attention network in layer j.

[0131] S602. For any third predicted trajectory, generate the Gaussian distribution parameters corresponding to the third predicted trajectory based on the first trajectory information of the third predicted trajectory.

[0132] In one embodiment, the first trajectory information may include the position information of each trajectory point in the third predicted trajectory, or it may only include the position information of a certain trajectory point in the third predicted trajectory. For example, it may only include the position information of the trajectory endpoint in the third predicted trajectory.

[0133] In this embodiment, since it is necessary to output the Gaussian distribution parameters corresponding to the Gaussian distributions of each third predicted trajectory, the first trajectory information mentioned above includes the position information of each trajectory point in the third predicted trajectory.

[0134] The Gaussian distribution parameters mentioned above may include the mean μ of the x-coordinates of all trajectory points in the third predicted trajectory. x Standard deviation σ x Correlation coefficient ρx, mean μ of the ordinates of all trajectory points y Standard deviation σ y And the correlation coefficient ρy.

[0135] Based on the above explanation, the Gaussian distribution parameters corresponding to multiple third predicted trajectories can be represented as follows:

[0136]

[0137] in, Let R represent the Gaussian distribution parameters corresponding to the multiple third predicted trajectories obtained based on the third lane features and third historical trajectory features output by the j-th layer second attention network; R represents the Gaussian distribution parameters, S represents the number of third predicted trajectories, and 6 represents the dimension of the Gaussian distribution parameters. That is, 6-dimensional (μ x μ y σ x σ y , ρx, ρy).

[0138] S603. Based on Gaussian distribution parameters, determine the second predicted trajectory from multiple third predicted trajectories.

[0139] In one embodiment, after generating the Gaussian distribution parameters of all third predicted trajectories, the target probability of the first vehicle choosing to travel on a third predicted trajectory can be determined based on the Gaussian distribution parameters. Then, the third predicted trajectory corresponding to the maximum target probability can be determined as the second predicted trajectory, or the second predicted trajectory can be obtained by sampling from multiple third predicted trajectories based on each target probability; there is no limitation on this.

[0140] As an example, the second predicted trajectory can be represented as follows:

[0141]

[0142] in, This represents the second predicted trajectory based on the second attention network at layer j.

[0143] As explained above, each third predicted trajectory corresponds to a Gaussian distribution parameter. Therefore, the method for calculating the target probability can be considered the same as the method for calculating the Gaussian distribution probability, and no limitation is imposed on it.

[0144] For example, the above target probability can be calculated as follows:

[0145]

[0146] Where o represents the position of the first vehicle within the future time period, and P j (o) represents the probability that position o appears in the k-th predicted trajectory of the third prediction trajectory corresponding to the j-th layer of the second attention network; K represents the number of Gaussian distribution parameters corresponding to the third predicted trajectory; p k N represents the probability of selecting the k-th third predicted trajectory from a Gaussian distribution; k This represents the probability density function of the mixture of the parameters of the k-th Gaussian distribution, which is used to evaluate the above parameters (o x -μ x ,σ x ;o y -μ y ,σ y We can calculate the probability density using ρx,ρy).

[0147] It should be noted that after obtaining the target probability corresponding to each of the above third predicted trajectories, a second predicted trajectory can be obtained by sampling from multiple third predicted trajectories based on each target probability. This allows all generated third predicted trajectories to have a chance of being selected, thus making the generation of the second predicted trajectory more random and representative.

[0148] For example, consider three third predicted trajectories, A, B, and C. The target probability of third predicted trajectory A is 0.2, that of third predicted trajectory B is 0.3, and that of third predicted trajectory C is 0.5. In this case, the electronic device does not directly determine third predicted trajectory C as the second predicted trajectory, but instead performs random sampling. Specifically, among the three predicted trajectories, the probability of sampling third predicted trajectory A is 20%, that of sampling third predicted trajectory B is 30%, and that of sampling third predicted trajectory C is 50%.

[0149] It is understandable that when j is the last layer of the second attention network, the second predicted trajectory corresponding to the j-th layer of the second attention network is the first predicted trajectory in step S102 above.

[0150] The above description provides an example of obtaining the second predicted trajectory. Specifically, after obtaining the second predicted trajectory corresponding to the previous layer's second attention network, the electronic device can generate the third query vector corresponding to the current layer's second attention network based on the first predicted point in the second predicted trajectory of the previous layer's second attention network.

[0151] In one embodiment, the first predicted point can be the endpoint of the second predicted trajectory, or it can be a trajectory point at the same time as the target point, and there is no limitation thereto.

[0152] After obtaining the first predicted point, the electronic device can first acquire the location encoding features of the first predicted point, and then process the location encoding features using an MLP network to obtain the aforementioned third query vector. At this point, the third query vector can be considered as a query vector related to the second predicted trajectory.

[0153] As an example, the generated representation of the third query vector can be as follows:

[0154]

[0155] in, This represents the third query vector corresponding to the second attention network in the j-th layer; This represents the first predicted point at time T in the second predicted trajectory corresponding to the (j-1)th layer of the second attention network; This represents the positional coding feature obtained after sinusoidal positional coding of the first prediction point; MLP represents the MLP network.

[0156] The content following step S501 above is an explanation of the generation of target query vectors (second query vector and third query vector). Subsequently, the electronic device can also generate, according to steps S502 and S503, a second numerical vector and a second key-value vector for updating lane features in the current layer's second attention network, and a third numerical vector and a third key-value vector for updating trajectory features, respectively, to perform attention calculation based on the target query vectors and generate corresponding second lane features and second historical trajectory features.

[0157] S502. Based on the first lane feature, generate the second numerical vector and the second key value vector when updating the lane feature in the current layer second attention network, and perform attention calculation with the target query vector to generate the third lane feature corresponding to the current layer second attention network; the second lane feature is the third lane feature corresponding to the last layer second attention network.

[0158] As an example, an electronic device can use the first lane feature as the second numerical vector, the first lane feature and the position encoding feature corresponding to the first lane feature as the second key value vector, and combine the target query vector to perform attention calculation to generate the third lane feature of the current layer's second attention network.

[0159] Based on this, when generating the third lane features of each layer of the second attention network, the initial first lane features are used as a fixed quantity for attention calculation. This allows the lane-related information in the third lane features generated by each layer of the second attention network to be enhanced. Furthermore, by using the target query vector as a variable, the connection between the third lane features and the first predicted trajectory can be gradually established and strengthened.

[0160] As an example, the representation of generating the second lane feature described above can be illustrated as follows:

[0161]

[0162] key=[α(M encoder ),PE α(M) ],

[0163] value=α(M encoder ));

[0164] in, M represents the third lane feature corresponding to the second attention network in layer j. encoder Represents all the first lane features obtained, α(M) encoder ) represents the first lane feature located in a predetermined number of lanes surrounding the first vehicle among all first lane features; PE α(M) This represents the position-coded feature obtained after sinusoidal position coding of the first lane feature. This represents the second query vector corresponding to the second attention network in the j-th layer. This represents the third query vector corresponding to the j-th layer of the second attention network; MultiHeadAttn indicates that attention calculation is performed. Where j is the last layer of the second attention network, This is the characteristic of the second lane.

[0165] S503. Based on the first historical trajectory features, generate the third numerical vector and the third key value vector when updating the trajectory features in the current layer of the second attention network, and perform attention calculation with the target query vector to generate the third historical trajectory features corresponding to the current layer of the second attention network; the second historical trajectory features are the third historical trajectory features corresponding to the last layer of the second attention network.

[0166] In one embodiment, the method for generating the second historical trajectory feature is similar to the method for generating the second lane feature, and will not be explained further.

[0167] For example, the representation of the generated second historical trajectory features can be described as follows:

[0168]

[0169] key = [A encoder PE Aencoder ],

[0170] value = A encoder );

[0171] in, A represents the third historical trajectory feature corresponding to the second attention network in the j-th layer.encoder Represents all acquired first historical trajectory features; PE α(M) This represents the position-coded feature obtained after sinusoidal position coding of the first historical trajectory feature. This represents the second query vector corresponding to the second attention network in the j-th layer. This represents the third query vector corresponding to the j-th layer of the second attention network; MultiHeadAttn indicates that attention calculation is performed. Where j is the last layer of the second attention network, This is the second historical trajectory feature.

[0172] S402. Generate the first predicted trajectory based on the features of the second lane and the features of the second historical trajectory.

[0173] In one embodiment, the above Figure 6 Steps S601-S603 have described how to generate the second predicted trajectory of the first vehicle based on the third lane features and the third historical trajectory features. The method for generating the first predicted trajectory is similar to the method for generating the second predicted trajectory, and will not be described further.

[0174] S103. Calculate the target training loss based on the first predicted trajectory and the target point.

[0175] In one embodiment, the target training loss is calculated based on the first predicted trajectory and the target point. This can be achieved by generating a trajectory prediction loss based on the first predicted trajectory and a preset real trajectory; simultaneously, a prediction point loss is generated based on a second predicted point corresponding to the first predicted trajectory and the target point. Finally, the trajectory prediction loss and / or the prediction point loss are determined as the aforementioned target training loss.

[0176] In this embodiment, in order to perform loss calculation on the first predicted trajectory selected by the trajectory prediction model to improve the accuracy of the first predicted trajectory, and at the same time perform loss calculation on the deviation of the first predicted trajectory from the target point to be passed through, to further improve the accuracy of the first predicted trajectory, the electronic device can determine the sum of trajectory prediction loss and prediction point loss as the above-mentioned target training loss.

[0177] Understandably, by calculating the target training loss using two different attributes, the subsequent iterations of the initial trajectory prediction model based on this target training loss can more comprehensively optimize the prediction performance of the initial trajectory prediction model, ensuring that the final trajectory prediction model outputs the best performance under various attribute requirements. That is, it not only ensures that the output first predicted trajectory has the highest possible overlap with the true trajectory but also passes through the pre-set target point.

[0178] As an example, the electronic device can calculate the trajectory prediction loss and prediction point loss as described above, as detailed below:

[0179]

[0180] Among them, Loss gaussian This represents the trajectory prediction loss; (μ) x μ y σ x σ y ρx, ρy) represent the Gaussian distribution parameters corresponding to the first predicted trajectory h; N represents the mean of the x and y coordinates of the true trajectory. h This represents the probability density function of the mixture of the h-th Gaussian distribution parameters, which is used to evaluate the above parameters. Calculations are performed to obtain the corresponding probability density; p h This represents the Gaussian distribution probability corresponding to the first predicted trajectory h.

[0181] Loss goal =|P goal，v -Y m，y |;

[0182] Among them, Loss goal P represents the loss at the prediction point. goal，v The y-coordinate of the target point represents its position. m，y This represents the ordinate position of the m-th trajectory point (the second predicted point) in the first predicted trajectory.

[0183] It should be noted that, as explained in S101 above, the target point can be the center position of the target lane to which the first vehicle is expected to arrive. Furthermore, this center position is located at a predetermined distance in the direction of travel of the first vehicle, and its horizontal direction can coincide with the center of the target lane. Therefore, when calculating the predicted point loss, only the distance between the vertical coordinates needs to be considered.

[0184] It should be added that, in order to accurately calculate the loss at the predicted point, the m-th trajectory point should correspond to the target point. For example, the target point is a preset distance in the direction of travel of the first vehicle, and is the position that the first vehicle is expected to pass through in the future time period.

[0185] However, in real-world prediction scenarios, the distance of the first predicted trajectory corresponding to the predicted future duration may be less than the preset distance. In this case, the first predicted trajectory typically does not contain a second predicted point corresponding to the target point.

[0186] Therefore, to improve the accuracy of predicting point loss calculation, the electronic device can determine the travel time of the first vehicle to the target point based on its current position and speed. Then, it determines the minimum travel time between the current travel time and the future travel time. Finally, the prediction point with the minimum travel time in the first predicted trajectory is determined as the second prediction point.

[0187] Both the current location and the current vehicle speed can be preset and are not limited. The electronic device can determine the aforementioned travel time by the ratio of the ordinate distance between the current location and the target point to the current vehicle speed.

[0188] It should be noted that when the travel time is less than or equal to the future travel time, the first predicted trajectory can be considered to contain a second predicted point that corresponds to the target point in time.

[0189] For example, if the travel time is 2 seconds and the future travel time is 3 seconds, then the trajectory point corresponding to the 2nd second in the first predicted trajectory can be considered as the second predicted point corresponding to the target point in time. That is, the predicted point with the shortest duration (2 seconds) in the first predicted trajectory is determined as the second predicted point.

[0190] Furthermore, when the travel time exceeds the future travel time, it can be assumed that there is no second predicted point in the first predicted trajectory that corresponds to the target point in time. In this case, to reduce the loss of calculated predicted points, the electronic device can determine the predicted point in the first predicted trajectory for the future travel time as the second predicted point. That is, the endpoint of the first predicted trajectory is determined as the aforementioned second predicted point.

[0191] For example, if the travel time is 5 seconds and the future time is 3 seconds, it can be assumed that there is no second predicted point in the first predicted trajectory that corresponds to the target point in time. In this case, the predicted point with the shortest duration (3 seconds) in the first predicted trajectory needs to be determined as the second predicted point. That is, the trajectory endpoint of the first predicted trajectory is determined as the aforementioned second predicted point.

[0192] As an example, the second prediction point can be determined as follows:

[0193]

[0194] in, This indicates that the first vehicle starts from its current position and travels at its current speed v. x The travel time required to reach the preset distance s′ is represented by F, which indicates the sampling frame rate of the first vehicle trajectory (e.g., sampling at 10 frames per second). For example, t represents the number of sampling frames corresponding to the future duration. The expression indicates that the calculated value is rounded down, and min() means taking the minimum value of the two parameters.

[0195] S104. Iteratively update the model parameters of the initial trajectory prediction model based on the target training loss to obtain the trajectory prediction model.

[0196] In one embodiment, the method for iteratively updating the model parameters includes, but is not limited to, gradient descent, momentum update, learned annealing, and other methods, and no limitation is made thereto.

[0197] It should be noted that if the initial trajectory prediction model does not meet the convergence condition after iterating the model parameters, the above steps S102-S104 need to be repeated based on the iterated initial trajectory model until the convergence condition is met and the trajectory prediction model is obtained.

[0198] The convergence conditions include, but are not limited to, reaching a preset number of iterations or the target training loss being less than a preset loss. The preset number of iterations and the preset loss can be set according to actual conditions.

[0199] In this embodiment, when acquiring training traffic features as training data for the model, it includes not only the first lane features of the road and the first historical trajectory features of the target, but also the target location features corresponding to the target points that the first vehicle is expected to pass through in the future. The first vehicle is the vehicle whose trajectory is to be predicted within the target. Then, the electronic device can generate the first predicted trajectory of the first vehicle in the future based on the target location features, the first lane features, and the first historical trajectory features. Furthermore, the target training loss is calculated by combining the first predicted trajectory and the target points to iteratively update the model parameters of the initial trajectory prediction model, thus obtaining the trajectory prediction model. Based on this, by introducing the target points that the first vehicle needs to reach as training traffic features, the first predicted trajectory output by the trajectory prediction model during training can be strongly correlated with the target points, thereby increasing the relevance, controllability, and interpretability of the output first predicted trajectory. Moreover, when calculating the target training loss, it is calculated not only based on the first predicted trajectory but also based on the target points. Furthermore, by using the semantically information-rich "target point" as input and incorporating it into the calculation of the target training loss, the first predicted trajectory output by the initial trajectory prediction model during iteration can be correlated with the target point and thus influenced by the semantic information of the target point. Based on this, the prediction accuracy of the final generated trajectory prediction model can be improved.

[0200] In another embodiment, reference is made to Figure 7 , Figure 7 This is a schematic diagram illustrating an application scenario of trajectory prediction model training in an embodiment of this application. The diagram uses the vehicle in the target as the first vehicle, other obstacles as the second vehicle, and a Transformer model for trajectory prediction as an example for explanation.

[0201] During training, electronic devices can acquire traffic data in traffic scenarios and input it into the encoder in the Transformer model for processing to obtain training traffic features.

[0202] For example, traffic data can be lane map data (e.g., lane line data), historical trajectory data of all vehicles, and preset target points. The processing can involve encoding the traffic data using a fully connected network to obtain initial traffic features, and then using a first multi-layer attention network to perform attention calculations on the initial traffic features to obtain trained traffic features.

[0203] Subsequently, the electronic device can input all the trained traffic features into the decoder in the Transformer model for processing and trajectory prediction. At this point, after acquiring the target location features, the first lane features, and the first historical trajectory features, the electronic device can input the first lane features and the first historical trajectory features into the MLP network for processing, obtaining the third lane features and the third historical trajectory features corresponding to the second attention network at layer 0. That is, C 0 .

[0204] Afterwards, electronic devices can be based on C 0 After performing steps S601-S603 once, multiple third-predicted trajectories corresponding to the second attention network at layer 0 are obtained. Then, the second predicted trajectory is determined from multiple third predicted trajectories. in, Figure 7 The Gaussian mixture model in C is the same as that in C. 0 The model structure after executing steps S601-S603 once.

[0205] Then, in order to generate the third lane feature and the third historical trajectory feature C of the first layer second attention network, it is necessary to generate the third lane feature and the third historical trajectory feature. 1 At this time, we can first consider the features of the endpoint location, the target location, the third lane features output by the second attention network in the previous layer, and the third historical trajectory features (i.e., C). 0 Generate the second query vector of the second attention network in the current layer. That is, generating the second query vector of the first layer second attention network.

[0206] In addition, based on the third lane feature and the third historical trajectory feature (i.e., C) output by the second attention network in the previous layer. 0 This generates the second predicted trajectory of the first vehicle corresponding to the second attention network in the previous layer. That is, Y... 0 Furthermore, the electronic device can be based on the second predicted trajectory Y. 0 The first prediction point in the vector is used to generate the third query vector of the second attention network in the current layer. That is, generating the third query vector of the first layer second attention network. Furthermore, based on and The target query vector in the first layer of the second attention network is obtained.

[0207] Then, the electronic device can generate a second numerical vector and a second key-value vector based on the first lane feature, during the lane feature update in the first-layer second attention network, and perform attention calculation with the target query vector in the first-layer second attention network to generate the third lane feature corresponding to the first-layer second attention network.

[0208] Furthermore, based on the first historical trajectory features, a third numerical vector and a third key-value vector are generated during trajectory feature updates in the first-layer second attention network, and attention is calculated with the target query vector to generate the third historical trajectory features corresponding to the first-layer second attention network.

[0209] At this time, and The input is fed into an MLP network for processing, resulting in the third lane feature and the third historical trajectory feature corresponding to the first layer and second attention network. That is, C 1 That is, in the above process, Figure 7 Target query vector in Third lane characteristics and third historical trajectory characteristics (C j All of these need to be updated.

[0210] Then, the above steps can be repeated until C is obtained as the output of the second attention network in the last layer when j is the last layer. j And based on C j Output the first predicted trajectory. That is, first based on C j Output the Gaussian distribution of the multiple third predicted trajectories obtained from the second attention network at layer j. That is, Figure 7 In Then, based on Output the second predicted trajectory corresponding to the second attention network in layer j. That is, Figure 7 In Wherein, when j is the last layer of the second attention network, the second predicted trajectory corresponding to the j-th layer of the second attention network. This is the first predicted trajectory.

[0211] Finally, the electronic device can calculate the trajectory prediction loss and prediction point loss based on the first predicted trajectory to iterate the model parameters of the initial trajectory prediction model until the convergence condition is met, thus generating the aforementioned trajectory prediction model.

[0212] Please see Figure 8 , Figure 8 This is a structural block diagram of a trajectory prediction model training device provided in an embodiment of this application. The modules included in the trajectory prediction model training device in this embodiment are used to execute... Figures 1 to 6 The steps in the corresponding embodiments. Please refer to the details. Figures 1 to 6 as well as Figures 1 to 6 The relevant descriptions in the corresponding embodiments are shown below. For ease of explanation, only the parts relevant to this embodiment are shown. See also... Figure 8 The trajectory prediction model training device 800 may include: an acquisition module 810, a generation module 820, a calculation module 830, and an iteration module 840, wherein:

[0213] The acquisition module 810 is used to acquire training traffic features; the training traffic features include the first lane features, the first historical trajectory features of all targets, and the target location features corresponding to the target points that the first vehicle is expected to pass through in the future time period; the first vehicle is the vehicle in the target whose trajectory is to be predicted.

[0214] The generation module 820 is used to generate a first predicted trajectory of the first vehicle in the future time period based on the target location features, the first lane features, and the first historical trajectory features.

[0215] The calculation module 830 is used to calculate the target training loss based on the first predicted trajectory and the target point.

[0216] The iteration module 840 is used to iteratively update the model parameters of the initial trajectory prediction model based on the target training loss, so as to obtain the trajectory prediction model.

[0217] In one embodiment, the acquisition module 810 is further configured to:

[0218] Acquire traffic data in traffic scenarios; traffic data includes lane map data, historical trajectory data of all targets, and preset target points; for any traffic data, encode the traffic data using a fully connected network to obtain initial traffic features; use a first-layer attention network to perform attention calculation on the initial traffic features to obtain training traffic features.

[0219] In one embodiment, the acquisition module 810 is further configured to:

[0220] For any layer of the first attention network, the historical traffic features output by the previous layer of the first attention network are processed by the nearest neighbor algorithm and location encoding respectively to obtain the nearest neighbor feature and the location encoded feature; the historical traffic features of the first layer of the first attention network are the initial traffic features; the nearest neighbor feature is determined as the first numerical vector in the first multi-layer attention network, the concatenation feature of the location encoded feature and the nearest neighbor feature is determined as the first key-value vector in the first multi-layer attention network, and the concatenation feature of the location encoded feature and the historical traffic features is determined as the first query vector in the first multi-layer attention network; attention is calculated based on the first numerical vector, the first key-value vector, and the first query vector to obtain the current traffic feature output by the current layer of the first attention network; the current traffic feature output by the last layer of the first attention network is the training traffic feature.

[0221] In one embodiment, the generation module 820 is further configured to:

[0222] A second multi-layer attention network is used to perform attention calculations on the target location features, the first lane features, and the first historical trajectory features to obtain the second lane features and the second historical trajectory features; the first predicted trajectory is generated based on the second lane features and the second historical trajectory features.

[0223] In one embodiment, training traffic features further includes endpoint location features corresponding to a preset trajectory endpoint; the generation module 820 is also used for:

[0224] For any layer of the second attention network, based on the endpoint location features, target location features, the third lane features output by the previous layer of the second attention network, and the third historical trajectory features, a target query vector for the current layer of the second attention network is generated. Based on the first lane features, a second numerical vector and a second key-value vector are generated when the lane features in the current layer of the second attention network are updated, and attention is calculated with the target query vector to generate the third lane feature corresponding to the current layer of the second attention network. The second lane feature is the third lane feature corresponding to the last layer of the second attention network. Based on the first historical trajectory features, a third numerical vector and a third key-value vector are generated when the trajectory features in the current layer of the second attention network are updated, and attention is calculated with the target query vector to generate the third historical trajectory feature corresponding to the current layer of the second attention network. The second historical trajectory feature is the third historical trajectory feature corresponding to the last layer of the second attention network.

[0225] In one embodiment, the generation module 820 is further configured to:

[0226] Based on the endpoint location features, target location features, third lane features output by the previous layer's second attention network, and third historical trajectory features, a second query vector for the current layer's second attention network is generated. Based on the third lane features output by the previous layer's second attention network and the third historical trajectory features, a second predicted trajectory for the first vehicle corresponding to the previous layer's second attention network is generated. Based on the first predicted point in the second predicted trajectory, a third query vector for the current layer's second attention network is generated. The first predicted trajectory is the second predicted trajectory corresponding to the last layer's second attention network. The second query vector and the third query vector are determined as the target query vector.

[0227] In one embodiment, the generation module 820 is further configured to:

[0228] The third lane feature and the third historical trajectory feature output by the previous layer's second attention network are determined as the fourth numerical vector of the current layer's second attention network. The concatenated feature of the third lane feature, the third historical trajectory feature, the endpoint position feature, and the target position feature output by the previous layer's second attention network is determined as the fourth key value vector and the fourth query vector of the current layer's second attention network. Attention is calculated based on the fourth numerical vector, the fourth key value vector, and the fourth query vector to obtain the second query vector of the current layer's second attention network.

[0229] In one embodiment, the generation module 820 is further configured to:

[0230] Based on the features of the third lane and the features of the third historical trajectory, multiple third predicted trajectories of the first vehicle are predicted; for any third predicted trajectory, Gaussian distribution parameters corresponding to the third predicted trajectory are generated according to the first trajectory information of the third predicted trajectory; based on the Gaussian distribution parameters, a second predicted trajectory is determined from multiple third predicted trajectories.

[0231] In one embodiment, the calculation module 830 is further configured to:

[0232] Based on the first predicted trajectory and the preset real trajectory, a trajectory prediction loss is generated; based on the second predicted point corresponding to the first predicted trajectory and the target point, a prediction point loss is generated; the sum of the trajectory prediction loss and the prediction point loss is determined as the target training loss.

[0233] In one embodiment, the trajectory prediction model training device 800 further includes:

[0234] The first determining module is used to determine the travel time of the first vehicle to the target point based on the current position and current speed of the first vehicle.

[0235] The second determining module is used to determine the minimum duration between the travel duration and the future duration.

[0236] The third determination module is used to determine the prediction point with the shortest duration in the first prediction trajectory as the second prediction point.

[0237] When it is understood that, Figure 8 In the structural block diagram of the trajectory prediction model training device shown, each module is used to perform... Figures 1 to 6 The steps in the corresponding embodiments, and for Figures 1 to 6 The steps in the corresponding embodiments have been explained in detail in the above embodiments. Please refer to them for details. Figures 1 to 6 as well as Figures 1 to 6 The relevant descriptions in the corresponding embodiments will not be repeated here.

[0238] Figure 9 This is a structural block diagram of an electronic device provided in one embodiment of this application. Figure 9 As shown, the electronic device 900 of this embodiment includes: a processor 910, a memory 920, and a computer program 930 stored in the memory 920 and executable by the processor 910, such as a program for a trajectory prediction model training method. When the processor 910 executes the computer program 930, it implements the steps in the various embodiments of the trajectory prediction model training methods described above, for example... Figure 1 S101 to S104 are shown. Alternatively, the processor 910 implements the above when executing the computer program 930. Figure 8 The functions of each module in the corresponding embodiments, for example, Figure 8 For details on the functions of each module shown, please refer to [link / reference]. Figure 8 The relevant descriptions in the corresponding embodiments.

[0239] For example, the computer program 930 can be divided into one or more modules, one or more of which are stored in the memory 920 and executed by the processor 910 to implement the trajectory prediction model training method provided in the embodiments of this application. One or more modules can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 930 in the electronic device 900. For example, the computer program 930 can implement the trajectory prediction model training method provided in the embodiments of this application.

[0240] Electronic device 900 may include, but is not limited to, processor 910 and memory 920. Those skilled in the art will understand that... Figure 9 This is merely an example of electronic device 900 and does not constitute a limitation on electronic device 900. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device may also include input / output devices, network access devices, buses, etc.

[0241] The processor 910 may be a central processing unit, or it may be other general-purpose processors, digital signal processors, application-specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0242] The memory 920 can be an internal storage unit of the electronic device 900, such as a hard disk or RAM of the electronic device 900. The memory 920 can also be an external storage device of the electronic device 900, such as a plug-in hard disk, smart memory card, flash memory card, etc., equipped on the electronic device 900. Furthermore, the memory 920 can include both internal storage units and external storage devices of the electronic device 900.

[0243] This application provides a computer-readable storage medium, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the trajectory prediction model training method as described in the above embodiments.

[0244] This application provides a computer program product that, when run on an electronic device, causes the electronic device to execute the trajectory prediction model training method described in the above embodiments.

[0245] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A trajectory prediction model training method, characterized in that, The method comprises: acquiring training traffic features; the training traffic features comprise first lane features, first historical trajectory features of all targets, and target position features corresponding to target points expected to be passed by a first vehicle in a future time length; the first vehicle is a vehicle to be subjected to trajectory prediction among the targets; generating a first predicted trajectory of the first vehicle in the future time length based on the target position features, the first lane features, and the first historical trajectory features; calculating a target training loss according to the first predicted trajectory and the target points; iteratively updating model parameters of an initial trajectory prediction model based on the target training loss to obtain the trajectory prediction model.

2. The method of claim 1, wherein, The acquiring training traffic features comprises: acquiring traffic data in a traffic scene; the traffic data comprises lane map data, historical trajectory data of all the targets, and preset target points; encoding the traffic data by using a fully connected network to obtain initial traffic features for any traffic data; performing attention calculation on the initial traffic features by using a first multi-layer attention network to obtain the training traffic features.

3. The method of claim 2, wherein, The performing attention calculation on the initial traffic features by using the first multi-layer attention network to obtain the training traffic features comprises: performing nearest neighbor algorithm and position encoding processing on historical traffic features output by a previous layer of the first attention network to obtain nearest neighbor features and position encoding features for any layer of the first attention network; the historical traffic features of the first layer of the first attention network are the initial traffic features; determining the nearest neighbor features as a first numerical vector in the first multi-layer attention network, determining a splicing feature of the position encoding features and the nearest neighbor features as a first key-value vector in the first multi-layer attention network, and determining a splicing feature of the position encoding features and the historical traffic features as a first query vector in the first multi-layer attention network; performing attention calculation based on the first numerical vector, the first key-value vector, and the first query vector to obtain current traffic features output by a current layer of the first attention network; the current traffic features output by the last layer of the first attention network are the training traffic features.

4. The method of claim 1, wherein, The generating a first predicted trajectory of the first vehicle in the future time length based on the target position features, the first lane features, and the first historical trajectory features comprises: performing attention calculation on the target position features, the first lane features, and the first historical trajectory features by using a second multi-layer attention network to obtain second lane features and second historical trajectory features; generating the first predicted trajectory according to the second lane features and the second historical trajectory features.

5. The method of claim 4, wherein, The training traffic features further comprise end position features corresponding to preset trajectory endpoints; The performing attention calculation on the target position features, the first lane features, and the first historical trajectory features by using the second multi-layer attention network to obtain the second lane features and the second historical trajectory features comprises: For any second attention network of a layer, a target query vector of the second attention network of the current layer is generated according to the end position feature, the target position feature, third lane features output by a second attention network of a previous layer and third historical trajectory features; Second numerical vectors and second key-value vectors of the second attention network of the current layer are generated according to the first lane features, and attention calculation is performed on the second numerical vectors and the second key-value vectors and the target query vector to generate third lane features corresponding to the second attention network of the current layer; the second lane features are third lane features corresponding to a second attention network of a last layer; Third numerical vectors and third key-value vectors of the second attention network of the current layer are generated according to the first historical trajectory features, and attention calculation is performed on the third numerical vectors and the third key-value vectors and the target query vector to generate third historical trajectory features corresponding to the second attention network of the current layer; the second historical trajectory features are third historical trajectory features corresponding to the second attention network of the last layer.

6. The method of claim 5, wherein, The target query vector of the second attention network of the current layer is generated according to the end position feature, the target position feature, third lane features output by a second attention network of a previous layer and third historical trajectory features, and includes: The second query vector of the second attention network of the current layer is generated according to the end position feature, the target position feature, the third lane features output by the second attention network of the previous layer and the third historical trajectory features; The second predicted trajectory of the first vehicle corresponding to the second attention network of the previous layer is generated based on the third lane features output by the second attention network of the previous layer and the third historical trajectory features, and a third query vector of the second attention network of the current layer is generated according to a first prediction point in the second predicted trajectory; the first predicted trajectory is the second predicted trajectory corresponding to the second attention network of the last layer; The second query vector and the third query vector are determined as the target query vector.

7. The method of claim 6, wherein, The second query vector of the second attention network of the current layer is generated according to the end position feature, the target position feature, the third lane features output by the second attention network of the previous layer and the third historical trajectory features, and includes: The third lane features output by the second attention network of the previous layer and the third historical trajectory features are determined as fourth numerical vectors of the second attention network of the current layer, and a splicing feature of the third lane features output by the second attention network of the previous layer and the third historical trajectory features, the end position feature and the target position feature is determined as fourth key-value vectors and fourth query vectors of the second attention network of the current layer; Attention calculation is performed based on the fourth numerical vectors, the fourth key-value vectors and the fourth query vectors to obtain the second query vector of the second attention network of the current layer.

8. The method of claim 6, wherein, The second predicted trajectory of the first vehicle corresponding to the second attention network of the previous layer is generated based on the third lane features output by the second attention network of the previous layer and the third historical trajectory features, and includes: predict a plurality of third predicted trajectories of the first vehicle based on the third lane feature and the third historical trajectory feature; for any one of the third predicted trajectories, generate a Gaussian distribution parameter corresponding to the third predicted trajectory according to first trajectory information of the third predicted trajectory; determine the second predicted trajectory from the plurality of third predicted trajectories based on the Gaussian distribution parameter.

9. The method of claim 1, wherein, the calculating a target training loss according to the first predicted trajectory and the target point comprises: generating a trajectory prediction loss according to the first predicted trajectory and a preset real trajectory; generating a prediction point loss based on a second prediction point corresponding to the first predicted trajectory and the target point; determining the sum of the trajectory prediction loss and the prediction point loss as the target training loss.

10. The method of claim 9, wherein, before the generating a prediction point loss based on a second prediction point corresponding to the first predicted trajectory and the target point, the method further comprises: determining a driving time length for the first vehicle to drive to the target point according to a current position and a current speed of the first vehicle; determining a minimum time length from the driving time length and the future time length; determining a prediction point in the first predicted trajectory at the minimum time length as the second prediction point.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, the processor executes the computer program to implement the method in any one of claims 1 to 10.

Images