Trajectory prediction model generation method and device, model training method and device

Abstract

The present disclosure provides a trajectory prediction model generation method and device, a model training method and device, which can be applied to the technical field of intelligent driving. The trajectory prediction model generation method comprises: determining a target task category corresponding to a first driving scene according to first scene information associated with the first driving scene; in the case that a target trajectory prediction model corresponding to the target task category is not contained in a plurality of trajectory prediction models pre-stored in a model library, calling a parameter tuning model from the model library, and calling at least one trajectory prediction model from the plurality of trajectory prediction models as a basic trajectory prediction model; updating parameters of the parameter tuning model based on the first scene information to obtain a target parameter tuning model; and adjusting parameters of the basic trajectory prediction model by using the target parameter tuning model to generate a target trajectory prediction model.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of intelligent driving, in particular to a trajectory prediction model generation method and device, a model training method and device, equipment, a medium and a program product. BACKGROUND

[0002] In automatic driving, trajectory prediction is located downstream of the perception module and upstream of the planning control, and plays a role of connecting the upstream and downstream in the entire automatic driving algorithm module. In the driving process of an automatic driving vehicle, important information such as lane information, environmental information, high-definition map, trajectory intention of other vehicles and pedestrians (obstacles), path planning, etc. needs to be considered comprehensively to ensure the driving safety of the vehicle.

[0003] In the process of implementing the present disclosure, the inventors have found that at least the following problems exist in the related art: The prediction models currently used for trajectory prediction are trajectory prediction models designed for special scenarios or specific obstacles, but in actual application, such models are difficult to be well applicable after the scenario is switched, the model generalization is low, and the models cannot be applied to other scenarios. SUMMARY

[0004] In view of the above problems, the present disclosure provides a trajectory prediction model generation method and device, a model training method and device, equipment, a medium and a program product.

[0005] In one aspect of the present disclosure, a trajectory prediction model generation method is provided, comprising:

[0006] determining a target task category corresponding to the first driving scenario according to first scene information associated with the first driving scenario;

[0007] in a case where it is determined that the plurality of trajectory prediction models pre-stored in the model library do not contain a target trajectory prediction model corresponding to the target task category, calling a parameter tuning model from the model library, and calling at least one trajectory prediction model from the plurality of trajectory prediction models as a basic trajectory prediction model;

[0008] updating parameters of the parameter tuning model based on the first scene information to obtain a target parameter tuning model;

[0009] adjusting parameters of the basic trajectory prediction model by using the target parameter tuning model to generate a target trajectory prediction model.

[0010] According to the embodiments of the present disclosure, the above method further comprises:

[0011] storing the generated target trajectory prediction model in the model library;

[0012] receive second scene information associated with a second driving scene, the second scene information comprising: second motion characteristic information of a second host obstacle, second traffic indication information of a lane where the second host obstacle is located, and second relative motion information of a second environmental obstacle in the second driving scene, wherein the second driving scene corresponds to the target task category;

[0013] invoke a target trajectory prediction model corresponding to the target task category from a plurality of trajectory prediction models;

[0014] input the second scene information into the target trajectory prediction model, and output predicted trajectory information of the second host obstacle in the second driving scene by the target trajectory prediction model.

[0015] According to an embodiment of the present disclosure, wherein the updating the parameters of the tuning model based on the first scene information to obtain the target tuning model comprises:

[0016] input the first scene information into the base trajectory prediction model to output reference trajectory information of the first host obstacle;

[0017] obtain sample trajectory information of the first host obstacle;

[0018] calculate a loss value corresponding to the target task category according to the reference trajectory information of the first host obstacle and the sample trajectory information of the first host obstacle;

[0019] update the parameters of the tuning model based on the loss value corresponding to the target task category to obtain the target tuning model.

[0020] According to an embodiment of the present disclosure, wherein the calculating the loss value corresponding to the target task category according to the reference trajectory information of the first host obstacle and the sample trajectory information of the first host obstacle comprises:

[0021] calculate an average displacement error and an end point displacement error of the first host obstacle according to the reference trajectory information of the first host obstacle and the sample trajectory information of the first host obstacle;

[0022] determine a first weight corresponding to the average displacement error and a second weight corresponding to the end point displacement error according to a driving scene category corresponding to the first driving scene;

[0023] calculate the loss value corresponding to the target task category according to the average displacement error, the end point displacement error, the first weight, and the second weight.

[0024] According to an embodiment of the present disclosure, wherein:

[0025] The first scenario information includes: in the first driving scenario, the first motion characteristic information of the first main obstacle, the first traffic indication information of the lane where the first main obstacle is located, and the first relative motion information of the first environmental obstacle;

[0026] The first motion characteristic information of the first main obstacle includes: the category of the first main obstacle, and the reference velocity information and reference position information of the first main obstacle before the target time node;

[0027] The first traffic indication information for the lane where the first main obstacle is located includes: lane line direction information for the lane where the first main obstacle is located, and traffic light status information for the lane where the first main obstacle is located.

[0028] The first relative motion information of the first environmental obstacle includes: the relative distance between the first environmental obstacle and the first main obstacle, and the relative velocity of the first environmental obstacle relative to the first main obstacle.

[0029] Another aspect of this disclosure provides a model training method, including:

[0030] Obtain multiple task sample sets corresponding to multiple task categories, wherein each task sample set includes multiple sets of historical sample information associated with multiple historical driving scenarios;

[0031] Based on the meta-learning algorithm, the parameters of the training parameter tuning model and the training trajectory prediction model are updated using multiple task sample sets to obtain the trained parameter tuning model and multiple trajectory prediction models. Among them, the multiple trajectory prediction models correspond one-to-one with multiple task categories. The target trajectory prediction model corresponding to the target task category is used to predict the trajectory of the main obstacle in multiple driving scenarios under the target task category.

[0032] According to embodiments of this disclosure, multiple sets of historical sample information in each task sample set are classified into training sample sets and test sample sets. Updating the parameters of the parameter tuning model and the trajectory prediction model to be trained using multiple task sample sets includes:

[0033] Based on the training sample set of the nth task sample set and the parameter tuning model to be trained after the (n-1)th update, the parameters of the trajectory prediction model to be trained are updated to obtain the nth trajectory prediction model corresponding to the nth task category.

[0034] Based on the test sample group in the nth task sample set and the nth trajectory prediction model, determine the loss value corresponding to the nth task category;

[0035] Based on the loss value corresponding to the nth task category, the parameters of the model to be trained and tuned are updated for the nth time.

[0036] The parameter of the to-be-trained trajectory prediction model is updated based on the training sample set in the (n+1)th task sample set and the to-be-trained tuning model after the n th update to obtain an (n+1)th trajectory prediction model, the loss value corresponding to the (n+1)th task category is determined based on the test sample set in the (n+1)th task sample set and the (n+1)th trajectory prediction model, and the parameter of the to-be-trained tuning model is updated for the (n+1)th time based on the loss value corresponding to the (n+1)th task category, until the sum of the loss values corresponding to the plurality of task categories is less than a preset threshold, to obtain the trained tuning model and the plurality of trajectory prediction models.

[0037] According to an embodiment of the present disclosure, wherein:

[0038] The historical sample information of each group includes historical motion characteristic information of a main obstacle, historical traffic indication information of a lane where the main obstacle is located, historical trajectory information of the main obstacle, and historical relative motion information of an environmental obstacle in a respective historical driving scene;

[0039] The plurality of historical driving scenes are classified into historical training driving scenes and historical test driving scenes, the historical training driving scenes are associated with the training sample set, and the historical test driving scenes are associated with the test sample set;

[0040] The loss value corresponding to the nth task category is determined based on the test sample set in the nth task sample set and the nth trajectory prediction model, and includes:

[0041] The historical motion characteristic information of the main obstacle, the historical traffic indication information of the lane where the main obstacle is located, and the historical relative motion information of the environmental obstacle in the test sample set in the nth task sample set are input into the nth trajectory prediction model, and nth group historical test driving scene under the main obstacle prediction trajectory sample information is output, wherein the nth group historical test driving scene is a plurality of historical training driving scenes associated with the test sample set in the nth task sample set.

[0042] The loss value corresponding to the nth task category is calculated based on the main obstacle prediction trajectory sample information in the nth group historical test driving scene and the historical trajectory information of the main obstacle in the nth group historical test driving scene.

[0043] According to an embodiment of the present disclosure, wherein the loss value corresponding to the nth task category is calculated based on the main obstacle prediction trajectory sample information in the nth group historical test driving scene and the historical trajectory information of the main obstacle in the nth group historical test driving scene, and includes:

[0044] According to the main obstacle prediction trajectory sample information under the nth set of historical test driving scenes and the main obstacle historical trajectory information under the nth set of historical test driving scenes, a sample average displacement error and a sample end point displacement error associated with the nth task category are calculated;

[0045] According to the driving scene category corresponding to the nth set of historical test driving scenes, a sample first weight corresponding to the sample average displacement error is determined, and a sample second weight corresponding to the sample end point displacement error is determined;

[0046] According to the sample average displacement error, the sample end point displacement error, the sample first weight and the sample second weight associated with the nth task category, a loss value corresponding to the nth task category is calculated.

[0047] Another aspect of the present disclosure provides a trajectory prediction model generation device, comprising:

[0048] A determination module is configured to determine a target task category corresponding to a first driving scene according to first scene information associated with the first driving scene.

[0049] A first calling module is configured to call a parameter adjustment model from a model library and call at least one trajectory prediction model from a plurality of trajectory prediction models as a basic trajectory prediction model when the plurality of trajectory prediction models pre-stored in the model library do not contain a target trajectory prediction model corresponding to the target task category.

[0050] A first parameter adjustment module is configured to update parameters of the parameter adjustment model based on the first scene information to obtain a target parameter adjustment model.

[0051] A second parameter adjustment module is configured to adjust parameters of the basic trajectory prediction model using the target parameter adjustment model to generate the target trajectory prediction model.

[0052] Another aspect of the present disclosure provides a model training device, comprising:

[0053] An acquisition module is configured to acquire a plurality of task sample sets corresponding to a plurality of task categories, wherein each task sample set includes a plurality of sets of historical sample information associated with a plurality of historical driving scenes.

[0054] A training module is configured to update parameters of a to-be-trained parameter adjustment model and a to-be-trained trajectory prediction model using a plurality of task sample sets based on a meta-learning algorithm to obtain a trained parameter adjustment model and a plurality of trajectory prediction models, wherein the plurality of trajectory prediction models correspond one-to-one to the plurality of task categories, and a target trajectory prediction model corresponding to a target task category is used to predict trajectories of a main obstacle in a plurality of driving scenes under the target task category.

[0055] Another aspect of the present disclosure provides an electronic device, comprising: one or more processors; a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the above trajectory prediction model generation method or model training method.

[0056] Another aspect of the present disclosure also provides a computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the above trajectory prediction model generation method or model training method.

[0057] Another aspect of the present disclosure also provides a computer program product comprising a computer program that, when executed by a processor, implements the above trajectory prediction model generation method or model training method.

[0058] According to embodiments of the present disclosure, to solve the defect that the scene limitation of the trajectory prediction model in the related art is relatively large, the method of the embodiments of the present disclosure stores a pre-trained hyperparameter tuning model and a plurality of trajectory prediction models in a model library, and the same trajectory prediction model can be used to predict the trajectories of obstacles in a plurality of scenes under the same task category. The hyperparameter tuning model is used to tune the trajectory prediction model according to different scenes to adapt to trajectory prediction under different task scenes. Therefore, after determining the task category corresponding to the current driving scene and determining that there is no prediction model suitable for the current driving scene in the current model library, the trajectory prediction model is tuned according to the new scene information by using the hyperparameter tuning model, which can adapt to trajectory prediction under a new task scene. In this way, the defect that the scene limitation of the trajectory prediction model in the related art is relatively large can be solved, and trajectory prediction under different scenes can be quickly adapted, so that trajectory prediction is not limited by the scene and the type of obstacle, and has strong scene generalization ability.

[0059] Further, in the embodiments of the present disclosure, the process of tuning the trajectory prediction model according to the new scene information by using the hyperparameter tuning model is different from the traditional deep learning. The parameters of the hyperparameter tuning model are pre-trained, and when facing a new task scene, the hyperparameter tuning does not need to start from scratch, nor does it need to retrain with a large amount of labeled data. Only a small amount of data under the new scene is needed to automatically fine-tune the model parameters, without starting from scratch for each driving scene and spending a lot of time to train and test the effect as in the related art. The period of model training is greatly shortened, and the efficiency of model adjustment is improved. BRIEF DESCRIPTION OF DRAWINGS

[0060] The above and other objects, features and advantages of the present disclosure will become more apparent from the following description when taken in conjunction with the accompanying drawings, in which:

[0061] Figure 1 An application scenario diagram of the trajectory prediction model generation method, apparatus, device, medium and program product according to an embodiment of the present disclosure is schematically shown;

[0062] Figure 2 A flowchart of the trajectory prediction model generation method according to an embodiment of the present disclosure is schematically shown;

[0063] Figure 3 A principle diagram of trajectory prediction using the trajectory prediction model according to an embodiment of the present disclosure is schematically shown;

[0064] Figure 4 A flowchart of the model training method according to an embodiment of the present disclosure is schematically shown;

[0065] Figure 5 A principle diagram of the model training method according to an embodiment of the present disclosure is schematically shown;

[0066] Figure 6 A structural block diagram of the trajectory prediction model generation apparatus according to an embodiment of the present disclosure is schematically shown;

[0067] Figure 7 A structural block diagram of the model training apparatus according to an embodiment of the present disclosure is schematically shown;

[0068] Figure 8 A block diagram of an electronic device suitable for implementing the trajectory prediction model generation method or the model training method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0069] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood, however, that the description is merely for illustration, and is not intended to limit the scope of the present disclosure. In the following detailed description of embodiments of the present disclosure, numerous specific details are set forth in order to provide a thorough understanding of the embodiments. However, it would be apparent to those skilled in the art that the embodiments, or one or more embodiments, can be practiced without these specific details. In other instances, well-known structures and

[0070] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the present disclosure. As used herein, the term "includes" and "including" means the inclusion without limitation of the stated features, steps, operations, and / or components.

[0071] All terms used herein, including technical and scientific terms, have the meanings commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the use of any terms herein should not be interpreted to imply a certain technical detail or method unless otherwise defined.

[0072] In the case of using expressions similar to "at least one of A, B, and C", etc., it is generally intended to include any of the A, B, and C, to include any combination of A, B, and C, and to include all of A, B, and C, unless otherwise defined.

[0073] In embodiments of the present disclosure, the collection, updating, analysis, processing, use, transmission, provision, disclosure, storage, etc. of data (for example, including but not limited to user personal information) involved are in accordance with the provisions of relevant laws and regulations, are used for legal purposes, and do not violate public order and good customs. In particular, necessary measures are taken for user personal information to prevent illegal access to user personal information data, and to maintain user personal information security, network security and national security.

[0074] In embodiments of the present disclosure, the authorization or consent of the user is obtained before the user's personal information is acquired or collected.

[0075] In the related art, trajectory prediction models are trajectory prediction models designed for special scenarios or specific obstacles, but in actual use, such models are difficult to apply well after the scenario is switched. For example, the methods for obstacle trajectory prediction in the related art mainly include the following categories: trajectory prediction based on obstacle kinematics model, trajectory prediction based on lane, and trajectory prediction based on deep learning (for example, trajectory prediction based on transformer, trajectory prediction based on VectorNet, trajectory prediction based on TNT and related improved versions).

[0076] For example, the trajectory prediction model based on obstacle kinematics is mainly suitable for trajectory prediction of cars on high-speed and urban roads, and the prediction result is unstable for cars on urban auxiliary roads and rural roads, and is not suitable for trajectory prediction of bicycles, electric vehicles and pedestrians, and the intention of the obstacle itself is difficult to determine. The method based on the obstacle kinematics model deduces the future trajectory of the obstacle based on the perception result, which is more consistent with the actual motion trajectory of the vehicle. At present, in order to avoid the error caused by perception, the perception result is filtered by traditional and deep learning methods, which can realize cross-environment use without training data. However, this kind of method cannot well capture the complexity of the real world, and the transfer (derivation) equation lacks global information in space and time, which may obtain a local optimal solution, and cannot completely eliminate the perception error, and the prediction trajectory is closely related to the input information transmitted by perception.

[0077] For example, the lane-based trajectory prediction method is limited to the lane lines provided by the map, and when the autonomous vehicle reaches the area not marked on the map, the trajectory prediction model for other obstacles is difficult to adapt well.

[0078] For example, the trajectory prediction method based on deep learning is limited to the type of obstacle and the scene where the host vehicle is located, that is, the car can perform stably on high-speed or regular urban roads, but when entering other scenes, the trajectory prediction method cannot be well applied, which is mainly because the distribution of data in the real scene deviates from the training data of the model.

[0079] The above several types of trajectory prediction models in the related art are all designed for special scenes or specific obstacles, such as trajectory algorithm models designed for intersection scenes, or trajectory prediction models designed for pedestrians. Therefore, the sample data used in the training process is also data in a specific scene, and therefore the model trained in this way is difficult to adapt well after the scene is switched.

[0080] Therefore, the embodiments of the present disclosure provide a trajectory prediction method, which comprises:

[0081] Before the target time node, a plurality of frames of reference driving scene graphs related to the target vehicle are continuously collected according to a first predetermined time interval; a plurality of sets of state information are obtained after the plurality of frames of reference driving scene graphs are preprocessed, wherein the state information is used to represent traffic indication information and obstacle information displayed by the reference driving scene graph; the plurality of sets of state information are input into a trajectory prediction model, and a plurality of prediction position information of the target vehicle after the target time node is output by the trajectory prediction model, wherein the plurality of prediction position information is continuously distributed according to a second predetermined time interval.

[0082] Figure 1An application scenario diagram of the trajectory prediction model generation method, apparatus, device, medium, and program product according to an embodiment of the present disclosure is schematically shown.

[0083] As shown in Figure 1 The application scenario 100 according to this embodiment can include a vehicle 101, a trajectory prediction module 102. The vehicle 101 and the trajectory prediction module 102 can communicate through a network, which can include various connection types, such as wired, wireless communication links, or optical fiber cables, and the like.

[0084] An automatic driving module is installed in the vehicle 101 for guiding the vehicle to realize automatic driving. An image acquisition device, such as a camera, is installed in the automatic driving module for acquiring surrounding environment images during the driving of the vehicle 101, and a source radar can also be installed for acquiring surrounding environment point cloud data.

[0085] The trajectory prediction module 102 is located downstream of the perception module and upstream of the planning control, and plays a role of connecting the upstream and downstream in the entire automatic driving algorithm module. During driving, the autonomous vehicle needs to comprehensively consider important information such as lane information, environmental information, high-definition map, trajectory intention of other vehicles and pedestrians (obstacles), path planning, etc., to ensure its own driving safety.

[0086] In the application scenario of the present disclosure, the vehicle 101 needs to know the predicted trajectory of the surrounding obstacles during driving in order to take measures to avoid obstacles in advance. The data acquisition device installed in the vehicle 101 acquires surrounding environment images and surrounding environment point cloud data, etc., and converts them into scene information of the surrounding environment, such as motion characteristic information of the main obstacle, traffic indication information of the lane where the main obstacle is located, relative motion information of the environmental obstacle, etc., and sends these scene information to the trajectory prediction module 102. The trajectory prediction module 102 can determine the corresponding target task category according to the scene information associated with the driving scene, call the target trajectory prediction model corresponding to the target task category from multiple trajectory prediction models, output the predicted trajectory information of the main obstacle through the target trajectory prediction model, and return to the vehicle 101, telling the vehicle 101 the driving intention of the obstacle, so that the vehicle 101 makes corresponding decisions to avoid collision accidents.

[0087] The following will describe the scene based on Figure 1 The trajectory prediction model generation method and model training method of the present disclosure will be described in detail. Figures 2-8

[0088] Figure 2 A flowchart of the trajectory prediction model generation method according to an embodiment of the present disclosure is schematically shown. Figure 3 ​A schematic diagram illustrating a principle of trajectory prediction using a trajectory prediction model according to an embodiment of the present disclosure is shown. The following describes the principle of trajectory prediction using the trajectory prediction model according to the embodiment of the present disclosure in conjunction with Figure 2 , Figure 3 The trajectory prediction model generation method according to the embodiment of the present disclosure is described.

[0089] As shown in Figure 2 , the trajectory prediction model generation method according to the embodiment includes operations S201-S204.

[0090] In operation S201, a target task category corresponding to a first driving scene is determined according to first scene information associated with the first driving scene, where the first scene information includes: first motion characteristic information of a first main obstacle, first traffic indication information of a lane where the first main obstacle is located, and first relative motion information of a first environmental obstacle in the first driving scene.

[0091] In operation S202, in a case where it is determined that a target trajectory prediction model corresponding to the target task category is not included in a plurality of trajectory prediction models pre-stored in a model library, a parameter-adjusting model is invoked from the model library, and at least one trajectory prediction model from the plurality of trajectory prediction models is invoked as a base trajectory prediction model.

[0092] In operation S203, parameters of the parameter-adjusting model are updated based on the first scene information to obtain a target parameter-adjusting model.

[0093] In operation S204, parameters of the base trajectory prediction model are adjusted using the target parameter-adjusting model to generate the target trajectory prediction model.

[0094] According to the embodiments of the present disclosure, the above method can be applied to scenarios of obstacle zone trajectory prediction in different driving scenes. Different driving scenes refer to driving scenes formed by random combinations of different types of obstacles and different types of roads. For example, the types of obstacles include: family cars, medium-sized trucks, large-sized trucks, trailers, electric vehicles, motorcycles, bicycles, pedestrians, wheelchairs, buses, dogs, cats, urban work vehicles, special-purpose vehicles (police cars, fire engines, ambulances, etc.), and the like. For example, the types of roads include: urban roads without traffic lights, urban roads with traffic lights, non-intersection urban roads, urban road auxiliary roads, and rural roads without traffic lights, etc. For example, a driving scene is: a bus driving on an urban road without traffic lights, an electric vehicle driving on an urban road without traffic lights, a bicycle driving on a non-intersection urban road, etc.

[0095] According to embodiments of this disclosure, multiple similar driving scenarios belong to the same task category. For example, driving scenarios associated with the same road type can belong to the same task category. For instance, a driving scenario involving a bus traveling on an urban road without traffic lights, an electric vehicle traveling on an urban road without traffic lights, and a pedestrian traveling on an urban road without traffic lights belong to the first task category; a driving scenario involving a bus traveling on an urban road with traffic lights, an electric vehicle traveling on an urban road with traffic lights, and a pedestrian traveling on an urban road with traffic lights belong to the second task category, and so on.

[0096] According to embodiments of this disclosure, multiple trajectory prediction models can be pre-trained using a meta-learning strategy for multiple different task categories and stored in a model library. One task type corresponds to one trajectory prediction model, and the same trajectory prediction model can predict the trajectories of obstacles in multiple scenarios within the same task category.

[0097] In traditional deep learning methods, the most labor-intensive and time-consuming process in machine learning-based trajectory prediction algorithms is parameter tuning. Typically, parameter tuning needs to be done from scratch for each driving scenario, followed by a significant amount of time spent training and testing the results. Furthermore, it cannot be guaranteed that the algorithm model can adapt well to different driving scenarios.

[0098] In contrast, the core of meta-learning is "learning how to learn". Therefore, by training models using meta-learning strategies, the machine can learn to tune parameters on its own, and when encountering similar tasks, it can apply its knowledge to other situations and generalize from one instance to another. When entering different scenarios or dealing with different types of obstacles, there is no need to start tuning parameters from scratch, nor is there a need to retrain with a large amount of labeled data.

[0099] According to embodiments of this disclosure, after training the model using a meta-learning strategy based on sample data from various driving scenarios, a trained parameter-tuned model and multiple trajectory prediction models for different task categories can be obtained. The trajectory prediction models can output predicted trajectories of different types of obstacles in different road scenarios, informing the driver (autonomous vehicle) of the intentions of other obstacles, enabling the driver to make corresponding decisions, avoid collisions, and improve the driver's traffic efficiency.

[0100] like Figure 3 As shown, the hyperparameter tuning model Fφ obtained based on the meta-learning strategy allows the trajectory prediction model to automatically learn the hyperparameter fθ, instead of requiring manual tuning based on the scene and obstacle type. These hyperparameters include initialization parameters, learning rate, and network architecture. The main objective of this invention is to expand on the initialization parameters. Once the model learns the initialization parameters for the same task category, it can predict trajectories for all driving scenarios within that task category.

[0101] The trained hyperparameter-tuning model F(φ) can fine-tune the hyperparameter fθ of the trajectory prediction model. When switching task types, only a small amount of data is needed to automatically fine-tune the model parameters before predicting obstacle trajectories in the new task type scenario.

[0102] like Figure 3 As shown, for the scenario of pedestrians walking in an intersection, the hyperparameters of the trajectory prediction model generated by the parameter tuning model F(φ) are fθ1, which is used to predict the trajectory of pedestrians in the intersection. When the task type is switched to the scenario of pedestrians walking in a straight road, the model parameters are fine-tuned by the parameter tuning model F(φ), and the hyperparameters of the trajectory prediction model generated are fθ2, which is used to predict the trajectory of pedestrians in the straight road.

[0103] Based on this, upon receiving the first scene information associated with the first driving scene, the target task category corresponding to the first driving scene is first determined through operation S201. The first scene information includes the first motion characteristic information of the first main obstacle, such as the category of the first main obstacle, the location information of the first main obstacle, and the road category where the first main obstacle is located. Based on the category of the first main obstacle and the road category where the first main obstacle is located, the target task category corresponding to the first driving scene can be determined.

[0104] Based on the fact that multiple trajectory prediction models for different task categories are pre-trained and stored in the model library, after determining the target task category, operation S202 can be used to further determine whether the multiple trajectory prediction models pre-stored in the model library contain a target trajectory prediction model corresponding to the target task category.

[0105] The same trajectory prediction model can predict the trajectories of obstacles in multiple scenarios under the same task category. When the model library contains a target trajectory prediction model corresponding to the target task category, the scene information can be directly input into the target trajectory prediction model, and the predicted trajectory information of the main obstacle will be output.

[0106] The trained hyperparameter-tuning model F(φ) can fine-tune the hyperparameter fθ of the trajectory prediction model. Before switching task types to predict obstacle trajectories in the new task type scenario, only a small portion of data is needed for automatic fine-tuning of the model parameters. Therefore, when the model library does not contain a target trajectory prediction model corresponding to the target task category, operations S203 and S204 can be performed to fine-tune the hyperparameter fθ of the existing basic trajectory prediction model using the hyperparameter-tuning model F(φ). The new model after hyperparameter tuning can be applied to trajectory prediction in the target task category scenario.

[0107] According to an embodiment of the present disclosure, the method of the present embodiment can be applied to a scenario of online vehicle end real-time prediction. When facing an existing task type, a pre-trained trajectory prediction model in a model library can be directly applied to output a prediction result. When facing a new task type, because there is no applicable model in the model library at the moment, in order to ensure timeliness, a rough trajectory prediction result can be obtained by using other feasible methods as a bottom-line feedback to the host vehicle. For example, a general trajectory prediction model can be used to output a trajectory prediction result, and the host vehicle can make an action feedback based on the prediction result. Meanwhile, after the actual trajectory of the obstacle occurs, actual trajectory data of the obstacle is collected as sample data, so that a new trajectory prediction model can be obtained by using a tuning model F(φ) to fine-tune the hyperparameters fθ of the existing basic trajectory prediction model based on the sample data. Subsequently, when receiving a prediction task of the same task scenario again, the new model after tuning can be directly applied to online real-time prediction.

[0108] According to an embodiment of the present disclosure, in operation S203 and operation S204, the hyperparameters fθ of the existing basic trajectory prediction model are updated by using the tuning model F(φ). Specifically, the parameters of the tuning model can be updated based on the first scene information (which can be a predetermined number of groups of first scene information) to obtain a target tuning model, and then the parameters of the basic trajectory prediction model are adjusted by using the target tuning model to generate a target trajectory prediction model, for example, the adjusted hyperparameters of the target tuning model can be assigned to the basic trajectory prediction model. Here, the process of model parameter adjustment is different from traditional deep learning. The parameters of the tuning model are pre-trained, and when facing a new task scenario, only a small amount of data under the new scene is needed for parameter updating, without a long training process.

[0109] According to an embodiment of the present disclosure, to solve the problem of large scene limitation of the trajectory prediction model in the related art, the method of the present embodiment pre-trains a tuning model and stores a plurality of trajectory prediction models in a model library. The same trajectory prediction model can be used to predict the trajectory of the obstacle in a plurality of scenes under the same task category. The tuning model is used to tune the trajectory prediction model according to different scenes to adapt to trajectory prediction under different task scenarios. Therefore, after determining the task category corresponding to the current driving scene and determining that there is no prediction model suitable for the current driving scene in the current model library, the trajectory prediction model can be tuned according to the new scene information by using the tuning model, which can adapt to trajectory prediction under a new task scenario. In this way, the problem of large scene limitation of the trajectory prediction model in the related art can be solved, and the trajectory prediction can be quickly applied to different scenes, so that the trajectory prediction is not limited by the scene and the type of the obstacle, and has strong scene generalization ability.

[0110] Further, in the embodiments of the present disclosure, the process of adjusting the trajectory prediction model according to the new scene information by using the parameter adjustment model is different from the traditional deep learning. The parameters of the parameter adjustment model are pre-trained. When facing a new task scene, the parameter adjustment does not need to start from scratch, nor does it need to retrain with a large amount of labeled data. Only a small amount of data in the new scene is needed to automatically fine-tune the model parameters. Unlike the related art, the parameter adjustment needs to start from scratch for each driving scene, and then a large amount of time is spent on training and testing the effect. The model training period is greatly shortened, and the model adjustment efficiency is improved.

[0111] According to the embodiments of the present disclosure, after receiving the prediction task, the trajectory prediction model needs to calculate according to the scene information to output the predicted trajectory of the main obstacle. The scene information includes: in the corresponding driving scene, the motion characteristic information of the main obstacle, the traffic indication information of the lane where the main obstacle is located, and the relative motion information of the environmental obstacle (other obstacles around the main obstacle).

[0112] The motion characteristic information of the main obstacle can include but is not limited to: the category of the main obstacle (pedestrian, tricycle, large truck, trailer, etc.), the reference speed information (which can be absolute speed, for example) and the reference position information (which can be position coordinates, for example) of the main obstacle before the target time node (current time), the road category where the main obstacle is located (non-intersection urban road, intersection urban road, non-intersection urban road, urban road auxiliary road, etc.), and the like. The traffic indication information of the lane where the main obstacle is located includes the lane line pointing information (left turn, right turn, straight, etc.) of the lane where the main obstacle is located, and the traffic signal light state information (red, yellow, green) of the lane where the main obstacle is located. The relative motion information of the environmental obstacle includes the relative distance between the environmental obstacle and the main obstacle, and the relative speed of the environmental obstacle relative to the main obstacle.

[0113] The scene information corresponds to one driving scene, and one driving scene corresponds to a set of scene information. The first scene information is associated with the first driving scene, and the second scene information is associated with the second driving scene.

[0114] According to the embodiments of the present disclosure, after generating the target trajectory prediction model corresponding to the target task category, the generated target trajectory prediction model is newly stored in the model library. In this way, when the same scene task is received subsequently, the newly added trajectory prediction model can be directly used for online real-time trajectory prediction.

[0115] For example, after receiving the second scene information associated with the second driving scene, it is determined that the second driving scene corresponds to a target task category, and then a target trajectory prediction model corresponding to the target task category is called from the plurality of trajectory prediction models. The second scene information is input into the target trajectory prediction model, and the predicted trajectory information of the second main obstacle in the second driving scene is output by the target trajectory prediction model.

[0116] According to an embodiment of the present disclosure, although the first driving scene and the second driving scene can be different driving scenes, for example, the first driving scene is a scene in which a bus drives on a city road at a road intersection without a traffic light, and the second driving scene is a scene in which an electric vehicle drives on a city road at a road intersection without a traffic light. However, the first driving scene and the second driving scene are similar driving scenes, are associated with the same type of road, and belong to the same task category.

[0117] According to an embodiment of the present disclosure, the trajectory prediction model is trained based on a meta-learning algorithm. The model trained by the algorithm has strong generalization ability and can explore scenes, and can be applicable to multiple scenes. Therefore, the same trajectory prediction model can be used to predict and output the predicted trajectories of obstacles in the two driving scenes.

[0118] According to an embodiment of the present disclosure, the trajectory prediction model can output multiple predicted position information (such as coordinate values) of the obstacle after the target time node (current time) based on the scene information before the target time node. For example, according to the collected scene information, including at least one position coordinate of the main obstacle before the current time, at least one speed value before the current time, lane line pointing information of the lane where the main obstacle is located, traffic signal lamp state information of the lane where the main obstacle is located, at least one relative distance between the environmental obstacle and the main obstacle, and at least one relative speed of the environmental obstacle relative to the main obstacle, after inputting the above scene information into the trajectory prediction model, the predicted position information corresponding to multiple (the number can be freely set) trajectory points is output, and the multiple trajectory points can be continuously distributed according to a predetermined time interval (such as 0.5 seconds).

[0119] According to an embodiment of the present disclosure, by using the method, when facing an existing task type, a pre-trained trajectory prediction model in the model library can be directly used to output a prediction result, and when facing a new task type, a new model can be established by adjusting parameters to adapt to the new task scene. In this way, trajectory prediction in new and old task scenes can be simultaneously compatible, trajectory prediction is not limited by the scene and the type of obstacle, the timeliness of trajectory prediction is guaranteed, and the model library is continuously updated, the types of models are updated in real time, and better scene adaptability is achieved.

[0120] According to an embodiment of the present disclosure, specifically, updating the parameters of the tuning model based on the first scene information to obtain the target tuning model includes the following operations.

[0121] Operation 11, input the first scene information (which can be a predetermined number of groups of first scene information) into the basic trajectory prediction model, and output the reference trajectory information of the first main obstacle after the target time node, that is, output the initial trajectory prediction value through the model. Since the basic trajectory prediction model is not established for this scene task category, the prediction accuracy at this time is low.

[0122] Operation 12, obtain the sampling trajectory information of the first main obstacle (which can be a predetermined number of groups of sampling trajectory information), for example, obtain a plurality of groups of real trajectory information of the main obstacle after the target time node by sampling.

[0123] Operation 13, according to the reference trajectory information of the first main obstacle and the sampling trajectory information of the first main obstacle, the loss value corresponding to the target task category is calculated. The loss value can represent the error between the predicted trajectory (reference trajectory information) of the main obstacle and the real trajectory (sampling trajectory information).

[0124] Operation 14, based on the loss value corresponding to the target task category, the parameters of the tuning model are updated to obtain the target tuning model, and the parameters are adjusted in the direction of reducing the loss value.

[0125] Wherein, the method for calculating the loss value can be:

[0126] First, according to the reference trajectory information of the first main obstacle and the sampling trajectory information of the first main obstacle, the average displacement error ADE (Average displacement error) and the final displacement error FDE (Final displacement error) of the first main obstacle are calculated. The average displacement error ADE can be the average of the position errors between the plurality of predicted trajectory points and the plurality of real trajectory points of the main obstacle; the final displacement error FDE can be the position error between the last predicted trajectory point and the last real trajectory point of the main obstacle.

[0127] Then, according to the driving scene category corresponding to the first driving scene, the first weight corresponding to the average displacement error is determined, and the second weight corresponding to the final displacement error is determined. The values of the first weight and the second weight are different according to different driving scenes, for example, for the scene of pedestrians walking in a straight road, when designing the weight parameter, the second weight value corresponding to the final displacement error can be greater than the first weight corresponding to the average displacement error; for the scene of a truck driving in a straight road, when designing the weight parameter, the second weight value corresponding to the final displacement error can be less than the first weight corresponding to the average displacement error.

[0128] Then, according to the average displacement error, the end point displacement error, the first weight, the second weight, a loss value l corresponding to the target task category is calculated i The calculation method of the loss value l can be shown in the following formula (1).

[0129] l i = w1*ADE + w2*FDE (1)

[0130] In the formula (1), the meanings of the parameters are as follows:

[0131] ADE----average displacement error;

[0132] FDE----end point displacement error;

[0133] w1----first weight corresponding to the average displacement error;

[0134] w2----second weight corresponding to the end point displacement error.

[0135] According to the embodiments of the present disclosure, in the process of adjusting the parameters of the model, the error between the predicted trajectory and the real trajectory of the main obstacle is calculated as the loss value of the model adjustment, and the parameters are adjusted in the direction that reduces the loss value. In this way, the adjusted parameter adjustment model can adaptively adjust the trajectory prediction model in the new task scene, and can make the trajectory prediction model have relatively accurate prediction ability in the new task scene.

[0136] According to the embodiments of the present disclosure, by determining the first weight corresponding to the average displacement error and determining the second weight corresponding to the end point displacement error according to the driving scene category, the bias of the trajectory error in different scenes is fully considered, and further, the adaptability of the parameter adjustment model to the new task scene is enhanced. Furthermore, by adjusting the trajectory prediction model, the prediction accuracy of the trajectory prediction model is improved.

[0137] Another aspect of the present disclosure provides a model training method, Figure 4 A flowchart of the model training method according to the embodiments of the present disclosure is schematically shown.

[0138] As Figure 4 shown, the model training method of the embodiments includes operation S401 to operation S402.

[0139] In operation S401, a plurality of task sample sets corresponding to a plurality of task categories are obtained, wherein each task sample set includes a plurality of sets of historical sample information associated with a plurality of historical driving scenes, and each set of historical sample information includes main obstacle historical motion characteristic information, historical traffic indication information of a lane where the main obstacle is located, main obstacle historical trajectory information, and environmental obstacle historical relative motion information in a respective historical driving scene.

[0140] Before model training, scene data under multiple task types need to be collected in advance as training data, that is, a plurality of task sample sets corresponding to a plurality of task categories are collected, wherein one task sample set corresponds to one task category. Among them, a plurality of similar driving scenes belong to the same task category, and driving scenes associated with the same type of road can belong to the same task category. Based on this, the plurality of task categories may include, but are not limited to, for example: trajectory prediction of obstacles on urban roads without traffic lights at intersections, trajectory prediction of obstacles on urban roads with traffic lights at intersections, trajectory prediction of obstacles on non-intersection urban roads, trajectory prediction of obstacles on urban auxiliary roads, trajectory prediction of obstacles on township roads without traffic lights at intersections, trajectory prediction of obstacles on township roads with traffic lights at intersections, trajectory prediction of obstacles on non-intersection township roads, trajectory prediction of obstacles on highways, trajectory prediction of obstacles on rural roads, etc.

[0141] The same task sample set includes a plurality of sets of historical sample information under the task category, and the plurality of sets of historical sample information correspond to a plurality of similar driving scenes. For example, the plurality of sets of historical sample information in the first task sample set include: scene information of a bus driving on an urban road, scene information of an electric vehicle driving on an urban road, scene information of a pedestrian driving on an urban road, etc.; the plurality of sets of historical sample information in the second task sample set include: scene information of a motorcycle driving on a township road, scene information of an electric vehicle and a motorcycle driving on a township road, scene information of a pedestrian and a motorcycle driving on a township road, etc. Any set of scene information specifically includes the main obstacle historical motion characteristic information, the historical traffic indication information of the lane where the main obstacle is located, the main obstacle historical trajectory information, and the environmental obstacle historical relative motion information.

[0142] In operation S402, based on a meta-learning algorithm, the parameters of the to-be-trained trajectory prediction model and the to-be-trained parameter adjustment model are updated using the plurality of task sample sets, to obtain a trained parameter adjustment model and a plurality of trajectory prediction models, wherein the plurality of trajectory prediction models correspond one-to-one to the plurality of task categories, and a target trajectory prediction model corresponding to a target task category is used to predict the trajectory of a main obstacle in a plurality of driving scenes under the target task category.

[0143] The trajectory prediction model in the related art is mostly designed for a special scene or a specific obstacle, such as a trajectory algorithm model designed for an intersection scene or a trajectory prediction model designed for a pedestrian. Therefore, the sample data used in the training process is also data in a specific scene. Therefore, the model trained is difficult to adapt well after the scene is switched.

[0144] According to an embodiment of the present disclosure, because the plurality of task sample sets used for training the model correspond to a plurality of different task categories, the generalization ability of the model in the case of task category change can be learned by decomposing the data set into different tasks. Therefore, the model trained can adapt to the trajectory prediction of obstacles in a plurality of different task scenes. In addition, the model training is performed based on the meta-learning algorithm in the embodiment of the present disclosure. After the training is completed, not only a plurality of trajectory prediction models can be obtained, but most importantly, a hyperparameter tuning model can be trained. The hyperparameter tuning model obtained based on the meta-learning strategy can enable the trajectory prediction model to automatically learn the hyperparameters, instead of manually tuning the hyperparameters according to the scene and the type of obstacle. By fine-tuning the hyperparameters of the trajectory prediction model through the trained hyperparameter tuning model, a trajectory prediction model applicable to any new task scene can be obtained. When facing a completely new scene, only one or a few steps of tuning are needed to adapt to the prediction task of the new scene, greatly shortening the training period of the model and improving the timeliness of trajectory prediction.

[0145] According to an embodiment of the present disclosure, further, the plurality of groups of historical sample information in each task sample set are classified into a training sample group and a test sample group. For example, the trajectory prediction of a bus on a city road at an intersection without a traffic light is regarded as a training sample. If a new sample, the trajectory prediction of a bicycle on a city road at an intersection without a traffic light, is used to test the effect of the hyperparameters found through training, the task is called a test task. The test sample can have appeared in the training sample or can not have appeared in the training task. Using a sample that has not appeared in the training sample as a test sample can improve the generalization ability of the model.

[0146] Figure 5 The principle diagram of the model training method according to an embodiment of the present disclosure is schematically shown. The following describes the model training method in combination with Figure 5 The model training method is described.

[0147] According to an embodiment of the present disclosure, as shown in Figure 5 Each task category corresponds to its own loss value li. The training strategy based on meta-learning is not to find a good hyperparameter for a specific task category, but to find a good hyperparameter for all these task categories. In this way, when a similar task is encountered, the learned hyperparameter can be directly used without manual tuning.

[0148] The total loss function is defined as: wherein N is the number of task categories.

[0149] The method of the embodiments of the present disclosure is: first, a trajectory prediction model is trained using training samples in a task set, and then the effect of the hyperparameters found by training is tested using test samples in the task set, that is, the prediction effect of the trajectory prediction model obtained by training is tested using test samples in the task set, so as to train the parameters of the hyperparameter tuning model.

[0150] Specifically, as shown in Figure 5 for training task 1, based on the hyperparameters φ of the hyperparameter tuning function F(φ) (for example, directly assigning the hyperparameters φ to the trajectory prediction model f), the trajectory prediction model f is trained using the training samples to obtain the parameters θ1, and then the trained model fθ1 is tested using the test samples, and the loss value obtained by testing is l 1 ; similarly, for training task 2, based on the hyperparameters φ of the hyperparameter tuning function F(φ) (for example, directly assigning the hyperparameters φ to the trajectory prediction model f), the trajectory prediction model f is trained using the training samples to obtain the parameters θ2, and then the trained model fθ2 is tested using the test samples, and the loss value obtained by testing is l 2 ; the goal of the meta-learning-based trajectory prediction algorithm is to minimize the total loss L(φ).

[0151] Specifically, updating the parameters of the to-be-trained hyperparameter tuning model and the to-be-trained trajectory prediction model using multiple task sample sets includes the following operations.

[0152] Operation 21, based on the training sample group in the nth task sample set and the to-be-trained hyperparameter tuning model updated for the n-1th time, updating the parameters of the to-be-trained trajectory prediction model to obtain the nth trajectory prediction model corresponding to the nth task category.

[0153] For example, a training task is sampled, the hyperparameters φ0 of the hyperparameter tuning function F(φ) in training are directly assigned to the trajectory prediction model f to obtain the initial parameters θ0, the training sample group in the task sample set is used to optimize the parameters θ0 based on the learning rate of the task to obtain the updated parameters θ1, and the initial trajectory prediction model fθ1 is obtained.

[0154] Operation 22, based on the test sample group in the nth task sample set and the nth trajectory prediction model, determining the loss value corresponding to the nth task category.

[0155] For example, based on the optimized parameters θ1, the loss value 1 1 of the task is calculated using the test sample group.

[0156] Operation 23, based on the loss value corresponding to the nth task category, the parameters of the to-be-trained model are updated for the nth time.

[0157] Here, the model parameters can be updated based on the gradient descent method. For example, the gradient of the gradient loss function is calculated The gradient is multiplied by the learning rate of the model adjustment function network to update the hyperparameters φ0 of the model adjustment function F(φ) to obtain φ1.

[0158] Operation 24, iteratively perform the above operations 21-23, that is, based on the training sample set in the (n+1)th task sample set and the to-be-trained model after the nth update, the parameters of the to-be-trained trajectory prediction model are updated to obtain the (n+1)th trajectory prediction model, the loss value corresponding to the (n+1)th task category is determined based on the test sample group in the (n+1)th task sample set and the (n+1)th trajectory prediction model, and the parameters of the to-be-trained model are updated for the (n+1)th time based on the loss value corresponding to the (n+1)th task category. The sum of the loss values corresponding to the plurality of task categories is less than a predetermined threshold, and the trained model adjustment model and the plurality of trajectory prediction models are obtained.

[0159] For example, another training task is sampled, the hyperparameters φ1 of the model adjustment function F(φ) in the training are directly assigned to the trajectory prediction model f to obtain the initial parameters θ1, the training sample set in the task sample set is used to optimize the parameters θ1 based on the learning rate of the task to obtain the updated parameters θ2, and the initial trajectory prediction model fθ2 is obtained. Based on the optimized parameters θ2, the loss value l of the task is calculated using the test sample set 2 The gradient of the gradient loss function is calculated The gradient is multiplied by the learning rate of the model adjustment function network to update the hyperparameters φ1 of the model adjustment function F(φ) to obtain φ2… The sum of the loss values corresponding to the plurality of task categories L(φ) is less than a predetermined threshold, and the trained model adjustment model and the plurality of trajectory prediction models are obtained.

[0160] According to an embodiment of the present disclosure, the plurality of historical driving scenes are classified into historical training driving scenes and historical test driving scenes, the historical training driving scenes are associated with the training sample set, and the historical test driving scenes are associated with the test sample set.

[0161] After training the trajectory prediction model using the training samples in the task set, it is necessary to test the prediction effect of the trained trajectory prediction model using the test samples in the task set, so as to train the parameters of the model adjustment model. The model adjustment is based on the task loss value, and the method for calculating the task loss value is as follows.

[0162] Specifically, based on the test sample group in the nth task sample set and the nth trajectory prediction model, the loss value corresponding to the nth task category is determined.

[0163] Firstly, the main obstacle historical motion characteristic information of the test sample group in the nth task sample set, the historical traffic indication information of the lane where the main obstacle is located, and the historical relative motion information of the environmental obstacle are input into the nth trajectory prediction model, and the main obstacle predicted trajectory sample information in the nth group of historical test driving scenes is output. The nth group of historical test driving scenes are a plurality of historical training driving scenes associated with the test sample group in the nth task sample set. That is, the scene information in the test sample data in the task category scene is input into the trajectory prediction model after parameter adjustment (the model trained by using the training sample data in the task category scene), and the prediction result is output.

[0164] Then, the loss value corresponding to the nth task category is calculated according to the main obstacle predicted trajectory sample information in the nth group of historical test driving scenes and the main obstacle historical trajectory information in the nth group of historical test driving scenes. The loss value can represent the error between the predicted trajectory (main obstacle predicted trajectory sample information) and the real trajectory (main obstacle historical trajectory information).

[0165] Further, the specific method for calculating the loss value according to the predicted trajectory and the real trajectory is as follows:

[0166] Firstly, the sample average displacement error and the sample end point displacement error associated with the nth task category are calculated according to the main obstacle predicted trajectory sample information in the nth group of historical test driving scenes and the main obstacle historical trajectory information in the nth group of historical test driving scenes.

[0167] Then, according to the driving scene category corresponding to the nth group of historical test driving scenes, the sample first weight corresponding to the sample average displacement error is determined, and the sample second weight corresponding to the sample end point displacement error is determined.

[0168] Then, the loss value corresponding to the nth task category is calculated according to the sample average displacement error, the sample end point displacement error, the sample first weight, and the sample second weight associated with the nth task category.

[0169] For the specific operation of the loss value calculation, reference can be made to the description of calculating the loss value corresponding to the target task category according to the reference trajectory information of the main obstacle and the sampling trajectory information of the main obstacle in the embodiments of the trajectory prediction model generation method described above, which will not be repeated here.

[0170] Based on the trajectory prediction model generation method described above, the present disclosure also provides a trajectory prediction model generation device. The following will be combined with the description of the trajectory prediction model generation device to further illustrate the technical solutions of the present disclosure. Figure 8The device is described in detail.

[0171] Figure 6 A structural block diagram of a trajectory prediction model generation device according to an embodiment of the present disclosure is schematically shown.

[0172] As Figure 6 shown, the model generation device 600 for predicting trajectories of this embodiment includes a determination module 601, a first calling module 602, a first parameter adjustment module 603, and a second parameter adjustment module 604.

[0173] The determination module 601 is configured to determine a target task category corresponding to a first driving scene according to first scene information associated with the first driving scene, wherein the first scene information includes first motion characteristic information of a first main obstacle, first traffic indication information of a lane where the first main obstacle is located, and first relative motion information of a first environmental obstacle in the first driving scene.

[0174] The first calling module 602 is configured to call a parameter adjustment model from a model library and call at least one trajectory prediction model from a plurality of trajectory prediction models in the model library as a basic trajectory prediction model in a case where the plurality of trajectory prediction models do not include a target trajectory prediction model corresponding to the target task category.

[0175] The first parameter adjustment module 603 is configured to update parameters of the parameter adjustment model based on the first scene information to obtain a target parameter adjustment model.

[0176] The second parameter adjustment module 604 is configured to adjust parameters of the basic trajectory prediction model using the target parameter adjustment model to generate the target trajectory prediction model.

[0177] According to the device of the embodiment of the present disclosure, in view of the defect that the scene limitation of the trajectory prediction model in the related art is relatively large, the determination module 601 determines a task category corresponding to a current driving scene, and after it is determined that a prediction model suitable for the current driving scene does not exist in the current model library, the first calling module 602 uses a parameter adjustment model, and the first parameter adjustment module 603 and the second parameter adjustment module 604 use the parameter adjustment model to adjust parameters of a trajectory prediction model according to new scene information, which can adapt to trajectory prediction in a new task scene. Since the same trajectory prediction model can be used to predict trajectories of obstacles in multiple scenes under the same task category, the parameter adjustment model is used to adjust the trajectory prediction model according to different scenes to adapt to trajectory prediction in different task scenes. In this way, the defect that the scene limitation of the trajectory prediction model in the related art is relatively large can be solved, and trajectory prediction in different scenes can be quickly adapted, so that trajectory prediction is not limited by the scene and the type of obstacles, and has strong scene generalization ability.

[0178] According to an embodiment of the present disclosure, the device further comprises a storage module, a receiving module, a second calling module, and a prediction module.

[0179] The storage module is configured to store the generated target trajectory prediction model in a model library. The receiving module is configured to receive second scene information associated with a second driving scene, the second scene information comprising second motion characteristic information of a second main obstacle, second traffic indication information of a lane where the second main obstacle is located, and second relative motion information of a second environmental obstacle in the second driving scene, wherein the second driving scene corresponds to the target task category. The second calling module is configured to call a target trajectory prediction model corresponding to the target task category from the plurality of trajectory prediction models. The prediction module is configured to input the second scene information into the target trajectory prediction model, and output predicted trajectory information of the second main obstacle in the second driving scene through the target trajectory prediction model.

[0180] According to an embodiment of the present disclosure, the first parameter adjusting module 603 comprises a first input-output unit, an obtaining unit, a first calculation unit, and a parameter adjusting unit.

[0181] The first input-output unit is configured to input the first scene information into the basic trajectory prediction model, and output reference trajectory information of the first main obstacle. The obtaining unit is configured to obtain sampling trajectory information of the first main obstacle. The first calculation unit is configured to calculate a loss value corresponding to the target task category according to the reference trajectory information of the first main obstacle and the sampling trajectory information of the first main obstacle. The parameter adjusting unit is configured to update parameters of the parameter adjusting model based on the loss value corresponding to the target task category, and obtain a target parameter adjusting model.

[0182] According to an embodiment of the present disclosure, the first calculation unit comprises a first calculation sub-unit, a first determination sub-unit, and a second calculation sub-unit.

[0183] The first calculation sub-unit is configured to calculate an average displacement error and a terminal displacement error of the first main obstacle according to the reference trajectory information of the first main obstacle and the sampling trajectory information of the first main obstacle. The first determination sub-unit is configured to determine a first weight corresponding to the average displacement error and a second weight corresponding to the terminal displacement error according to a driving scene category corresponding to the first driving scene. The second calculation sub-unit is configured to calculate the loss value corresponding to the target task category according to the average displacement error, the terminal displacement error, the first weight, and the second weight.

[0184] According to an embodiment of the present disclosure, the first motion characteristic information of the first main obstacle includes: a category of the first main obstacle, and reference speed information and reference position information of the first main obstacle before a target time node. The first traffic indication information of the lane where the first main obstacle is located includes: lane line pointing information of the lane where the first main obstacle is located, and traffic signal light state information of the lane where the first main obstacle is located. The first relative motion information of the first environmental obstacle includes: a relative distance between the first environmental obstacle and the first main obstacle, and a relative speed of the first environmental obstacle relative to the first main obstacle.

[0185] Based on the above model training method, the present disclosure further provides a model training device. The following will be described in detail in combination with Figure 7 The device is described in detail.

[0186] Figure 7 The structure block diagram of the model training device according to the embodiment of the present disclosure is schematically shown.

[0187] As Figure 7 shown, the model training device 700 of the embodiment includes an acquisition module 701 and a training module 702.

[0188] The acquisition module 701 is configured to acquire a plurality of task sample sets corresponding to a plurality of task categories, wherein each task sample set includes a plurality of sets of historical sample information associated with a plurality of historical driving scenes, and each set of historical sample information includes main obstacle historical motion characteristic information, main obstacle lane historical traffic indication information, main obstacle historical trajectory information, and environmental obstacle historical relative motion information in the respective historical driving scene.

[0189] The training module 702 is configured to update parameters of a to-be-trained parameter adjustment model and a to-be-trained trajectory prediction model based on a meta-learning algorithm using the plurality of task sample sets, to obtain a trained parameter adjustment model and a plurality of trajectory prediction models, wherein the plurality of trajectory prediction models correspond one-to-one to the plurality of task categories, and a target trajectory prediction model corresponding to a target task category is used to predict a trajectory of a main obstacle in a plurality of driving scenes under the target task category.

[0190] According to an embodiment of the present disclosure, the plurality of task sample sets obtained by the obtaining module 701 correspond to a plurality of different task categories, and by decomposing the data set into different tasks, the generalization ability of the model under the condition of learning the change of the task category can be learned, and therefore the model trained by the training module 702 can adapt to the trajectory prediction of obstacles in a plurality of different task scenarios. In addition, the training module 702 trains the model based on the meta-learning algorithm, and after training, not only a plurality of trajectory prediction models can be obtained, but most importantly, a hyperparameter tuning model can be trained. The hyperparameter tuning model obtained based on the meta-learning strategy can enable the trajectory prediction model to automatically learn the hyperparameters, rather than manually tuning the hyperparameters according to the scene and the type of obstacle. By fine-tuning the hyperparameters of the trajectory prediction model through the trained hyperparameter tuning model, a trajectory prediction model suitable for any new task scenario can be obtained. When facing a completely new scene, only one or a few steps of parameter tuning are needed to adapt to the prediction task of the new scene, greatly shortening the training period of the model and improving the timeliness of trajectory prediction.

[0191] According to an embodiment of the present disclosure, the plurality of groups of historical sample information in each task sample set are classified into a training sample group and a test sample group, and the training module 702 includes a first updating submodule, a determining submodule, a second updating submodule, and an iteration submodule.

[0192] The first updating submodule is configured to update the parameters of the to-be-trained trajectory prediction model based on the training sample group in the nth task sample set and the to-be-trained hyperparameter tuning model updated for the n-1th time, to obtain an nth trajectory prediction model corresponding to an nth task category.

[0193] The iteration submodule is configured to iteratively perform the following operations: updating the parameters of the to-be-trained trajectory prediction model based on the training sample group in the (n+1)th task sample set and the to-be-trained hyperparameter tuning model updated for the nth time to obtain an (n+1)th trajectory prediction model, determining a loss value corresponding to an (n+1)th task category based on the test sample group in the (n+1)th task sample set and the (n+1)th trajectory prediction model, and updating the parameters of the to-be-trained hyperparameter tuning model for the (n+1)th time based on the loss value corresponding to the (n+1)th task category, until the sum of the loss values corresponding to the plurality of task categories is less than a preset threshold, to obtain the trained hyperparameter tuning model and the plurality of trajectory prediction models.

[0194] According to an embodiment of the present disclosure, the plurality of historical driving scenes are classified into historical training driving scenes and historical testing driving scenes, the historical training driving scenes are associated with the training sample set, and the historical testing driving scenes are associated with the testing sample set.

[0195] The determining sub-module includes a second input-output unit and a second calculation unit.

[0196] The second input-output unit inputs the main obstacle historical motion characteristic information of the testing sample set in the nth task sample set, the historical traffic indication information of the lane where the main obstacle is located, and the historical relative motion information of the environmental obstacle into the nth trajectory prediction model, and outputs the main obstacle predicted trajectory sample information in the nth group of historical testing driving scenes.

[0197] According to an embodiment of the present disclosure, the second calculation unit includes a third calculation sub-unit, a second determining sub-unit, and a fourth calculation sub-unit.

[0198] The third calculation sub-unit calculates the sample average displacement error and the sample end-point displacement error associated with the nth task category according to the main obstacle predicted trajectory sample information in the nth group of historical testing driving scenes and the historical trajectory information of the main obstacle in the nth group of historical testing driving scenes.

[0199] According to an embodiment of the present disclosure, any of the modules of the determining module 601, the first calling module 602, the first parameter adjusting module 603, and the second parameter adjusting module 604, or the obtaining module 701 and the training module 702 can be combined in one module for implementation, or any of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of the modules can be combined with at least part of the functions of other modules, and implemented in one module. According to an embodiment of the present disclosure, at least one of the determining module 601, the first calling module 602, the first parameter adjusting module 603, and the second parameter adjusting module 604, or the obtaining module 701 and the training module 702 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on board, a system on package, an application specific integrated circuit (ASIC), or any other reasonable manner of hardware or firmware that can be integrated or packaged with a circuit, or implemented in any one of software, hardware, and firmware or in a proper combination of any of the foregoing. Alternatively, at least one of the determining module 601, the first calling module 602, the first parameter adjusting module 603, and the second parameter adjusting module 604, or the obtaining module 701 and the training module 702 can be at least partially implemented as a computer program module that can perform corresponding functions when the computer program module is run.

[0200] Figure 8 A block diagram of an electronic device suitable for implementing the trajectory prediction model generation method or the model training method according to an embodiment of the present disclosure is schematically shown.

[0201] As Figure 8 shown, the electronic device 800 according to an embodiment of the present disclosure includes a processor 801 that can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 802 or loaded from a storage portion 808 into a random access memory (RAM) 803. The processor 801 can include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor, and / or a related chipset, and / or a special-purpose microprocessor (such as an application specific integrated circuit (ASIC)), and the like. The processor 801 can also include an on-board memory for cache use. The processor 801 can include a single processing unit or a plurality of processing units for performing different actions of the method processes according to embodiments of the present disclosure.

[0202] In the RAM 803, various programs and data required for the operation of the electronic device 800 are stored. The processor 801, the ROM 802, and the RAM 803 are connected to each other via the bus 804. The processor 801 performs various operations of the method flow according to the embodiments of the present disclosure by executing the programs in the ROM 802 and / or the RAM 803. It should be noted that the programs can also be stored in one or more memories other than the ROM 802 and the RAM 803. The processor 801 can also perform various operations of the method flow according to the embodiments of the present disclosure by executing the programs stored in the one or more memories.

[0203] According to embodiments of the present disclosure, the electronic device 800 can further include an input / output (I / O) interface 805, which is also connected to the bus 804. The electronic device 800 can further include one or more of the following components connected to the I / O interface 805: an input part 806 including a keyboard, a mouse, and the like; an output part 807 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), and the like, and a speaker, and the like; a storage part 808 including a hard disk, and the like; and a communication part 809 including a network interface card such as a LAN card, a modem, and the like. The communication part 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as necessary. A removable recording medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like is attached to the drive 810 as necessary, so that a computer program read therefrom is installed in the storage part 808 as necessary.

[0204] The present disclosure also provides a computer readable storage medium, which can be included in the device / apparatus / system described in the above embodiments; or can exist separately without being assembled into the device / apparatus / system. The above computer readable storage medium carries one or more programs, when the one or more programs are executed, the method according to the embodiments of the present disclosure is implemented.

[0205] According to an embodiment of the present disclosure, the computer readable storage medium can be a nonvolatile computer readable storage medium, for example, can include, but is not limited to, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination thereof. In the present disclosure, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present disclosure, the computer readable storage medium can include the ROM 802 and / or the RAM 803 described above and / or one or more memory other than the ROM 802 and the RAM 803.

[0206] Embodiments of the present disclosure also include a computer program product, which includes a computer program containing program codes for executing the methods shown in the flowcharts. When the computer program product is run in a computer system, the program codes are used to make the computer system implement the trajectory prediction model generation method or the model training method provided by the embodiments of the present disclosure.

[0207] The above-described functions defined in the system / device of the embodiments of the present disclosure are performed when the computer program is executed by the processor 801. According to an embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by computer program modules.

[0208] In one embodiment, the computer program can rely on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program can also be transmitted, distributed, and downloaded in the form of a signal on a network medium, and be downloaded and installed through the communication part 809, and / or be installed from the detachable medium 811. The program codes contained in the computer program can be transmitted by any appropriate network medium, including but not limited to wireless, wired, etc., or any appropriate combination thereof.

[0209] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or be installed from the detachable medium 811. When the computer program is executed by the processor 801, the above-described functions defined in the system of the embodiments of the present disclosure are performed. According to an embodiment of the present disclosure, the system, device, apparatus, module, unit, etc. described above can be implemented by computer program modules.

[0210] According to embodiments of the present disclosure, program code of the computer program for performing the methods provided by the embodiments of the present disclosure can be written in any combination of one or more programming languages, and can be implemented in a computer program product. Specifically, the computer program can be implemented in a high-level procedural and / or object-oriented programming language, and / or in assembly / machine language. The programming language includes, but is not limited to, Java, C++, python, “C” language, or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's device, and partly on a remote computing device, or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider (ISP).

[0211] The computer program product of the present disclosure can be a computer program product, which is a machine-readable medium (or computer readable medium) having stored therein a sequence of instructions executable by a machine such as a personal digital assistant (PDA), a laptop, a desktop computer, or a server. Alternatively, the computer program product can be a propagated signal per se generated by using the program code, and the program code can be stored on a machine-readable medium that can be read by a machine such as a personal digital assistant (PDA), a laptop, a desktop computer, or a server. The machine-readable medium can be a machine-readable storage medium that stores program codes, or a machine-readable transmission medium that carries program codes. For example, the program code can be stored on a removable storage medium, such as a floppy diskette, a hard disk, a CD-ROM, a DVD, a Blu-ray Disc, a memory stick, or a memory card, or a storage medium that is fixedly installed within a machine, such as a hard disk. The program code can also be transmitted by a machine-readable transmission medium, such as a modem, a network cable, a wireless signal, or a combination thereof, or transmitted by a computer network, such as the Internet.

[0212] Those skilled in the art will understand that features of various embodiments and / or claims of the present disclosure can be combined or / and integrated with one another, even though such a combination or integration is not expressly mentioned in the present disclosure. In particular, features of various embodiments and / or claims of the present disclosure can be combined and / or integrated with one another without departing from the spirit and teachings of the present disclosure. All such combinations and / or integrations are within the scope of the present disclosure.

[0213] The above describes embodiments of the present disclosure. However, these embodiments are merely for illustrative purposes, and are not intended to limit the scope of the present disclosure. Although each embodiment is described above separately, this does not mean that the measures in each embodiment cannot be used advantageously in combination. The scope of the present disclosure is defined by the appended claims and their equivalents. Those skilled in the art can make various substitutions and modifications without departing from the scope of the present disclosure, and these substitutions and modifications should all fall within the scope of the present disclosure.

Claims

1. A meta-learning based trajectory prediction model generation method, comprising: determining a target task category corresponding to a first driving scene according to first scene information associated with the first driving scene; in a case where it is determined that a plurality of trajectory prediction models pre-stored in a model library do not include a target trajectory prediction model corresponding to the target task category, calling a trained hyperparameter tuning model from the model library, and calling at least one trajectory prediction model from the plurality of trajectory prediction models as a base trajectory prediction model, wherein each trajectory prediction model outputs a predicted trajectory of a different type of obstacle under a corresponding task category; updating parameters of the trained hyperparameter tuning model based on the first scene information to obtain a target hyperparameter tuning model; adjusting hyperparameters of the base trajectory prediction model using the target hyperparameter tuning model to generate a target trajectory prediction model, wherein the hyperparameters include initialization parameters, learning rates, and network architectures, wherein updating the parameters of the trained hyperparameter tuning model based on the first scene information to obtain the target hyperparameter tuning model comprises: inputting the first scene information into the base trajectory prediction model to output reference trajectory information of a first main obstacle; obtaining sampling trajectory information of the first main obstacle; calculating a loss value corresponding to the target task category according to the reference trajectory information of the first main obstacle and the sampling trajectory information of the first main obstacle; updating the parameters of the trained hyperparameter tuning model based on the loss value corresponding to the target task category to obtain the target hyperparameter tuning model.

2. The method of claim 1, further comprising: storing the generated target trajectory prediction model in the model library; receiving second scene information associated with a second driving scene, the second scene information including second motion characteristic information of a second main obstacle, second traffic indication information of a lane in which the second main obstacle is located, and second relative motion information of a second environmental obstacle in a second driving scene corresponding to the target task category; calling the target trajectory prediction model corresponding to the target task category from the plurality of trajectory prediction models; inputting the second scene information into the target trajectory prediction model to output, by the target trajectory prediction model, predicted trajectory information of the second main obstacle in the second driving scene.

3. The method of claim 1, wherein, calculating a loss value corresponding to the target task category according to the reference trajectory information of the first main obstacle and the sampling trajectory information of the first main obstacle comprises: calculating an average displacement error and an end point displacement error of the first main obstacle according to the reference trajectory information of the first main obstacle and the sampling trajectory information of the first main obstacle; determining a first weight corresponding to the average displacement error and a second weight corresponding to the end point displacement error according to a driving scene category corresponding to the first driving scene; The loss value corresponding to the target task category is calculated according to the average displacement error, the terminal displacement error, the first weight, and the second weight.

4. The method according to any one of claims 1 to 3, Wherein: Wherein, The first scene information includes: in the first driving scene, the first motion characteristic information of the first main obstacle, the first traffic indication information of the lane where the first main obstacle is located, and the first relative motion information of the first environmental obstacle. The first motion characteristic information of the first main obstacle includes: the category of the first main obstacle, and the reference speed information and the reference position information of the first main obstacle before the target time node. The first traffic indication information of the lane where the first main obstacle is located includes: the lane line pointing information of the lane where the first main obstacle is located, and the traffic signal lamp state information of the lane where the first main obstacle is located. The first relative motion information of the first environmental obstacle includes: the relative distance between the first environmental obstacle and the first main obstacle, and the relative speed of the first environmental obstacle relative to the first main obstacle.

5. A model training method, comprising: obtaining a plurality of task sample sets corresponding to a plurality of task categories, wherein each of the task sample sets includes a plurality of sets of historical sample information associated with a plurality of historical driving scenes; updating parameters of a to-be-trained tuning model and a to-be-trained trajectory prediction model based on a meta-learning algorithm using the plurality of task sample sets to obtain a trained tuning model and a plurality of trajectory prediction models, wherein the plurality of trajectory prediction models correspond one-to-one to the plurality of task categories, and wherein each trajectory prediction model is used to predict trajectories of a main obstacle in a plurality of driving scenes in a corresponding task category corresponding to the trajectory prediction model, wherein the trained tuning model is configured to support the following operations in a case where the plurality of trajectory prediction models corresponding one-to-one to the plurality of task categories do not include a target trajectory prediction model corresponding to a target task category: updating parameters based on first scene information to obtain a target tuning model; adjusting hyperparameters of a base trajectory prediction model using the target tuning model to generate a target trajectory prediction model for the target task category, wherein the hyperparameters include initialization parameters, learning rates, and network architectures, wherein updating parameters based on first scene information to obtain a target tuning model includes: inputting the first scene information into the base trajectory prediction model to output reference trajectory information of the first main obstacle; obtaining sampling trajectory information of the first main obstacle; calculating a loss value corresponding to the target task category according to the reference trajectory information of the first main obstacle and the sampling trajectory information of the first main obstacle; updating parameters of the trained tuning model based on the loss value corresponding to the target task category to obtain a target tuning model.

6. The method of claim 5, wherein, The plurality of sets of historical sample information in each of the task sample sets are classified into training sample sets and test sample sets, and updating parameters of the to-be-trained tuning model and the to-be-trained trajectory prediction model using the plurality of task sample sets includes: updating the parameters of the to-be-trained trajectory prediction model based on the training sample set in the n th task sample set and the to-be-trained hyperparameter tuning model after the n-1 th update, to obtain an n th trajectory prediction model corresponding to the n th task category; determining a loss value corresponding to the n th task category based on the test sample set in the n th task sample set and the n th trajectory prediction model; updating the parameters of the to-be-trained hyperparameter tuning model based on the loss value corresponding to the n th task category; iteratively performing the operations of updating the parameters of the to-be-trained trajectory prediction model based on the training sample set in the n th task sample set and the to-be-trained hyperparameter tuning model after the n th update to obtain an n th trajectory prediction model, determining a loss value corresponding to the n th task category based on the test sample set in the n th task sample set and the n th trajectory prediction model, and updating the parameters of the to-be-trained hyperparameter tuning model based on the loss value corresponding to the n th task category, until the sum of the loss values corresponding to the plurality of task categories is less than a preset threshold, to obtain a trained hyperparameter tuning model and a plurality of trajectory prediction models.

7. The method of claim 6, wherein: each group of the historical sample information comprises historical motion characteristic information of a host obstacle, historical traffic indication information of a lane where the host obstacle is located, historical trajectory information of the host obstacle, and historical relative motion information of an environmental obstacle in a respective historical driving scene; the plurality of historical driving scenes are classified into historical training driving scenes and historical test driving scenes, the historical training driving scenes are associated with the training sample set, and the historical test driving scenes are associated with the test sample set; determining a loss value corresponding to the n th task category based on the test sample set in the n th task sample set and the n th trajectory prediction model comprises: inputting the historical motion characteristic information of the host obstacle, the historical traffic indication information of the lane where the host obstacle is located, and the historical relative motion information of the environmental obstacle in the test sample set in the n th task sample set into the n th trajectory prediction model, and outputting host obstacle predicted trajectory sample information in an n th group of historical test driving scenes, wherein the n th group of historical test driving scenes are a plurality of historical training driving scenes associated with the test sample set in the n th task sample set; calculating a loss value corresponding to the n th task category based on the host obstacle predicted trajectory sample information in the n th group of historical test driving scenes and the historical trajectory information of the host obstacle in the n th group of historical test driving scenes.

8. The method of claim 7, wherein, calculating a loss value corresponding to the n th task category based on the host obstacle predicted trajectory sample information in the n th group of historical test driving scenes and the historical trajectory information of the host obstacle in the n th group of historical test driving scenes comprises: According to the main obstacle prediction track sample information under the nth group of historical test driving scenes and the main obstacle historical track information under the nth group of historical test driving scenes, a sample average displacement error and a sample end-point displacement error associated with the nth task category are calculated; According to the driving scene category corresponding to the nth group of historical test driving scenes, a sample first weight corresponding to the sample average displacement error and a sample second weight corresponding to the sample end-point displacement error are determined; According to the sample average displacement error, the sample end-point displacement error, the sample first weight and the sample second weight associated with the nth task category, a loss value corresponding to the nth task category is calculated.

9. A trajectory prediction model generation device based on meta-learning, comprising: A determination module configured to determine a target task category corresponding to a first driving scene according to first scene information associated with the first driving scene; A first calling module configured to, in a case where a target trajectory prediction model corresponding to the target task category is not included in a plurality of trajectory prediction models pre-stored in a model library and corresponding to a plurality of task categories, call a trained hyperparameter tuning model from the model library and call at least one trajectory prediction model from the plurality of trajectory prediction models as a base trajectory prediction model, wherein each trajectory prediction model outputs a predicted trajectory of a different type of obstacle under a corresponding task category; A first hyperparameter tuning module configured to update parameters of the trained hyperparameter tuning model based on the first scene information to obtain a target hyperparameter tuning model; A second hyperparameter tuning module configured to adjust hyperparameters of the base trajectory prediction model using the target hyperparameter tuning model to generate a target trajectory prediction model, wherein the hyperparameters include initialization parameters, learning rates, and network architectures. The first hyperparameter tuning module is further configured to: - input the first scene information into the base trajectory prediction model to output reference track information of a first main obstacle; - obtain sampling track information of the first main obstacle; - calculate a loss value corresponding to the target task category according to the reference track information of the first main obstacle and the sampling track information of the first main obstacle; - update the parameters of the trained hyperparameter tuning model based on the loss value corresponding to the target task category to obtain a target hyperparameter tuning model.

10. A model training device, comprising: An acquisition module configured to acquire a plurality of task sample sets corresponding to a plurality of task categories, wherein each task sample set includes a plurality of groups of historical sample information associated with a plurality of historical driving scenes; A training module configured to update parameters of a trained hyperparameter tuning model and a trained trajectory prediction model based on a meta-learning algorithm using the plurality of task sample sets, wherein the plurality of trajectory prediction models correspond one-to-one to the plurality of task categories, and wherein each target trajectory prediction model is used to predict a track of a main obstacle in a plurality of driving scenes under a corresponding task category corresponding thereto. The trained parameter adjustment model is configured to support the following operations in a case where the plurality of trajectory prediction models corresponding to the plurality of task categories does not include a target trajectory prediction model corresponding to a target task category: updating the parameters based on the first scene information to obtain a target parameter adjustment model; adjusting hyperparameters of a basic trajectory prediction model using the target parameter adjustment model to generate a target trajectory prediction model for the target task category, wherein the hyperparameters include initialization parameters, learning rates, and network architectures, wherein updating the parameters based on the first scene information to obtain a target parameter adjustment model includes: inputting the first scene information into the basic trajectory prediction model to output reference trajectory information of a first main obstacle; obtaining sampling trajectory information of the first main obstacle; calculating a loss value corresponding to the target task category according to the reference trajectory information of the first main obstacle and the sampling trajectory information of the first main obstacle; updating the parameters of the trained parameter adjustment model based on the loss value corresponding to the target task category to obtain a target parameter adjustment model. 11.An electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method according to any one of claims 1-4 or 5-8. 12.A computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the method according to any one of claims 1-4 or 5-8. 13.A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-4 or 5-8.

Images