Trajectory prediction model training method, trajectory prediction method, and electronic device

Abstract

The application provides a trajectory prediction model training method, a trajectory prediction method and an electronic device. The training method can include: preprocessing obtained data to obtain training samples; the training samples include vehicle driving data samples; recording a segmented area passed by a driving data sample of a specified driving trajectory; the segmented area is obtained by segmenting each lane in a road driven by a vehicle according to the width of each lane; taking the driving data sample of the vehicle passing through the segmented area as the input of the trajectory prediction model to be trained, taking the label corresponding to the specified driving trajectory as the output of the trajectory prediction model to be trained, and training the trajectory prediction model. According to the embodiment of the application, on the one hand, the sample condition is enriched, and on the other hand, the lane changing condition is more refined, which can make up for the defects of the insufficient existing training scene.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, and in particular to a training method for a trajectory prediction model, a trajectory prediction method, and an electronic device. Background Technology

[0002] Assisted driving and autonomous driving methods mainly include environmental perception, driving decision-making, and vehicle control. Driving decision-making relies on data obtained from environmental perception to predict the trajectories of other obstacles, thereby assisting the vehicle in its response. Currently, mainstream obstacle trajectory prediction solutions in the industry include trajectory point prediction and lane-level prediction. However, these methods require large amounts of data and lack fine-grained processing of samples during model training, resulting in less than ideal accuracy. Summary of the Invention

[0003] This application provides a training method for a trajectory prediction model, a trajectory prediction method, an apparatus, an electronic device, and a storage medium.

[0004] In a first aspect, embodiments of this application provide a method for training a trajectory prediction model, which may include:

[0005] The acquired data is preprocessed to obtain training samples; the training samples include vehicle driving data samples.

[0006] Record the segmented regions traversed by driving data samples that exhibit a specified driving trajectory; the segmented regions are obtained by dividing each lane according to the width of each lane in the road the vehicle is traveling on.

[0007] The trajectory prediction model is trained by using driving data samples containing records of vehicles passing through segmented regions as input and labels corresponding to specified driving trajectories as output.

[0008] Secondly, embodiments of this application provide a trajectory prediction method, which may include:

[0009] The collected driving data containing the vehicle to be predicted is sent to a pre-trained trajectory prediction model, which is trained according to the training method of the trajectory prediction model involved in the first aspect.

[0010] Based on the output of the trajectory prediction model, the probability of lane change and the timing of lane change for the vehicle to be predicted are obtained.

[0011] Thirdly, embodiments of this application provide a training apparatus for a trajectory prediction model, which may include:

[0012] The preprocessing module is used to preprocess the acquired data to obtain training samples; the training samples include vehicle driving data samples.

[0013] The segmentation recording module is used to record the segmented regions traversed by driving data samples that exhibit a specified driving trajectory; the segmented regions are obtained by dividing each lane according to the width of each lane in the road the vehicle is traveling on.

[0014] The model training module is used to train the trajectory prediction model by taking driving data samples containing records of vehicles passing through segmented regions as input and labels corresponding to specified driving trajectories as output.

[0015] Fourthly, embodiments of this application provide a trajectory prediction apparatus, which may include:

[0016] The data input module uses its eyes to send the collected driving data containing the vehicle to be predicted to a pre-trained trajectory prediction model, which is trained according to the method in the first embodiment.

[0017] The prediction result acquisition module is used to obtain the lane change probability and lane change timing of the vehicle to be predicted based on the output of the trajectory prediction model.

[0018] Fifthly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor, when executing the computer program, implements the method described in any of the above-mentioned embodiments.

[0019] Sixthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in any of the preceding claims.

[0020] Compared with the prior art, this application has the following advantages:

[0021] According to the embodiments of this application, a scheme of refining lanes is adopted, which exponentially increases the number of samples corresponding to the same specified driving trajectory. On the one hand, this enriches the sample types, and on the other hand, further refining the classification of lane change situations can compensate for the deficiencies of existing training scenarios. Using the above training samples to complete the training of the trajectory prediction model, according to actual test results, the accuracy of predicting lane change intentions has been improved, especially in cases with few lane changes, which can also be accurately predicted, thus improving the accuracy of trajectory prediction.

[0022] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application, it can be implemented according to the contents of the specification. In order to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0023] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments according to this application and should not be construed as limiting the scope of this application.

[0024] Figure 1 A flowchart illustrating the training method for the trajectory prediction model provided in this application;

[0025] Figure 2 This is a schematic diagram illustrating a training method for a trajectory prediction model according to an embodiment of this application.

[0026] Figure 3 This is a flowchart of a trajectory prediction method according to another embodiment of this application;

[0027] Figure 4 This is a structural block diagram of a training device for a trajectory prediction model according to an embodiment of this application;

[0028] Figure 5 This is a structural block diagram of a trajectory prediction apparatus according to another embodiment of this application; and

[0029] Figure 6 This is a block diagram of an electronic device used to implement embodiments of this application. Detailed Implementation

[0030] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the concept or scope of this application. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0031] To facilitate understanding of the technical solutions of the embodiments of this application, the relevant technologies of the embodiments of this application are described below. The following relevant technologies are optional solutions and can be combined with the technical solutions of the embodiments of this application in any way, and all of them fall within the protection scope of the embodiments of this application.

[0032] This application provides a method for training a trajectory prediction model, such as... Figure 1 The diagram shown is a flowchart of a training method for a trajectory prediction model according to an embodiment of this application. Corresponding to the first embodiment, the method may include:

[0033] Step S101: Preprocess the acquired data to obtain training samples; the training samples include vehicle driving data samples.

[0034] Step S102: Record the segmented region traversed by the driving data sample that exhibits the specified driving trajectory; the segmented region is obtained by dividing each lane according to the width of each lane in the road where the vehicle is traveling.

[0035] Step S103: Use driving data samples containing records of vehicles passing through segmented regions as input to the trajectory prediction model to be trained, and use the labels corresponding to the specified driving trajectory as output to the trajectory prediction model to be trained, and train the trajectory prediction model.

[0036] The executing entity of this application can be a server or cloud device or other device with strong computing power. The acquired data can be collected from multiple scenarios; for example, the acquired data can be video data collected and uploaded by users, or video data actively collected by a data collection vehicle.

[0037] To improve the training effect of the model, the collected data can be preprocessed. Since the acquired data may contain duplicates, anomalies, and missing values, preprocessing can alleviate these problems to some extent. For example, preprocessing may include deleting duplicate and anomaly data, and filling in missing values. The preprocessed data can then be used as training samples for the trajectory prediction model. These training samples include driving data samples of the vehicle to be predicted. The driving data of the vehicle to be predicted may include its driving position and speed. Since driving position and speed data have higher confidence than acceleration, rate of change of velocity, and heading angle data, using driving position and speed data (discarding acceleration, rate of change of velocity, and heading angle data) as samples can, to some extent, ensure the training effect of the trajectory prediction model. In addition, the driving data also includes other high-confidence data derived from the driving position and speed of the vehicle to be predicted; for example, the driving position can be used to determine the distance between the vehicle and the lane lines on the left and right. By using the vehicle's position and speed, the rate of change of the distance between the vehicle and the lane lines can be determined.

[0038] Based on the location data in the driving data sample of the vehicle to be predicted, combined with information such as high-precision maps or GPS, the road traveled by the vehicle to be predicted can be obtained. Figure 2 As shown in the diagram, the driving direction is from... Figure 2 Drive from the left to the right. Figure 2The system comprises three lanes from top to bottom, designated as lane 1 to lane 3. Solid lines indicate the boundaries between lanes, while dashed lines composed of line segments in the middle of each lane represent the lane's center line. In this embodiment, the lanes can be divided into different zones based on their width. Figure 2 The dashed lines composed of dots are used to represent the boundaries of the segmented areas in the lane. In the current embodiment, each lane is divided into three equal parts according to its width: lane 1 includes segmented areas 1 to 3; lane 2 includes segmented areas 4 to 6; and lane 3 includes segmented areas 7 to 9.

[0039] The specified driving trajectory can correspond to the vehicle's left lane change trajectory, right lane change trajectory, U-turn trajectory, etc. Figure 2 This scenario involves two vehicles. The first vehicle changes lanes from segment 1 of lane 1 to the right, moving to segment 4 of lane 2. The second vehicle changes lanes from segment 6 of lane 2 to the right, moving to segment 8 of lane 3. In other words, considering only right-side lane changes, there are nine possible scenarios for changing lanes from lane 1 to lane 2: segment 1 to segment 4, segment 2 to segment 4, segment 3 to segment 4, segment 1 to segment 5, segment 2 to segment 5, segment 3 to segment 5, segment 1 to segment 6, segment 2 to segment 6, and segment 3 to segment 6. Similarly, there are also nine possible scenarios for left-side lane changes.

[0040] Therefore, the first training sample includes the driving data of the first vehicle, and the second training sample includes the driving data of the second vehicle. The driving data can be used as input to the trajectory prediction model to be trained, and the right lane change serves as the label, which is the output of the trajectory prediction model to be trained, thus training the trajectory model. The training result needs to satisfy the expected performance of the specified loss function. The specified loss function can be the focal-loss function. The advantage of the focal-loss function is that it can address the model training problem caused by sample imbalance from the perspective of sample difficulty classification. The focal-loss function focuses on difficult-to-classify samples, which can address the problem of low classification accuracy for categories with few samples. Of course, difficult-to-classify samples are not limited to categories with few samples; that is, focal loss not only solves the problem of sample imbalance but also helps improve the overall performance of the model. During model training, the difficulty and ease of classification of samples are dynamically changing. That is, whether a sample is difficult to classify changes with the training process. Samples that were originally easy to classify may become difficult-to-train samples as the training process progresses. In the loss gradient, difficult-to-train samples play a dominant role; that is, the parameter changes mainly shift towards optimizing difficult-to-train samples. When parameters change, previously easy-to-train samples may become difficult-to-train samples. When this happens, it can slow down the model's convergence speed, meaning the model training may be nearing completion.

[0041] In this embodiment, a lane refinement scheme is adopted, which exponentially increases the number of samples corresponding to the same specified driving trajectory. This enriches the sample types and, by further refining the lane change scenarios, compensates for the shortcomings of existing training scenarios. For example, if only the scenario of changing lanes from lane 1 to lane 2 is used (without considering the refined lane regions), the model training results are easily concentrated on the scenario of changing lanes from segmented region 2 of lane 1 to segmented region 5 of lane 2, ignoring other scenarios. Therefore, by using the aforementioned training samples to train the trajectory prediction model, according to actual test results, the accuracy of predicting lane change intentions is significantly improved, especially in cases with few lane changes, thus enhancing the overall prediction accuracy.

[0042] In one implementation, the preprocessing of the collected data involved in step S101 may include:

[0043] Step S1011: Detect the data collected at each sampling point to identify duplicate data and data lacking specified content;

[0044] Step S1012: Remove duplicate data by deleting data that is missing specified content to obtain the processed data;

[0045] Step S1013: Select data collected from multiple consecutive sampling points from the processed data as training samples.

[0046] The acquired data can be collected from multiple consecutive sampling points. Due to network transmission issues, sampling accuracy limitations of the sampling equipment, etc., even though the data is collected from multiple consecutive sampling points, there may be issues such as missed sampling points, duplicate sampling, or incomplete data. The purpose of preprocessing is to address these issues and improve data quality.

[0047] For example, duplicate sampling can be deduplicated. Data lacking specified information can be deleted. This specified information can be such as driving location or speed. In other words, if the acquired data does not contain or cannot determine the driving location or speed, the sampled data can be deleted.

[0048] From the processed data, data collected from multiple consecutive sampling points are selected as training samples. In other words, the training samples need to be continuous and uninterrupted. If 100 frames of data are collected, corresponding to 100 sampling points, and the third frame is missing, but the data is continuous from the fourth frame onwards, then the fourth frame is used as the first sampling point. Alternatively, if the total amount of data is small, and the third frame is missing, it can be padded with data from multiple adjacent frames to obtain data from multiple consecutive sampling points.

[0049] Furthermore, loopback sampling can be used to increase the amount of data. For example, experimental testing has shown that the optimal amount of data for trajectory prediction is 13 consecutive frames. More than 13 frames would increase computational power and waste unnecessary resources, while fewer than 13 frames would result in a significant decrease in prediction accuracy compared to 13 frames. Therefore, 13 frames can be used as a data unit group. Loopback sampling refers to using 100 frames of data, where frames 1 to 13 form the first data unit group, frames 2 to 14 form the second data unit group, and so on, creating multiple data unit groups. This increases the amount of data by repeating some of the data.

[0050] Furthermore, in the later stages of model training, preprocessing of the collected data can include adding noise to the collected data to improve the model training effect.

[0051] In one implementation, the segmented region traversed by the driving data sample of the vehicle to be predicted that exhibits the specified driving trajectory, involved in step S102, may include:

[0052] Step S1021: Determine the width of each lane in the road the vehicle is traveling on;

[0053] Step S1022: Divide each lane into multiple areas according to its width;

[0054] Step S1023: When a vehicle follows a specified driving trajectory, record the starting and ending areas traversed by the vehicle following the specified driving trajectory.

[0055] Still combined Figure 2 As shown, by using high-precision maps, GPS, and other technologies, the location of a vehicle can be determined, i.e., the road it is traveling on. Furthermore, with the help of high-precision maps or vehicle-mounted image acquisition equipment, information such as the number of lanes and the width of the road can be determined.

[0056] Based on the width of the lanes, each lane is divided into multiple zones. For example... Figure 2 In the example shown, the three lanes are divided equally according to their width. In real-world scenarios, unequal division can also be used. For instance, based on experience, frequently used and infrequently used areas can be determined, and these can be used as width coefficients for lane division. Figure 2 Areas near the boundaries, such as segment 1 in lane 1 and segment 9 in lane 3, can be considered as less frequently used areas, and their width coefficients can be 0.8 or 0.9, etc. Segment 2 to segment 8, on the other hand, can be considered as frequently used areas, and their width coefficients can be 1 or 1.1, etc.

[0057] If a vehicle follows a designated driving trajectory, such as changing lanes to the left, changing lanes to the right, or making a U-turn, the system can record the various areas the vehicle passes through during that trajectory, thus recording the starting and ending areas. (Continuing with...) Figure 2 For example, both vehicles exhibited a right-side lane change trajectory. The first vehicle, during its right-side lane change, passed through dividing areas 1 to 4. Therefore, the starting area is dividing area 1, and the ending area is dividing area 4. The second vehicle, during its right-side lane change, passed through dividing areas 6 to 8. Therefore, the starting area is dividing area 6, and the ending area is dividing area 8.

[0058] Figure 2 The example shown represents an ideal scenario where the vehicle is exactly within the segmented area. In real-world scenarios, a vehicle might span two segmented areas. Therefore, a vehicle can be assigned to a corresponding segmented area based on the proportion of its location within those areas.

[0059] Through the above process, the road is refined into multiple segmented regions according to its width, thereby enabling the collection of scenarios where lane changes occur between different segmented regions.

[0060] In one implementation, when a specified driving trajectory includes multiple starting areas or multiple ending areas, it may further include:

[0061] Step S1024: Count the region pairs consisting of the starting region and the ending region corresponding to each specified driving trajectory;

[0062] Step S1025: Perform clustering processing on multiple matching pairs to obtain multiple clustering results; each clustering result has the same starting region and the same ending region;

[0063] Step S1026: Determine the differences in the number of matching pairs in each clustering result;

[0064] Step S1027: If the difference exceeds the corresponding threshold, downsample the clustering results with a large number of matching pairs so that the difference in the number of matching pairs in each clustering result is not greater than the corresponding threshold.

[0065] Still combined Figure 2 As shown, as previously mentioned, Figure 2 In the example shown, the specified driving trajectory is a lane change to the right. Therefore, when the first vehicle changes lanes to the right, the starting area is segmented region 1, and the ending area is segmented region 4. When the second vehicle changes lanes to the right, the starting area is segmented region 6, and the ending area is segmented region 8. In other words, a lane change to the right may contain multiple starting areas (segmented regions 1 and 6) or multiple ending areas (segmented regions 4 and 8).

[0066] With a sufficient amount of data, the statistically analyzed driving trajectories between adjacent lanes (taking lane 1 and lane 2 as an example) will cover many possibilities. Therefore, clustering is performed by grouping lanes with the same starting and ending regions together. For example, taking right-side lane changes between adjacent lanes as an example, with a sufficient amount of data, the clustering results might cover nine possibilities: lane change from segmentation region 1 to segmentation region 4, from segmentation region 2 to segmentation region 4, from segmentation region 3 to segmentation region 4, from segmentation region 1 to segmentation region 5, from segmentation region 2 to segmentation region 5, from segmentation region 3 to segmentation region 5, from segmentation region 1 to segmentation region 6, from segmentation region 2 to segmentation region 6, and from segmentation region 3 to segmentation region 6. In other words, for right-side lane changes, there are nine possible clustering results for lanes with the same starting and ending regions.

[0067] Furthermore, the number of matching pairs in each cluster can be counted, thereby ranking the number of matching pairs in each cluster. The purpose of ranking is to understand the differences in the number of matching pairs in each cluster. The difference can be determined based on a reference value for the number of matching pairs in each cluster. For example, the reference value can be the mean, median, weighted average, etc. If the difference between the number of matching pairs in the i-th cluster and the reference value exceeds the corresponding threshold, then the number of matching pairs in the i-th cluster is downsampled so that the difference between the number of matching pairs in the downsampled i-th cluster and the reference value is not greater than the corresponding threshold. Generally speaking, the reference value can be set based on clusters with fewer matching pairs, so that downsampling only needs to be performed on clusters with more matching pairs.

[0068] After downsampling, the number of matching pairs in each cluster is more balanced, which is more conducive to model training, making the training results more generalizable and avoiding overfitting.

[0069] In one implementation, it further includes:

[0070] A time window is defined for a specified driving trajectory, and the start time of the time window is dynamically adjusted based on the start time of the specified driving trajectory.

[0071] Once a specified driving trajectory is detected, the start and end times of that trajectory can be further determined. For example, if a right-side lane change is detected, the start and end times can be determined based on changes in the vehicle's steering angle or wheel angle. Then, a time window is defined for the specified driving trajectory. The time window must at least include the start and end times of the specified driving trajectory.

[0072] Furthermore, the time window can be adjusted based on downstream forecast demand. For example, if the forecast demand is aggressively anticipated, the start time of the time window should be earlier than the start time of the specified travel trajectory, while the end time of the time window should not be earlier than the end time of the specified travel trajectory. Conversely, if the forecast demand is conservatively delayed, the start time of the time window can be either the start time of the specified travel trajectory or slightly later, while the end time of the time window should again not be earlier than the end time of the specified travel trajectory. "Slightly later" can be determined empirically as a time range later than the start time, ensuring that the start time of the time window falls within this range.

[0073] Therefore, by defining a time window, trajectory prediction models that meet different prediction needs can be trained.

[0074] In one implementation, the trajectory prediction model includes:

[0075] The vehicle feature extraction layer is used to extract features related to the vehicle to be predicted from the training samples: the features related to the vehicle to be predicted include location features and driving speed features;

[0076] The obstacle vehicle feature extraction layer is used to extract obstacle vehicle-related features from the training samples: the obstacle vehicle is the obstacle vehicle of the vehicle to be predicted, and the obstacle vehicle-related features include the relative position of the obstacle vehicle and the vehicle to be predicted, and the rate of change of the relative position of the obstacle vehicle and the vehicle to be predicted.

[0077] The feature stitching layer is used to stitch together the features related to the vehicle to be predicted and the features related to the obstacle vehicle, and to perform feature re-extraction processing.

[0078] The environmental feature extraction layer is used to determine the feature points of the road that the vehicle to be predicted will travel on in the training samples. The feature points are used to characterize the curvature changes of the road.

[0079] Fully connected layer; used to determine the lane change probability and timing of the vehicle to be predicted based on the output of the feature splicing layer and the environmental feature extraction layer.

[0080] Trajectory models are used to predict whether vehicles on a road will follow a specified driving trajectory, and when such a trajectory will occur. The object of the prediction is the vehicle to be predicted, while other vehicles within a certain distance of the vehicle to be predicted can be considered as obstacles.

[0081] The vehicle feature extraction layer is used to extract features related to the vehicle to be predicted from the training samples, such as features representing the vehicle's driving position and speed. The driving position can include the segmented region to which the vehicle belongs; that is, the vehicle's segmented region and the segmented region it passes through during lane changes can be determined by the distance between the vehicle and the lane lines and the rate of change of that distance.

[0082] The obstacle vehicle feature extraction layer is used to extract features related to the vehicle to be predicted from the training samples, such as features representing the obstacle vehicle's driving position, driving speed, and relative position to the vehicle to be predicted.

[0083] The feature concatenation layer is used to concatenate the features extracted by the vehicle feature extraction layer and the obstacle vehicle feature extraction layer to obtain a feature set. The features in this feature set are then further processed. This further extraction may include feature filtering, feature dimensionality reduction, etc., to create a set of vehicle-related features.

[0084] The environmental feature extraction layer is used to determine feature points of the road traveled by the vehicle to be predicted from the training samples. These road feature points characterize the curvature changes of the road. For example, after obtaining the location of the vehicle to be predicted, the road traveled can be determined by combining information such as high-precision maps or GPS. Based on the road conditions, lane points can be obtained in the direction the vehicle is to be predicted; these lane points reflect the curvature of the road in front of the vehicle. The features determined by the environmental feature extraction layer are considered environment-related features.

[0085] The fully connected layer is used to predict the probability and timing of a vehicle's lane change based on the features output from the feature concatenation layer and the environmental feature extraction layer, and outputs the prediction result. For example, the prediction result could be that the vehicle will change lanes to the right in 2 seconds.

[0086] The entire model can be trained using the focal-loss function from object recognition. The hyperparameters of the loss function are chosen as alpha = 0.23, gamma = 2, the optimizer is adamw, and the learning rate is 0.004. The focal-loss function determines the performance of the trajectory prediction model by comparing its predicted output with the expected output (left lane change label, right lane change label, U-turn label, etc.), thereby identifying optimization directions. If the deviation between the two is very large, the loss value will be large; if the deviation is small or almost the same, the loss value will be very low.

[0087] Figure 3 This is a flowchart of a trajectory prediction method according to another embodiment of this application, which may include:

[0088] Step S301: The collected driving data containing the vehicle to be predicted is sent to the pre-trained trajectory prediction model, which is trained according to the method of the first embodiment.

[0089] Step S302: Based on the output of the trajectory prediction model, obtain the lane change probability and lane change timing of the vehicle to be predicted.

[0090] The executing entity of this application can be an in-vehicle infotainment system or a cloud-based system communicating with the in-vehicle infotainment system. Taking the in-vehicle infotainment system as an example, it acquires multi-dimensional data collected by the main vehicle. The multi-dimensional data includes driving data of other vehicles on the road where the main vehicle is traveling. The so-called other vehicles can correspond to the vehicle to be predicted. This application is used to predict the probability and timing of lane changes by the vehicle to be predicted, thereby providing data support for the main vehicle's autonomous driving or assisted driving.

[0091] The multi-dimensional data collected by the main vehicle is input into a pre-trained trajectory prediction model. The vehicle feature extraction layer of the trajectory prediction model is used to extract features related to the vehicle from the multi-dimensional data, such as driving position and speed. The driving position can include the segmented region to which the vehicle belongs; that is, the segmented region to which the vehicle belongs and the segmented region traversed during lane changes can be determined by the distance between the vehicle and the lane lines and the rate of change of that distance.

[0092] The obstacle vehicle feature extraction layer of the trajectory prediction model extracts features related to obstacle vehicles. An obstacle vehicle is defined relative to the vehicle being predicted; that is, it is a vehicle within a certain distance threshold range around the vehicle being predicted. Obstacle vehicle features can include characteristics representing the obstacle vehicle's position, speed, and relative position to the vehicle being predicted.

[0093] The feature concatenation layer of the trajectory prediction model concatenates the features obtained from the feature extraction layers for the vehicle to be predicted and the obstacle vehicle feature extraction layers to obtain a feature set. The features in this feature set are then further extracted. This further extraction may include feature filtering, feature dimensionality reduction, and other processing to create a set of vehicle-related features.

[0094] The environmental feature extraction layer of the trajectory prediction model extracts feature points of the road the vehicle is traveling forward. These feature points characterize the curvature changes of the road. For example, after obtaining the vehicle's position, combined with information such as high-precision maps or GPS, the road the vehicle is traveling on can be determined. Furthermore, based on the road conditions, feature points characterizing the forward path of the vehicle to be predicted can be obtained.

[0095] The fully connected layer of the trajectory prediction model predicts the lane change probability and timing of the vehicle based on the features output by the feature concatenation layer and the environmental feature extraction layer. In this application, the trajectory prediction model is trained according to the training method in the first embodiment.

[0096] By utilizing trajectory prediction models, high-confidence features such as driving speed and position can be extracted, while low-confidence features commonly used in related technologies, such as acceleration, rate of change of velocity, and heading angle, are discarded, thus effectively improving prediction accuracy. Furthermore, since the trajectory prediction model divides the lane into multiple segments according to width during training, it covers lane-changing scenarios between various segments. Therefore, in real-world prediction scenarios, it can achieve excellent prediction results even in complex situations.

[0097] Corresponding to the application scenarios and methods provided in the embodiments of this application, the embodiments of this application also provide a training device for a trajectory prediction model. For example... Figure 4The diagram shown is a structural block diagram of a training device for a trajectory prediction model according to an embodiment of this application. The training device for the trajectory prediction model may include:

[0098] The preprocessing module 401 is used to preprocess the acquired data to obtain training samples; the training samples include vehicle driving data samples.

[0099] The segmentation recording module 402 is used to record the segmented regions traversed by driving data samples that exhibit a specified driving trajectory; the segmented regions are obtained by dividing each lane according to the width of each lane in the road where the vehicle is traveling;

[0100] The model training module 403 is used to train the trajectory prediction model by taking driving data samples containing records of vehicles passing through segmented regions as input to the trajectory prediction model to be trained, and taking the labels corresponding to the specified driving trajectory as output to the trajectory prediction model to be trained.

[0101] In one embodiment, the preprocessing module 401 may include:

[0102] The error data detection submodule is used to detect the data collected at each sampling point and identify duplicate data and data missing specified content;

[0103] The first preprocessing submodule is used to remove duplicate data and delete data that is missing specified content to obtain the processed data.

[0104] The second preprocessing submodule is used to select data collected from multiple consecutive sampling points from the processed data as training samples.

[0105] In one embodiment, the segmentation recording module 402 may include:

[0106] The width determination submodule is used to determine the width of each lane in the road on which the vehicle travels;

[0107] The region segmentation submodule is used to segment each lane into regions based on its width, dividing each lane into multiple regions;

[0108] The area recording submodule is used to record the starting and ending areas traversed by a vehicle that follows a specified driving trajectory.

[0109] In one implementation, when a specified driving trajectory includes multiple starting areas or multiple ending areas, a data balancing submodule is further included; the data balancing submodule may specifically include:

[0110] The region pair statistics unit is used to count the region pairs consisting of the starting region and the ending region corresponding to each specified driving trajectory.

[0111] Clustering units are used to cluster multiple matching pairs to obtain multiple clustering results; each clustering result has the same starting region and the same ending region.

[0112] The quantity difference determination unit is used to determine the difference in the number of matching pairs in each clustering result;

[0113] The data balancing execution unit is used to downsample clustering results with a large number of matching pairs when the difference exceeds the corresponding threshold, so that the difference in the number of matching pairs in each clustering result is not greater than the corresponding threshold.

[0114] In one implementation, a time window creation module is also included, specifically for:

[0115] A time window is defined for a specified driving trajectory, and the start time of the time window is dynamically adjusted based on the start time of the specified driving trajectory.

[0116] In one implementation, a module building module is also included, wherein the trajectory prediction model built by the model building module includes:

[0117] The vehicle feature extraction layer is used to extract features related to the vehicle to be predicted from the training samples: the features related to the vehicle to be predicted include location features and driving speed features;

[0118] The obstacle vehicle feature extraction layer is used to extract obstacle vehicle-related features from the training samples: the obstacle vehicle is the obstacle vehicle of the vehicle to be predicted, and the obstacle vehicle-related features include the relative position of the obstacle vehicle and the vehicle to be predicted, and the rate of change of the relative position of the obstacle vehicle and the vehicle to be predicted.

[0119] The feature stitching layer is used to stitch together the features related to the vehicle to be predicted and the features related to the obstacle vehicle, and to perform feature re-extraction processing.

[0120] The environmental feature extraction layer is used to determine the feature points of the road that the vehicle to be predicted will travel on in the training samples. The feature points are used to characterize the curvature changes of the road.

[0121] Fully connected layer; used to determine the lane change probability and timing of the vehicle to be predicted based on the output of the feature splicing layer and the environmental feature extraction layer.

[0122] Corresponding to the application scenarios and methods provided in the embodiments of this application, the embodiments of this application also provide a trajectory prediction apparatus. For example... Figure 5 The diagram shown is a structural block diagram of a trajectory prediction device according to an embodiment of this application, which may include:

[0123] The data input module 501 uses its eyes to send the collected driving data containing the vehicle to be predicted to a pre-trained trajectory prediction model, which is trained according to the method in the first embodiment.

[0124] The prediction result acquisition module 502 is used to obtain the lane change probability and lane change timing of the vehicle to be predicted based on the output results of the trajectory prediction model.

[0125] The functions of each module in the devices of this application embodiment can be found in the corresponding descriptions in the above methods, and they have corresponding beneficial effects, which will not be repeated here. It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0126] Figure 6 This is a block diagram of an electronic device used to implement embodiments of this application. Figure 6 As shown, the electronic device includes a memory 610 and a processor 620. The memory 610 stores a computer program that can run on the processor 620. When the processor 620 executes the computer program, it implements the methods described in the above embodiments. The number of memories 610 and processors 620 can be one or more.

[0127] The electronic device also includes:

[0128] The communication interface 630 is used to communicate with external devices and perform data exchange and transmission.

[0129] If the memory 610, processor 620, and communication interface 630 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0130] Optionally, in a specific implementation, if the memory 610, processor 620, and communication interface 630 are integrated on a single chip, then the memory 610, processor 620, and communication interface 630 can communicate with each other through an internal interface.

[0131] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method provided in this application.

[0132] This application also provides a chip including a processor for calling and executing instructions stored in a memory, causing a communication device with the chip installed to perform the method provided in this application.

[0133] This application also provides a chip, including: an input interface, an output interface, a processor, and a memory. The input interface, output interface, processor, and memory are connected through an internal connection path. The processor is used to execute code in the memory. When the code is executed, the processor is used to execute the method provided in the application embodiment.

[0134] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Machines (ARM) architecture.

[0135] Further, optionally, the aforementioned memory may include read-only memory and random access memory. The memory may be volatile memory or non-volatile memory, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available. Examples include Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0136] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0137] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0138] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0139] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved.

[0140] The logic and / or steps described in the flowchart or otherwise herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0141] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. All or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware, the program being stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiments.

[0142] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. This storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0143] The above description is merely an exemplary embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope described in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A training method for a trajectory prediction model, characterized in that, include: The acquired data is preprocessed to obtain training samples; The training samples include vehicle driving data samples; Record the segmented regions traversed by driving data samples that exhibit a specified driving trajectory, including: determining the width of each lane in the road in which the vehicle is traveling; Based on the width, each lane is divided into multiple regions. When the vehicle follows the specified driving trajectory, record the starting and ending areas traversed by the vehicle following the specified driving trajectory. The trajectory prediction model is trained by taking driving data samples containing records of vehicles passing through segmented regions as input and the labels corresponding to the specified driving trajectories as output.

2. The method according to claim 1, characterized in that, The preprocessing of the collected data includes: The data collected at each sampling point is examined to identify duplicate data and data lacking specified content. The duplicate data is deduplicated, and the data lacking the specified content is deleted to obtain the processed data. From the processed data, data collected from multiple consecutive sampling points are selected as training samples.

3. The method according to claim 1, characterized in that, In cases where the specified driving trajectory includes multiple starting areas or multiple ending areas, the following additional methods are also included: Count the region pairs consisting of the starting region and the ending region corresponding to each specified driving trajectory; Multiple regions are clustered to obtain multiple clustering results; each clustering result has the same starting region and the same ending region. Determine the differences in the number of region pairs in each of the clustering results; If the difference exceeds the corresponding threshold, the clustering results with a large number of region pairs are downsampled so that the difference in the number of region pairs in each clustering result is not greater than the corresponding threshold.

4. The method according to claim 1, characterized in that, Also includes: A time window is defined for the specified driving trajectory, and the start time of the time window is dynamically adjusted according to the start time of the specified driving trajectory.

5. The method according to claim 1, characterized in that, The trajectory prediction model includes: A vehicle feature extraction layer is used to extract features related to the vehicle to be predicted from the training samples: the features related to the vehicle to be predicted include location features and driving speed features; An obstacle vehicle feature extraction layer is used to extract obstacle vehicle-related features from the training samples: the obstacle vehicle is an obstacle vehicle of the vehicle to be predicted, and the obstacle vehicle-related features include the relative position of the obstacle vehicle and the vehicle to be predicted, and the rate of change of the relative position of the obstacle vehicle and the vehicle to be predicted. The feature splicing layer is used to splice the features related to the vehicle to be predicted and the features related to the obstacle vehicle, and to perform feature re-extraction processing. An environmental feature extraction layer is used to determine feature points of the road traveled by the vehicle to be predicted in the training samples. The feature points are used to characterize the curvature changes of the road. A fully connected layer is used to determine the lane change probability and timing of the vehicle to be predicted based on the outputs of the feature splicing layer and the environmental feature extraction layer.

6. A method for trajectory prediction, characterized in that, include: The collected driving data containing the vehicle to be predicted is sent to a pre-trained trajectory prediction model, which is trained by the method according to any one of claims 1 to 5; based on the output of the trajectory prediction model, the lane change probability and lane change timing of the vehicle to be predicted are obtained.

7. A training device for a trajectory prediction model, characterized in that, include: The preprocessing module is used to preprocess the acquired data to obtain training samples; The training samples include vehicle driving data samples; The segmentation and recording module is used to record the segmented regions traversed by driving data samples that exhibit a specified driving trajectory. The segmented area is obtained by dividing each lane according to the width of each lane in the road in which the vehicle travels; The segmentation and recording module includes: a width determination submodule, used to determine the width of each lane in the road where the vehicle travels; a region segmentation submodule, used to segment each lane into regions according to the width, dividing each lane into multiple regions; and a region recording submodule, used to record the starting and ending regions traversed by a vehicle with a specified driving trajectory when the vehicle has a specified driving trajectory. The model training module is used to train the trajectory prediction model by taking driving data samples containing records of vehicles passing through segmented regions as input to the trajectory prediction model to be trained, and taking the labels corresponding to the specified driving trajectory as output to the trajectory prediction model to be trained.

8. A trajectory prediction device, characterized in that, include: The data input module uses visual input to send the collected driving data containing the vehicle to be predicted to a pre-trained trajectory prediction model, wherein the trajectory prediction model is trained by the method according to any one of claims 1 to 5. The prediction result acquisition module is used to obtain the lane change probability and lane change timing of the vehicle to be predicted based on the output results of the trajectory prediction model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory, wherein the processor, when executing the computer program, implements the method of any one of claims 1-6.

10. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method of any one of claims 1-6.

Images