Trajectory prediction method and system and driving equipment
By obtaining historical information of vulnerable traffic groups and using deep learning models and post-processing algorithms, the problem of modal diversity and accuracy of trajectory prediction in the existing technology is solved, and more accurate and diversified trajectory prediction is achieved.
Patent Information
- Application Number
- CN202510225241.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-26
- Publication Date
- 2025-06-13
AI Technical Summary
The existing trajectory prediction methods are difficult to ensure the diversity and accuracy of trajectory prediction modes when the intersection scene is severely blocked and the driving intentions of vulnerable traffic groups are flexible.
By obtaining the historical trajectory information of traffic vulnerable groups, surrounding lane information and historical state information of obstacles, and using a pre-constructed deep learning model to determine the first modal trajectory, and then determining the target modal trajectory through a preset rule post-processing algorithm, including a dynamic non-maximum suppression algorithm.
Improve the accuracy and diversity of trajectory prediction, enhance the complete description of the intentions of traffic vulnerable groups, and ensure reliable prediction results in complex environments.
Smart Images
Figure CN120147664A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of intelligent driving technology, and particularly to a trajectory prediction method, system, and driving device. Background Art
[0002] Trajectory prediction technology is used to predict the future trajectories of obstacles around the host vehicle. The predicted future trajectories will directly affect the downstream vehicle control module. Therefore, the autonomous driving system has high requirements for the accuracy of the prediction module. The commonly used prediction method is multi-modal model prediction. Due to severe occlusion in intersection scenarios, poor perception effect, and relatively flexible driving intentions of Vulnerable Road User (VRU) targets, using only the multi-modal model prediction method will cause the trajectory modality to collapse near the main modality, making it difficult to ensure the diversity of the trajectory prediction modality and the possibility of completely describing the VRU intention, resulting in low accuracy of trajectory prediction. Summary of the Invention
[0003] Object of the Invention: Embodiments of this application provide a trajectory prediction method, system, and driving device to improve the accuracy of trajectory prediction for vulnerable road users in the vehicle driving environment.
[0004] Technical Solution: A trajectory prediction method described in embodiments of this application includes:
[0005] Obtain the historical trajectory information of the vulnerable road user, the surrounding lane information of the vulnerable road user, and the historical state information of the surrounding obstacles of the vulnerable road user;
[0006] Determine the first-modal trajectory according to the historical trajectory information of the vulnerable road user, the surrounding lane information of the vulnerable road user, the historical state information of the surrounding obstacles of the vulnerable road user, and a pre-constructed deep learning model;
[0007] Determine the target-modal trajectory of the vulnerable road user according to the first-modal trajectory and a preset rule post-processing algorithm.
[0008] In some embodiments, the preset rule post-processing algorithm includes a preset dynamic non-maximum suppression algorithm; the method for determining the target-modal trajectory of the vulnerable road user includes:
[0009] Preprocess the first-modal trajectory;
[0010] Determine the target-modal trajectory according to the preprocessed first-modal trajectory and the preset dynamic non-maximum suppression algorithm.
[0011] In some embodiments, the method for preprocessing the first-modal trajectory includes:
[0012] Perform coordinate transformation on the first modal trajectory;
[0013] Mask the longitudinal coordinates of the coordinate system after coordinate transformation.
[0014] In some embodiments, the method for performing coordinate transformation on the first modal trajectory includes:
[0015] Arrange the first modal trajectory in descending order of confidence;
[0016] Convert the coordinates of the end point of the first modal trajectory into a local coordinate system.
[0017] In some embodiments, determining the target modal trajectory according to the preprocessed first modal trajectory and the preset dynamic non-maximum suppression algorithm includes:
[0018] Perform standard non-maximum suppression on the preprocessed first modal trajectory according to a first preset distance threshold to obtain an intermediate modal trajectory;
[0019] If the intermediate modal trajectory meets the preset judgment condition, determine the intermediate modal trajectory as the target modal trajectory;
[0020] If the intermediate modal trajectory does not meet the preset judgment condition, update the first preset distance threshold according to a preset update rule, and perform the operation of performing standard non-maximum suppression on the intermediate modal trajectory that does not meet the preset judgment condition according to the updated first preset distance threshold until the target modal trajectory is obtained.
[0021] In some embodiments, the preset judgment condition includes: the number of modes of the modal trajectory meets the preset number-of-modes condition, or, the first preset distance threshold is less than the second preset distance threshold.
[0022] In some embodiments, the preset update rule includes: reducing the first preset distance threshold according to a preset ratio.
[0023] In some embodiments, the method for determining the first modal trajectory includes:
[0024] Input the historical trajectory information of the traffic vulnerable group, the surrounding lane information of the traffic vulnerable group, and the historical state information of the surrounding obstacles of the traffic vulnerable group into the deep learning model;
[0025] And perform single-stage training on the deep learning model to obtain the first modal trajectory.
[0026] Correspondingly, an embodiment of the present application further provides a trajectory prediction system, including:
[0027] An information acquisition module, configured to acquire the historical trajectory information of the traffic vulnerable group, the surrounding lane information of the traffic vulnerable group, and the historical status information of the surrounding obstacles of the traffic vulnerable group;
[0028] A first determination module, configured to determine a first modal trajectory according to the historical trajectory information of the traffic vulnerable group, the surrounding lane information of the traffic vulnerable group, the historical status information of the surrounding obstacles of the traffic vulnerable group, and a pre-constructed deep learning model;
[0029] A second determination module, configured to determine the target modal trajectory of the traffic vulnerable group according to the first modal trajectory and a preset rule post-processing algorithm.
[0030] Correspondingly, an embodiment of the present application further provides a driving device, which includes the above-mentioned trajectory prediction system.
[0031] Beneficial effects: Compared with the prior art, the trajectory prediction method, system, and driving device in the embodiments of the present application. The trajectory prediction method includes: acquiring the historical trajectory information of the traffic vulnerable group, the surrounding lane information of the traffic vulnerable group, and the historical status information of the surrounding obstacles of the traffic vulnerable group; determining a first modal trajectory according to the historical trajectory information of the traffic vulnerable group, the surrounding lane information of the traffic vulnerable group, the historical status information of the surrounding obstacles of the traffic vulnerable group, and a pre-constructed deep learning model; determining the target modal trajectory of the traffic vulnerable group according to the first modal trajectory and a preset rule post-processing algorithm. The trajectory prediction method provided by the present application obtains a fallback first modal trajectory according to the historical trajectory information of the traffic vulnerable group, the surrounding lane information of the traffic vulnerable group, the historical status information of the surrounding obstacles of the traffic vulnerable group, and a pre-constructed deep learning model, ensuring the reliability of trajectory prediction, and processing the first modal trajectory through a preset rule post-processing algorithm to enhance the diversity of the modal trajectory, improving the possibility of completely describing the intention of the VRU, thereby improving the accuracy of modal trajectory prediction to provide accurate and stable prediction results. Description of the Drawings
[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those skilled in the art, without creative efforts, other drawings can be obtained according to these drawings.
[0033] Figure 1 It is a flowchart of a trajectory prediction method provided in an embodiment of the present application;
[0034] Figure 2It is a schematic structural diagram of the deep learning model provided in the embodiments of the present application;
[0035] Figure 3 It is a schematic diagram of an application scenario of a deep learning model provided in the embodiments of the present application;
[0036] Figure 4 It is a schematic overall flow diagram of a trajectory prediction method provided in the embodiments of the present application;
[0037] Figure 5 It is a schematic flow diagram of a preset rule post-processing algorithm provided in the embodiments of the present application;
[0038] Figure 6 It is a schematic flow diagram of a dynamic NMS provided in the embodiments of the present application;
[0039] Figure 7 It is a schematic principle block diagram of a trajectory prediction system provided in the embodiments of the present application;
[0040] Figure 8 It is a schematic structural diagram of the electronic device provided in the embodiments of the present application.
[0041] Reference numerals:
[0042] 101 - Information acquisition module; 102 - First determination module; 103 - Second determination module; 100 - Trajectory prediction system. Detailed implementation manners
[0043] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative efforts belong to the scope of protection of the present application.
[0044] It should be understood that although terms such as first and second may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another. Therefore, the first component described below may be referred to as the second component without departing from the teachings of the concept of the present application. As used herein, the term "and / or" includes any one of the associated listed items and all combinations of one or more of them.
[0045] Figure 1It is a flowchart of a trajectory prediction method provided in an embodiment of the present application. This method is applicable to the process of realizing the trajectory prediction of vulnerable road users in the driving environment of a vehicle in an autonomous driving management platform. This method can be executed by a trajectory prediction system, which can be implemented in a software and / or hardware manner, and can be configured in a processor or controller of an autonomous driving management platform. Please refer to Figure 1 , and the method includes the following steps:
[0046] Step 110, obtain the historical trajectory information of vulnerable road users, the surrounding lane information of vulnerable road users, and the historical state information of surrounding obstacles of vulnerable road users.
[0047] Among them, vulnerable road users refer to pedestrians, cyclists, motorcyclists, etc. in the driving environment of the target vehicle (usually referred to as the ego vehicle). Among them, the driving environment of the target vehicle includes the driving environment of intersection scenarios and non-intersection scenarios.
[0048] Among them, the historical trajectory information of vulnerable road users includes the historical trajectory information of vulnerable road users within the first preset time period. Among them, the historical trajectory information includes information such as the position, speed, heading, and acceleration of vulnerable road users. Among them, the first preset time period is a value such as 1 second, and can be specifically set according to actual situations, and no specific limitation is made here.
[0049] Among them, the surrounding lane information of vulnerable road users includes information such as the position and speed of the surrounding lanes of vulnerable road users.
[0050] Among them, the historical state information of surrounding obstacles of vulnerable road users includes the historical state information of surrounding obstacles within the second preset time period. Among them, the historical state information includes information such as the position, speed, heading, and acceleration of surrounding obstacles. Among them, the second preset time period is a value such as 2 seconds, and can be specifically set according to actual situations, and no specific limitation is made here.
[0051] Specifically, the specific acquisition methods of the historical trajectory information of vulnerable road users, the surrounding lane information of vulnerable road users, and the historical state information of surrounding obstacles of vulnerable road users are as follows: First, the original data is obtained by collecting the live data of the target vehicle. Among them, the original data includes the environmental perception data of vulnerable road users, the positioning information of the target vehicle, the chassis information of the target vehicle, etc. Then, the original data is processed offline into the format of a bag package. Thus, the historical trajectory information of vulnerable road users, the surrounding lane information of vulnerable road users, and the historical state information of surrounding obstacles of vulnerable road users (i.e., the subsequent deep learning model training and test sample data) are extracted from the encapsulated bag package.
[0052] Exemplarily, the historical trajectory information of traffic disadvantaged groups includes: the trajectory information of the prediction target in the past 1 second: where n represents the frame number, It means from the -10th frame in history to the current frame, with an inter-frame interval of 100 ms. The sampling frequency is 10 HZ, so a total of 10 position points are sampled within 1 s. Among them, U n =(x n , y n , vx n , vy n , yaw n ), which is used to represent the position, speed and heading of the position point. Among them, U n represents the trajectory information of the prediction target at each moment. Among them, x n and y n respectively represent the x coordinate and y coordinate under the historical trajectory. vx n and vy n respectively represent the speed component of the x-axis and the speed component of the y-axis under the historical trajectory. yaw n represents the heading under the historical trajectory. Among them, the historical state information of the surrounding obstacles of traffic disadvantaged groups includes: the future 6-second trajectory The historical 1-second trajectory of the surrounding obstacles of the prediction target, expressed as: O k =(x k , y k , vx k , vy k , yaw k ). Among them, F U represents the future prediction trajectory, and H O is the trajectory information of other obstacles around the obstacle to be predicted. Among them, k is the frame number. x k and y k respectively represent the x coordinate and y coordinate under the future trajectory. vx k and vy k respectively represent the speed component of the x-axis and the speed component of the y-axis under the future trajectory. yaw k represents the heading under the future trajectory. Among them, the surrounding lane information of traffic disadvantaged groups includes: the surrounding lane information of the prediction target, expressed as: R i =(x i , y i ), and H R is the surrounding lane information of the obstacle to be predicted, and R i represents the two-dimensional coordinates included in the lane.
[0053] In addition, to improve the efficiency of subsequent deep learning model training, when constructing model training sample data, the motion scenarios of VRUs are divided into six categories, namely: going straight along the lane, idling, crossing at an intersection, turning left at an intersection, crossing without a crosswalk, and standing still without a crosswalk. Among them, going straight along the lane means that the VRU goes straight along the extension direction of the lane. Idling means driving at a slow speed. Crossing at an intersection means that the VRU crosses the road. Turning left at an intersection means that the VRU turns left at the intersection. Crossing without a crosswalk means crossing where there is no crosswalk. Standing still without a crosswalk means standing still on a road without a crosswalk. And each type of motion scenario contains multiple small training subsets (mini-batch), that is, each mini-batch covers all categories to achieve data balance during training. Among them, the proportion of each motion scenario in the training set is 7:3:3:3:2:1, and the total amount of data is 15 million. Among them, the proportion of each motion scenario in the training set can also be set to other ratios, which can be specifically set according to the actual situation and will not be specifically limited here.
[0054] Step 120: Determine the first-modal trajectory according to the historical trajectory information of the vulnerable road users, the surrounding lane information of the vulnerable road users, the historical state information of the surrounding obstacles of the vulnerable road users, and the pre-constructed deep learning model.
[0055] Among them, the first-modal trajectory includes multiple modal trajectories, and the specific number of modalities is related to the actual model training situation.
[0056] Among them, the deep learning model can be constructed based on one or more of a neural network model (Transformer) based on self-attention mechanism, a recurrent neural network model (Recurrent Neural Network, RNN), and a long short-term memory network model (Long Short-Term Memory, LSTM).
[0057] Among them, Transformer is a neural network model based on self-attention mechanism, which is used to process sequence data and has made significant breakthroughs especially in natural language processing (Natural Language Processing, NLP) tasks.
[0058] Among them, when traditional recurrent neural network (Recurrent Neural Network, RNN) processes sequence data, it must process the input one by one in order and has the problem of being difficult to parallelize. The Transformer model allows parallel computing by introducing self-attention mechanism, which greatly accelerates the training and inference processes.
[0059] Among them, LSTM (Long Short-Term Memory) is a variant of the Recurrent Neural Network (RNN), which is used to process and model sequential data. LSTM focuses on solving the long-term dependence problem in traditional RNNs. By selectively forgetting, memorizing, and updating information, it can better capture and process the long-term dependence relationships in sequential data, and is widely used in various sequential modeling and prediction tasks.
[0060] Exemplarily, in the technical solution of the embodiment of the present application, a deep learning network model based on Transformer is taken as an example for illustration, and the same will not be repeated hereinafter.
[0061] Figure 2 It is a schematic structural diagram of the deep learning model provided in the embodiment of the present application. In some embodiments, the method for determining the first modal trajectory includes: inputting the historical trajectory information of traffic vulnerable groups, the surrounding lane information of traffic vulnerable groups, and the historical state information of surrounding obstacles of traffic vulnerable groups into the deep learning model; and performing single-stage training on the deep learning model to obtain the first modal trajectory.
[0062] Exemplarily, please refer to Figure 2 , the deep learning model includes Concat (Concatenation), Multi-Layer Perceptron (MLP), self-attention mechanism, Cross-Attention mechanism, Trajectory Prediction Head (Traj Header), and Confidence Score Head (Score Header). Among them, Concat is used for concatenation. MLP is a multi-layer perceptron, which is the simplest neural network. Self-attention is used to process the relationships between elements within a sequence. Cross-attention is used to process the relationships between different sequences. Traj header is the predicted trajectory obtained by training the deep learning model. Score header is the predicted trajectory score obtained by training the deep learning model.
[0063] Figure 3 It is a schematic diagram of an application scenario of a deep learning model provided in the embodiment of the present application. Exemplarily, refer to Figure 3, the deep learning model uses single-stage training, and inputs the historical trajectory information of traffic vulnerable groups, the surrounding lane information of traffic vulnerable groups, and the historical state information of surrounding obstacles of traffic vulnerable groups. For example, the state information of the to-be-predicted VRU target and surrounding obstacles in the past 1s is input. Among them, the state information includes position, speed, heading, acceleration, etc., and the map information of the lane information is represented by two-dimensional coordinate points. Among them, the trajectory prediction head is used to output the first-modal trajectory, for example, output 6 predicted modal trajectories (assuming the first-modal trajectory includes 6 modal trajectories). Among them, the confidence score head is used to output the confidence or probability corresponding to the first-modal trajectory. For example, the confidence corresponding to each of the 6 predicted modal trajectories is output. During the training process, the Negative Log Likelihood (NLL) loss function is used to constrain the trajectory generation process.
[0064] Specifically, the specific implementation process of outputting the first-modal trajectory and the corresponding confidence according to the deep learning model is as follows: First, multi-modal data fusion, input the historical trajectory information of traffic vulnerable groups, the surrounding lane information of traffic vulnerable groups, and the historical state information of surrounding obstacles of traffic vulnerable groups into the deep learning model, and use Transformer to extract the features of each modality. Then, feature fusion, fuse the features of different modalities through Transformer to generate joint features. Secondly, multi-modal trajectory prediction, use Transformer to predict future trajectory points and output possible trajectories, that is, output the first-modal trajectory. Finally, confidence evaluation, use Transformer to assign a confidence score to each modal trajectory, that is, output the confidence corresponding to the first-modal trajectory.
[0065] Step 130, determine the target modal trajectory of the traffic vulnerable group according to the first-modal trajectory and the preset rule post-processing algorithm.
[0066] Among them, the preset rule post-processing algorithm includes a preset dynamic non-maximum suppression algorithm. Among them, the main function of Non-Maximum Suppression (NMS) is to remove redundant bounding boxes during the detection process to ensure the accuracy and simplicity of the detection results. Specifically, NMS compares the overlap degree between different bounding boxes, selectively retains the bounding box with the highest confidence, and suppresses other bounding boxes that overlap with it and have lower confidence. In this way, for each target in the image, only one most likely bounding box containing the target is finally retained. Among them, the comparison of the overlap degree between different bounding boxes usually uses the Intersection over Union (IoU) as a measurement standard.
[0067] Figure 4It is a schematic diagram of the overall process of a trajectory prediction method provided in an embodiment of the present application. Exemplarily, please refer to Figure 4 , first, the perception (Agent) obtains the historical trajectory information of traffic vulnerable groups, the historical state information of surrounding obstacles of traffic vulnerable groups, and the surrounding lane information of traffic vulnerable groups. Then, the historical trajectory information of traffic vulnerable groups, the surrounding lane information of traffic vulnerable groups, and the historical state information of surrounding obstacles of traffic vulnerable groups are input into a deep learning model for training, and a fallback prediction modal trajectory (i.e., the first modal trajectory) can be obtained. Secondly, the first modal trajectory is processed by a post-processing algorithm according to a preset rule to output a multi-modal trajectory, so as to enhance the diversity of the trajectory, improve the possibility of completely describing the intention of the VRU, and thus improve the accuracy of trajectory prediction.
[0068] In some embodiments, the method for determining the target modal trajectory of traffic vulnerable groups includes the following steps:
[0069] Step 1: Preprocess the first modal trajectory.
[0070] Specifically, preprocessing the first modal trajectory can simplify the complexity of data processing, reduce the interference of unnecessary factor data, and thus improve the accuracy and reliability of subsequent target modal trajectory prediction.
[0071] In some embodiments, the method for preprocessing the first modal trajectory includes: performing coordinate transformation on the first modal trajectory; masking the longitudinal coordinates of the coordinate system after coordinate transformation.
[0072] Specifically, performing coordinate transformation on the first modal trajectory ensures the consistency and comparability of data, simplifies the calculation process, and improves the calculation accuracy. Moreover, masking the longitudinal coordinates of the coordinate system after coordinate transformation can mask the interference of possible noise and measurement errors on the longitudinal coordinates, reduce the complexity of calculation, enable the subsequent preset dynamic non-maximum suppression algorithm to focus more on the data of the transverse coordinates, reduce the influence of noise, simplify the complexity of the algorithm, and improve the accuracy and reliability of trajectory prediction.
[0073] In some embodiments, the method for performing coordinate transformation on the first modal trajectory includes: arranging the first modal trajectory in descending order of confidence; converting the coordinates of the end point of the first modal trajectory into a local coordinate system.
[0074] Among them, the confidence of the first modal trajectory is output by the confidence scoring head of the deep learning model.
[0075] Figure 5 It is a schematic diagram of the process of a preset rule post-processing algorithm provided in an embodiment of the present application. Exemplarily, please refer to Figure 5, the overall process of the preset rule post - processing algorithm is as follows: When performing coordinate transformation on the first - mode trajectory, each mode trajectory within the first - mode trajectory is arranged in descending order according to its confidence level (or probability). Then, the coordinates of the end - points of each mode trajectory in the first - mode trajectory after being arranged in descending order of confidence level are converted into a local coordinate system. Secondly, the longitudinal coordinates of the local coordinate system after coordinate transformation are masked. Then, the set of end - point vectors of each mode trajectory in the masked local coordinate system is fed into the preset dynamic non - maximum suppression algorithm for dynamic NMS, and through dynamic NMS, multi - mode trajectories and probability updates are output to obtain the target mode trajectory.
[0076] Step 2: Determine the target mode trajectory according to the pre - processed first - mode trajectory and the preset dynamic non - maximum suppression algorithm.
[0077] Specifically, the first - mode trajectory is output through a deep - learning model to obtain a prediction trajectory for fallback. Then, the first - mode trajectory is pre - processed, and the set of end - point vectors of each mode trajectory in the masked local coordinate system is fed into the preset dynamic non - maximum suppression algorithm for dynamic NMS. Through dynamic NMS, multi - mode trajectories are output, enhancing the diversity of trajectory modes, enhancing the model's description ability of the VRU intention space, and at the same time suppressing redundant trajectories that collapse near the high - confidence prediction trajectory, taking into account the rationality and efficiency of trajectory prediction.
[0078] In some embodiments, determining the target mode trajectory according to the pre - processed first - mode trajectory and the preset dynamic non - maximum suppression algorithm includes: performing standard non - maximum suppression on the pre - processed first - mode trajectory according to a first preset distance threshold to obtain an intermediate mode trajectory; if the intermediate mode trajectory meets the preset judgment condition, then determine the intermediate mode trajectory as the target mode trajectory; if the intermediate mode trajectory does not meet the preset judgment condition, then update the first preset distance threshold according to the preset update rule, and perform the operation of standard non - maximum suppression on the intermediate mode trajectory that does not meet the preset judgment condition according to the updated first preset distance threshold until the target mode trajectory is obtained.
[0079] Among them, the specific value of the first preset distance threshold can be set according to the actual situation and will not be specifically limited here.
[0080] In some embodiments, the preset judgment condition includes: the number of modes of the mode trajectory meets the preset mode - number condition, or the first preset distance threshold is less than the second preset distance threshold.
[0081] Among them, the preset mode - number condition can be, for example, outputting trajectories with 3 modes, etc., and can be specifically set according to the actual situation and will not be specifically limited here.
[0082] Among them, the second preset distance threshold is a value such as 1 meter, which can be specifically set according to the actual situation and will not be specifically limited here.
[0083] In some embodiments, the preset update rule includes: reducing the first preset distance threshold according to a preset ratio.
[0084] Among them, the preset ratio is a value such as 0.9, which can be specifically set according to the actual situation and will not be specifically limited here.
[0085] Figure 6 is a schematic flow chart of a dynamic NMS provided in an embodiment of the present application. Exemplarily, refer to Figure 6 , sort each modal trajectory in the first modal trajectory in descending order of confidence (or probability). And perform preprocessing such as coordinate transformation and masking. Then, perform a standard NMS on the preprocessed first modal trajectory according to the first preset distance threshold. If the modal trajectory (i.e., the intermediate modal trajectory) obtained by one standard NMS meets the preset judgment condition, the intermediate modal trajectory is determined as the target modal trajectory and the program ends. If the modal trajectory (i.e., the intermediate modal trajectory) obtained by one standard NMS does not meet the preset judgment condition, the first preset distance threshold is reduced according to the preset ratio and then the standard NMS operation is performed, and the standard NMS operation is repeatedly executed until the target modal trajectory is obtained. Among them, the trajectory suppressed by the local highest probability contributes its probability to the trajectory with the highest local probability. Thus, the modal diversity of the trajectory prediction result can be effectively enhanced, while suppressing repeated modes, ensuring the quality of multi-modal trajectories, and improving the accuracy of trajectory prediction.
[0086] It can be seen from this that the trajectory prediction method provided in the embodiment of the present application obtains a fallback first modal trajectory based on the historical trajectory information of traffic vulnerable groups, the surrounding lane information of traffic vulnerable groups, the historical state information of surrounding obstacles of traffic vulnerable groups, and a pre-constructed deep learning model, ensuring the reliability of trajectory prediction. And the first modal trajectory is processed by a preset rule post-processing algorithm to enhance the diversity of modal trajectories, improve the possibility of completely describing the intention of VRU, thereby improving the accuracy of modal trajectory prediction to provide accurate and stable prediction results.
[0087] Figure 7 is a schematic structural block diagram of the principle of a trajectory prediction system provided in an embodiment of the present application. Correspondingly, the embodiment of the present application also provides a trajectory prediction system. Please refer to Figure 7, the trajectory prediction system 100 includes: an information acquisition module 101, configured to acquire historical trajectory information of a traffic vulnerable group, surrounding lane information of the traffic vulnerable group, and historical state information of surrounding obstacles of the traffic vulnerable group; a first determination module 102, configured to determine a first modal trajectory according to the historical trajectory information of the traffic vulnerable group, the surrounding lane information of the traffic vulnerable group, the historical state information of the surrounding obstacles of the traffic vulnerable group, and a pre-constructed deep learning model; and a second determination module 103, configured to determine a target modal trajectory of the traffic vulnerable group according to the first modal trajectory and a preset rule post-processing algorithm.
[0088] The technical solution of the embodiment of the present application provides a trajectory prediction system, which includes an information acquisition module, configured to acquire the historical trajectory information of the traffic vulnerable group, the surrounding lane information of the traffic vulnerable group, and the historical state information of the surrounding obstacles of the traffic vulnerable group; a first determination module, configured to determine a first modal trajectory according to the historical trajectory information of the traffic vulnerable group, the surrounding lane information of the traffic vulnerable group, the historical state information of the surrounding obstacles of the traffic vulnerable group, and a pre-constructed deep learning model; and a second determination module, configured to determine the target modal trajectory of the traffic vulnerable group according to the first modal trajectory and a preset rule post-processing algorithm. The trajectory prediction system provided by the embodiment of the present application obtains a fallback first modal trajectory according to the historical trajectory information of the traffic vulnerable group, the surrounding lane information of the traffic vulnerable group, the historical state information of the surrounding obstacles of the traffic vulnerable group, and a pre-constructed deep learning model, ensuring the reliability of trajectory prediction. And the first modal trajectory is processed by a preset rule post-processing algorithm to enhance the diversity of the modal trajectory and improve the possibility of completely describing the intention of the VRU, thereby improving the accuracy of modal trajectory prediction and providing accurate and stable prediction results.
[0089] In some embodiments, the preset rule post-processing algorithm includes a preset dynamic non-maximum suppression algorithm. The second determination module 103 is further configured to: preprocess the first modal trajectory; and determine the target modal trajectory according to the preprocessed first modal trajectory and the preset dynamic non-maximum suppression algorithm.
[0090] In some embodiments, the second determination module 103 is further configured to: perform a coordinate transformation on the first modal trajectory; and mask the longitudinal coordinates of the coordinate system after the coordinate transformation.
[0091] In some embodiments, the second determination module 103 is further configured to: arrange the first modal trajectory in descending order of confidence; and convert the coordinates of the end point of the first modal trajectory into a local coordinate system.
[0092] In some embodiments, the second determination module 103 is further configured to: perform standard non-maximum suppression on the preprocessed first modal trajectory according to a first preset distance threshold to obtain an intermediate modal trajectory; if the intermediate modal trajectory meets a preset judgment condition, determine the intermediate modal trajectory as the target modal trajectory; if the intermediate modal trajectory does not meet the preset judgment condition, update the first preset distance threshold according to a preset update rule, and perform the operation of performing standard non-maximum suppression on the intermediate modal trajectory that does not meet the preset judgment condition according to the updated first preset distance threshold until the target modal trajectory is obtained.
[0093] In some embodiments, the preset judgment condition includes: the number of modes of the modal trajectory meets a preset number-of-modes condition, or the first preset distance threshold is less than a second preset distance threshold.
[0094] In some embodiments, the preset update rule includes: reducing the first preset distance threshold according to a preset ratio.
[0095] In some embodiments, the first determination module 102 is further configured to: input the historical trajectory information of the traffic vulnerable group, the surrounding lane information of the traffic vulnerable group, and the historical state information of the surrounding obstacles of the traffic vulnerable group into the deep learning model; and perform single-stage training on the deep learning model to obtain the first modal trajectory.
[0096] Correspondingly, an embodiment of the present application further provides a driving device, and the driving device includes the trajectory prediction system provided in any embodiment of the present application.
[0097] Wherein, the driving device can be an automobile, an electric vehicle, an engineering vehicle, etc.
[0098] Correspondingly, an embodiment of the present application further provides an electronic device. Please refer to Figure 8 , Figure 8 which illustrates the structural diagram of the electronic device according to the embodiment of the present application. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above-mentioned trajectory prediction method are implemented. Since the above-mentioned trajectory prediction method has been described in detail, it will not be elaborated here.
[0099] Correspondingly, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned trajectory prediction method are implemented. Since the above-mentioned trajectory prediction method has been described in detail, it will not be elaborated here.
[0100] In the above embodiments, the descriptions of the respective embodiments have their own focuses. For parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.
[0101] The above has introduced in detail the trajectory prediction method, system and driving device provided by the embodiments of the present application, and specific examples have been used to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the technical solution and its core idea of the present application; those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.
Claims
1. A trajectory prediction method, characterized in that: For predicting the trajectory of vulnerable groups in a target vehicle driving environment, the method comprises: Acquire historical trajectory information of the vulnerable traffic group, lane information surrounding the vulnerable traffic group, and historical status information of obstacles surrounding the vulnerable traffic group; Determine a first modal trajectory according to historical trajectory information of the vulnerable traffic group, surrounding lane information of the vulnerable traffic group, historical state information of surrounding obstacles of the vulnerable traffic group, and a pre-built deep learning model; The target modal trajectory of the traffic disadvantaged group is determined according to the first modal trajectory and a preset rule post-processing algorithm.
2. The trajectory prediction method according to claim 1, characterized in that: The preset rule post-processing algorithm includes a preset dynamic non-maximum suppression algorithm; The method for determining the target modal trajectory of the traffic vulnerable group comprises: Preprocessing the first modal trajectory; The target modal trajectory is determined according to the preprocessed first modal trajectory and the preset dynamic non-maximum suppression algorithm.
3. The trajectory prediction method according to claim 2, characterized in that: The method for preprocessing the first modal trajectory comprises: Performing coordinate transformation on the first modal trajectory; The vertical coordinate of the coordinate system after the mask coordinate transformation.
4. The trajectory prediction method according to claim 3, characterized in that: The method for performing coordinate transformation on the first modal trajectory comprises: Arrange the first modal trajectories in descending order of confidence; The coordinates of the end point of the first modal trajectory are converted into a local coordinate system.
5. The trajectory prediction method according to claim 2, characterized in that: The determining the target modal trajectory according to the preprocessed first modal trajectory and the preset dynamic non-maximum suppression algorithm includes: Performing standard non-maximum suppression on the preprocessed first modal trajectory according to a first preset distance threshold to obtain an intermediate modal trajectory; If the intermediate modal trajectory meets the preset judgment condition, determining the intermediate modal trajectory as the target modal trajectory; If the intermediate modal trajectory does not meet the preset judgment condition, the first preset distance threshold is updated according to the preset update rule, and the intermediate modal trajectory that does not meet the preset judgment condition is repeatedly subjected to the standard non-maximum suppression operation according to the updated first preset distance threshold until the target modal trajectory is obtained.
6. The trajectory prediction method according to claim 5, characterized in that: The preset judgment condition includes: the modal quantity of the modal trajectory meets the preset modal quantity condition, or the first preset distance threshold is less than the second preset distance threshold.
7. The trajectory prediction method according to claim 5, characterized in that: The preset update rule includes: reducing the first preset distance threshold according to a preset ratio.
8. The trajectory prediction method according to claim 1, characterized in that: The method for determining the first modal trajectory comprises: Inputting historical trajectory information of the vulnerable traffic group, surrounding lane information of the vulnerable traffic group, and historical status information of surrounding obstacles of the vulnerable traffic group into the deep learning model; And the deep learning model is trained in a single stage to obtain the first modal trajectory.
9. A trajectory prediction system, characterized in that: For predicting the trajectory of vulnerable groups in a target vehicle driving environment, the system comprises: An information acquisition module, used to acquire historical trajectory information of the vulnerable traffic group, lane information around the vulnerable traffic group, and historical status information of obstacles around the vulnerable traffic group; A first determination module is used to determine a first modal trajectory according to historical trajectory information of the vulnerable traffic group, surrounding lane information of the vulnerable traffic group, historical state information of surrounding obstacles of the vulnerable traffic group, and a pre-built deep learning model; The second determination module is used to determine the target modal trajectory of the traffic disadvantaged group according to the first modal trajectory and a preset rule post-processing algorithm.
10. A driving device, characterized in that: Comprising the trajectory prediction system as claimed in claim 9.