Vehicle track prediction method and device, vehicle, storage medium and product
By acquiring historical trajectory and map information of the target vehicle and related vehicles, and using an interactive coding module to predict the trajectory, the problem of insufficient trajectory prediction accuracy in existing technologies is solved, thereby improving the safety and reliability of autonomous driving.
Patent Information
- Application Number
- CN202410479938.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-19
- Publication Date
- 2025-10-24
AI Technical Summary
Existing trajectory prediction technologies lack accuracy in autonomous driving and struggle to cope with complex road scenarios, resulting in low safety and reliability of autonomous driving algorithms.
By acquiring the target vehicle's historical driving trajectory information, the historical driving trajectory information of associated vehicles, and map information, the interaction coding module is used to determine the interaction characteristics between vehicles and between vehicles and the map, thereby predicting the target vehicle's driving trajectory.
This improves the accuracy and reliability of trajectory prediction, thereby enhancing the driving safety of the vehicle.
Smart Images
Figure CN120828824A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present disclosure relates to the technical field of vehicles, and particularly relates to a vehicle trajectory prediction method and device, a vehicle, a storage medium and a product. BACKGROUND
[0002] In order to improve the safety of vehicle driving, intelligent driving technology is currently applied to vehicles to realize assisted driving and automatic driving of vehicles. Among them, trajectory prediction is a link in intelligent driving, which gives an automatic driving car the ability to dynamically predict the future possible running trajectory of the surrounding obstacles. Trajectory prediction takes over the perception module and provides the planning module with the ability to predict the future.
[0003] In related technologies, the trajectory prediction technical solution is mostly a rule-based model. An engineer predefines a mathematical modeling of vehicle driving, which is generally expressed as a physical model, and the simplest one is uniform acceleration straight line motion. According to the observed motion parameters (for example, speed, acceleration, orientation angle, orientation angle angular velocity, etc.) of the target vehicle, a set of rules pre-defined is used to calculate the parameters of the physical model, which are substituted into the physical model, and then a future predicted trajectory is generated in combination with the topological information of the lane line of the high-precision map. Although the rule-based model trajectory prediction method has good interpretability, on the one hand, it is difficult to perfectly express the motion law of the vehicle by physical modeling, and on the other hand, it is also difficult for engineers to define a detailed and accurate rule to calculate the parameters of the physical model. The actual road scene and driving scene are constantly changing, and some situations that have not been fully considered in advance will always occur, so the accuracy of the predicted trajectory will be reduced. The safety and reliability of the automatic driving algorithm are low. SUMMARY
[0004] In order to overcome the problems in the related art, the present disclosure provides a vehicle trajectory prediction method, device, vehicle, storage medium and product.
[0005] According to a first aspect of an embodiment of the present disclosure, a vehicle trajectory prediction method is provided, comprising: In the driving process of the ego vehicle, historical driving trajectory information of a target vehicle around the ego vehicle, historical driving trajectory information of an associated vehicle of the target vehicle, and map information of a driving environment in which the target vehicle is located are acquired; According to the historical driving trajectory information of the target vehicle and the historical driving trajectory information of the associated vehicle, an interaction feature between vehicles is determined; According to the interaction feature information between vehicles and the map information, an interaction feature between the vehicle and the map is determined; At least according to the interaction feature between the vehicle and the map, a driving trajectory of the target vehicle is predicted.
[0006] Optionally, the determining the interaction feature between vehicles according to the historical driving trajectory information of the target vehicle and the historical driving trajectory information of the associated vehicle comprises: processing the historical driving trajectory information of the target vehicle and the historical driving trajectory information of the associated vehicle by using a first interaction encoding module to obtain the interaction feature between vehicles; The determining the interaction feature between vehicles according to the historical driving trajectory information of the target vehicle and the historical driving trajectory information of the associated vehicle comprises: processing the interaction feature information between vehicles and the map information by using a second interaction encoding module to obtain the interaction feature between vehicles and the map, wherein the second interaction encoding module is different from the first interaction encoding module.
[0007] Optionally, the determining the interaction feature between vehicles according to the historical driving trajectory information of the target vehicle and the historical driving trajectory information of the associated vehicle further comprises: encoding the historical driving trajectory information of the target vehicle and the historical driving trajectory information of the associated vehicle by using a time sequence interaction encoding module to obtain the historical driving trajectory encoding feature of the target vehicle and the historical driving trajectory encoding feature of the associated vehicle; The processing the historical driving trajectory information of the target vehicle and the historical driving trajectory information of the associated vehicle by using a first interaction encoding module to obtain the interaction feature between vehicles comprises: processing the historical driving trajectory encoding feature of the target vehicle and the historical driving trajectory encoding feature of the associated vehicle by using a first interaction encoding module to obtain the interaction feature between vehicles.
[0008] Optionally, the predicting the driving trajectory of the target vehicle according to at least the interaction feature between vehicles and the map comprises: randomly generating a reference position point of the target vehicle by using a decoding module; predicting the driving trajectory of the target vehicle according to the reference position point and the interaction feature between vehicles and the map by using the decoding module.
[0009] Optionally, the reference position point is multiple, and the driving directions indicated by the multiple reference position points are different; the predicting the driving trajectory of the target vehicle according to the reference position point and the interaction feature between vehicles and the map by using the decoding module comprises: for each reference position point, predicting the driving trajectory of the target vehicle according to the reference position point and the interaction feature between vehicles and the map by using the decoding module to obtain multiple driving trajectories.
[0010] Optionally, the method further comprises: The decoding module is configured to predict a confidence degree of each of the driving trajectories, the confidence degree being used to represent a probability of the target vehicle driving according to the driving trajectory.
[0011] Optionally, the at least according to the interaction feature between the vehicle and the map, the driving trajectory of the target vehicle is predicted, including: According to the interaction feature between the vehicle and the map, the interaction feature between the vehicles and the historical driving trajectory coding feature of the target vehicle, the driving trajectory of the target vehicle is predicted.
[0012] Optionally, the target vehicle is multiple, and the driving trajectory of each of the target vehicles is predicted in parallel.
[0013] According to a second aspect of the embodiments of the present disclosure, a vehicle trajectory prediction device is provided, including: The first acquisition module is configured to acquire, in a driving process of a self vehicle, historical driving trajectory information of a target vehicle around the self vehicle, historical driving trajectory information of an associated vehicle of the target vehicle, and map information of a driving environment in which the target vehicle is located; The first determination module is configured to determine an interaction feature between vehicles according to the historical driving trajectory information of the target vehicle and the historical driving trajectory information of the associated vehicle; The second determination module is configured to determine an interaction feature between a vehicle and a map according to the interaction feature information between the vehicles and the map information; The first prediction module is configured to predict a driving trajectory of the target vehicle at least according to the interaction feature between the vehicle and the map.
[0014] According to a third aspect of the embodiments of the present disclosure, a vehicle is provided, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the vehicle trajectory prediction method of the first aspect of the embodiments of the present disclosure.
[0015] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, which stores computer program instructions, the program instructions being executed by a processor to implement the vehicle trajectory prediction method of the first aspect of the embodiments of the present disclosure.
[0016] According to a fifth aspect of the embodiments of the present disclosure, a computer program product is provided, including a computer program, the computer program being executed by a processor to implement the vehicle trajectory prediction method of the first aspect of the embodiments of the present disclosure.
[0017] According to the technical solution, after the historical driving track information of the target vehicle, the historical driving track information of the associated vehicle of the target vehicle, and the map information are acquired, the interaction feature between vehicles is determined, then the interaction feature between the vehicle and the map is determined according to the interaction feature information between vehicles and the map information, and finally, the driving track of the target vehicle is predicted according to at least the interaction feature between the vehicle and the map. In this way, when the driving track of the target vehicle is predicted, the mutual influence between the associated vehicle and the target vehicle and the influence of the map information on the associated vehicle and the target vehicle are considered, the reliability and accuracy of the predicted driving track of the target vehicle are improved, and the driving safety of the ego vehicle is improved.
[0018] It should be understood that the general description above and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS
[0019] The accompanying drawings, which are incorporated into and form part of the specification, illustrate an embodiment consistent with the present disclosure and, together with the specification, serve to explain the principles of the present disclosure.
[0020] Figure 1 is a schematic diagram of a driving scene according to an exemplary embodiment.
[0021] Figure 2 is a flowchart of a vehicle track prediction method according to an exemplary embodiment.
[0022] Figure 3 is a schematic diagram of determining an associated vehicle according to an exemplary embodiment.
[0023] Figure 4 is a schematic diagram of a neural network according to an exemplary embodiment.
[0024] Figure 5 is a block diagram of a vehicle track prediction device according to an exemplary embodiment.
[0025] Figure 6 is a block diagram of a vehicle according to an exemplary embodiment. DETAILED DESCRIPTION
[0026] The exemplary embodiments will be described in detail herein with reference to the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure. Instead, they are merely examples of apparatuses and methods consistent with some aspects of the present disclosure as detailed in the appended claims.
[0027] The implementations described in some embodiments of the present disclosure below do not represent all implementations consistent with the present disclosure. Instead, they are merely examples of apparatuses and methods consistent with some aspects of the present disclosure as detailed in the appended claims.
[0028] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the corresponding data protection regulations and policies of the country where the device is located, and with the authorization given by the owner of the corresponding device.
[0029] Currently, the algorithm with higher trajectory prediction accuracy is to generate a predicted trajectory by using a learning-based model, train a prediction model for outputting a future trajectory of a target vehicle, and iteratively update the model in a data-driven manner. This method has high accuracy, but requires high Graphics Processing Unit (GPU) resources. If this model is deployed in a vehicle, it will increase the overall cost of the vehicle, and the inference speed of the model is slow, making it difficult to make real-time prediction quickly.
[0030] Therefore, the present disclosure provides a vehicle trajectory prediction method and device, a vehicle, a storage medium and a product. During the driving of the ego vehicle, the historical trajectory information of the target vehicle around the ego vehicle, the historical trajectory information of the associated vehicle of the target vehicle, and the map information of the driving environment in which the target vehicle is located are acquired. Then, the interaction features between vehicles are determined based on the historical trajectory information of the target vehicle and the historical trajectory information of the associated vehicle. Subsequently, the interaction information between the vehicle and the map is determined according to the interaction features between the vehicles and the map information. Finally, the driving trajectory of the target vehicle is predicted according to at least the interaction features between the vehicle and the map. In this way, the influence of the driving trajectory of the associated vehicle and the map information on the driving trajectory of the target vehicle is considered, the accuracy of the predicted driving trajectory of the target vehicle is improved, and the driving safety of the ego vehicle is improved.
[0031] Figure 1 is a schematic diagram of a driving scene according to an example embodiment. As shown in Figure 1 In the driving scene, vehicle A is assumed to be the ego vehicle, and other vehicles B-F around vehicle A are target vehicles. The future driving trajectories of these target vehicles will affect the driving safety of the ego vehicle, i.e., vehicle A. If the ego vehicle can obtain the future driving trajectories of the target vehicles in advance, the driving trajectory of the ego vehicle can be adjusted in advance to improve driving safety.
[0032] Figure 2 is a flowchart of a vehicle trajectory prediction method according to an example embodiment. The method can be applied to an ego vehicle, such as a controller, a processor, etc. in the ego vehicle. As shown inFigure 2 As shown, the method can include the following steps.
[0033] In step S21, during the driving of the ego vehicle, the historical driving trajectory information of the target vehicle in the surrounding of the ego vehicle, the historical driving trajectory information of the associated vehicle of the target vehicle, and the map information of the driving environment in which the target vehicle is located are acquired.
[0034] In an embodiment, the target vehicle in the surrounding of the ego vehicle can be detected according to the sensors, such as cameras, radars, ultrasonic sensors, etc., provided on the ego vehicle. The target vehicle in the surrounding of the ego vehicle refers to the vehicle located within a preset range of the ego vehicle, for example, the vehicle located within the detection range of the sensors of the ego vehicle, or the vehicle located within the circular range with the ego vehicle as the center and a preset distance as the radius, etc.
[0035] The ego vehicle can detect the historical driving trajectory information of the target vehicle by using the sensors provided on the ego vehicle, wherein the historical driving trajectory information can be the position, speed, acceleration, heading angle, etc. of the target vehicle within the past 20s.
[0036] The associated vehicle of the target vehicle can include the vehicles of the adjacent lanes of the lane in which the target vehicle is located in the driving environment and the vehicles in the same lane as the target vehicle. In addition, the associated vehicle can include the ego vehicle or can not include the ego vehicle. In addition, in order to ensure that the ego vehicle can acquire the historical driving trajectory information of the associated vehicle, the associated vehicle is the vehicle located in the surrounding of the ego vehicle, that is, the vehicle that can be detected by the sensors of the ego vehicle.
[0037] For example, Figure 3 is a schematic diagram for determining the associated vehicle according to an exemplary embodiment. As shown in Figure 3 As shown, assuming that vehicle A is the ego vehicle, any vehicle located in the first area can be determined as the target vehicle. For the target vehicle B, other vehicles except the target vehicle B located in the first area and the second area at the same time can be determined as the associated vehicle of the target vehicle B, that is, the associated vehicle of the target vehicle B is vehicle D and vehicle F.
[0038] Similarly, the ego vehicle can detect the historical driving trajectory information of the associated vehicle by using the sensors provided on the ego vehicle, wherein the historical driving trajectory information can be the position, speed, acceleration, heading angle, etc. of the associated vehicle within the past 20s.
[0039] In addition, the map information of the driving environment in which the target vehicle is located can include lane line information, lane type information, traffic light information, and drivable area information within a preset range of the target vehicle. The lane line information can be the position of the lane line, the number of lanes, whether the lane line is a solid line, a dashed line, a double yellow line, etc. The lane type information can include a motor lane, a pedestrian lane, a two-way multi-lane, a fast lane, a slow lane, etc. The map information can be high-precision map information obtained according to the position of the ego vehicle, or can be perception map information generated according to the information perceived by the perception module on the ego vehicle, and the present disclosure does not make a specific limitation thereon.
[0040] For example, first, the ego vehicle can determine the map information of the driving environment in which the ego vehicle is located, then, the map information of the environment in which the target vehicle is located is intercepted from the map information according to the position of the target vehicle, and then, the driving trajectory of the target vehicle is predicted by using the map information of the environment in which the target vehicle is located. In this way, the data amount of the map information can be reduced, and the subsequent interaction workload can be reduced.
[0041] In step S22, the interaction feature between the vehicles is determined according to the historical driving trajectory information of the target vehicle and the historical driving trajectory information of the associated vehicle.
[0042] The interaction feature between the vehicles is used to represent the relationship between the historical driving trajectory information of the associated vehicle and the historical driving trajectory information of the target vehicle, or the influence of the historical driving trajectory information of the associated vehicle on the historical driving trajectory information of the target vehicle.
[0043] Since the associated vehicle is a vehicle around the target vehicle, the driving trajectory of the associated vehicle will affect the driving trajectory of the target vehicle. Therefore, in the present disclosure, in order to improve the accuracy of the predicted driving trajectory of the target vehicle, the influence of the historical driving trajectory information of the associated vehicle on the historical driving trajectory information of the target vehicle is determined in the process of predicting the driving trajectory, that is, the interaction feature between the vehicles is determined.
[0044] In step S23, the interaction feature between the vehicle and the map is determined according to the interaction feature information between the vehicles and the map information.
[0045] The interaction feature between the vehicle and the map is used to represent the relationship between the map information and the historical driving trajectory information of the associated vehicle and the historical driving trajectory information of the target vehicle, or the influence of the map information on the historical driving trajectory information of the associated vehicle and the historical driving trajectory information of the target vehicle.
[0046] As shown above, the map information can include lane line information, lane type information, traffic light information, and drivable area information within a preset range of the ego vehicle, and thus, the map information can also affect the decision of the target vehicle, i.e., the driving trajectory of the target vehicle, and thus, in the present disclosure, in order to further improve the accuracy of the predicted driving trajectory of the target vehicle, in the process of predicting the driving trajectory, the influence of the map information on the historical driving trajectory information of the associated vehicle and the target vehicle is determined, i.e., the interaction feature of the vehicle and the map is determined.
[0047] In step S24, the driving trajectory of the target vehicle is predicted according to at least the interaction feature of the vehicle and the map.
[0048] For example, the driving trajectory of the target vehicle within a preset time length in the future can be predicted. For example, the preset time length can be 6s, 8s, or 10s, etc., which is not limited in the present disclosure. Taking 8s as an example, the driving trajectory obtained by the coordinate points in the future 8s, assuming that the sampling frequency is 10Hz, a total of 80 coordinate points are obtained.
[0049] With the above technical solution, after obtaining the historical driving trajectory information of the target vehicle, the historical driving trajectory information of the associated vehicle of the target vehicle, and the map information, the interaction feature between the vehicles is determined, then the interaction feature of the vehicle and the map is determined according to the interaction feature information between the vehicles and the map information, and finally, the driving trajectory of the target vehicle is predicted according to at least the interaction feature of the vehicle and the map. In this way, when predicting the driving trajectory of the target vehicle, the mutual influence between the associated vehicle and the target vehicle and the influence of the map information on the associated vehicle and the target vehicle are considered, the reliability and accuracy of the predicted driving trajectory of the target vehicle are improved, and the driving safety of the ego vehicle is improved.
[0050] In the present disclosure, a neural network can be used to predict the driving trajectory of the target vehicle. For example, the neural network can include an input end, an encoding module, a decoding module, and an output end. The input end is used to input the historical driving trajectory information of the target vehicle, the historical driving trajectory information of the associated vehicle of the target vehicle, and the map information of the driving environment in which the target vehicle is located. The encoding module is used to determine the interaction feature between the vehicles according to the historical driving trajectory information of the target vehicle and the historical driving trajectory information of the associated vehicle, and to determine the interaction feature of the vehicle and the map according to the interaction feature information between the vehicles and the map information. The decoding module is used to predict the driving trajectory of the target vehicle according to at least the interaction feature of the vehicle and the map, for example, the decoding module decodes the interaction feature of the vehicle and the map to predict the driving trajectory of the target vehicle. The output end is used to output the predicted driving trajectory of the target vehicle.
[0051] It should be first understood that the neural network used above is a pre-trained network. In the training stage, first, a training sample is obtained, which includes sample historical driving trajectory information of a target vehicle around the ego vehicle, sample historical driving trajectory information of an associated vehicle of the target vehicle, sample map information of a driving environment where the target vehicle is located, and sample driving trajectory of the target vehicle. Then, the initial neural network is trained using the training sample to obtain the trained neural network. The specific training technique belongs to a relatively mature technology, and the present disclosure will not repeat it here.
[0052] Secondly, it should be understood that more training samples can be sorted out based on road test data for training the neural network. In this way, under the trend of data-driven, the training samples can be iteratively updated, thereby realizing the update of the neural network.
[0053] In an embodiment, step S22 determines the interaction feature between vehicles according to the historical driving trajectory information of the target vehicle and the historical driving trajectory information of the associated vehicle, which can include: processing the historical driving trajectory information of the target vehicle and the historical driving trajectory information of the associated vehicle using a first interaction encoding module to obtain the interaction feature between vehicles. Step S23 determines the interaction feature between the vehicle and the map according to the interaction feature information between the vehicles and the map information, which can include: processing the interaction feature information between the vehicles and the map information using a second interaction encoding module to obtain the interaction feature between the vehicle and the map, the second interaction encoding module being different from the first interaction encoding module.
[0054] For example, the encoding module can include the first interaction encoding module and the second interaction encoding module described above. Since the historical driving trajectory of the target vehicle is affected by the historical driving trajectory of the associated vehicle, in order to introduce the interaction information between the vehicles, the first interaction encoding module can be used to interactively process the historical driving trajectory information of the target vehicle and the historical driving trajectory information of the associated vehicle to obtain the interaction feature between the vehicles, which can represent the influence of the historical driving trajectory information of the associated vehicle on the historical driving trajectory information of the target vehicle. The first interaction encoding module can be a self-attention neural network, which learns the influence of the associated vehicle on the target vehicle using the self-attention neural network.
[0055] Similarly, the interaction feature information between vehicles and the map information are interactively processed by using a second interaction encoding module to obtain an interaction feature between the vehicle and the map, which can represent the influence of the map information on the historical driving trajectory information of the associated vehicle and the historical driving trajectory information of the target vehicle. The second interaction encoding module can also be a self-attention neural network, and the influence of the map information on the associated vehicle and the target vehicle is learned by using the self-attention neural network.
[0056] It should be understood that, in the present disclosure, before the interaction feature information between vehicles and the map information are interactively processed by using the second interaction encoding module, the map information can also be encoded by a map encoding module to obtain an encoded map encoding feature, and then the interaction feature information between vehicles and the map encoding feature are interactively processed by using the second interaction encoding module. For example, the map information can be encoded by the map encoding module to obtain the map encoding feature, and the map encoding module can be a fully connected neural network.
[0057] In this way, the interaction feature between vehicles and the interaction feature between the vehicle and the map are determined by using different interaction encoding modules to interact different information, which improves the accuracy of the determined interaction feature between vehicles and the interaction feature between the vehicle and the map, and further improves the accuracy of the predicted driving trajectory of the target vehicle.
[0058] The historical driving trajectory information obtained in step S21 can be the position, speed, acceleration, heading angle, etc. of the vehicle at each time detected by the sensor arranged on the ego vehicle, for example, assuming that the sensor detects 20 frames of driving trajectory information in the historical time period, and there is no interaction between each frame of driving trajectory information, therefore, in an embodiment, the obtained multiple frames of historical driving trajectory information can also be interacted in time to obtain more dimensional information. For example, the average speed is obtained according to the position in the multiple frames of historical driving trajectory information, the average acceleration is obtained according to the speed in the multiple frames of historical driving trajectory information, etc.
[0059] For example, the step S22 determines the interaction feature information between the vehicles according to the historical driving trajectory information of the target vehicle and the historical driving trajectory information of the associated vehicle, and can further include: encoding the historical driving trajectory information of the target vehicle and the historical driving trajectory information of the associated vehicle respectively by using a time sequence interaction encoding module to obtain historical driving trajectory encoding features of the target vehicle and historical driving trajectory encoding features of the associated vehicle. Correspondingly, the specific implementation of processing the historical driving trajectory information of the target vehicle and the historical driving trajectory information of the associated vehicle by using the first interaction encoding module to obtain the interaction feature between the vehicles is: processing the historical driving trajectory encoding features of the target vehicle and the historical driving trajectory encoding features of the associated vehicle by using the first interaction encoding module to obtain the interaction feature between the vehicles.
[0060] For example, the encoding module can further include a time sequence interaction encoding module, for example, a first time sequence interaction encoding module and a second time sequence interaction encoding module. The first time sequence interaction encoding module is used to encode the historical driving trajectory information of the target vehicle to obtain the historical driving trajectory encoding features of the target vehicle. The second time sequence interaction encoding module is used to encode the historical driving trajectory information of the associated vehicle to obtain the historical driving trajectory encoding features of the associated vehicle.
[0061] The first time sequence interaction encoding module and the second time sequence interaction encoding module can both be neural networks such as LSTM and conv1d.
[0062] Figure 4 is a schematic diagram of a neural network according to an example embodiment. As shown in Figure 4 The neural network includes an input end, an encoding module, a decoding module and an output end. The input end is used to input the historical driving trajectory information of the target vehicle, the historical driving trajectory information of the associated vehicle and the map information respectively. The decoding module includes a first time sequence interaction encoding module, a second time sequence interaction encoding module, a map encoding module, a first interaction encoding module and a second interaction encoding module. The historical driving trajectory information of the target vehicle is input into the first time sequence interaction encoding module to obtain the historical driving trajectory encoding features of the target vehicle, and the historical driving trajectory information of the associated vehicle is input into the second time sequence interaction encoding module to obtain the historical driving trajectory encoding features of the associated vehicle. At the same time, the map information is input into the map encoding module to obtain the map encoding features. Then, the historical driving trajectory encoding features of the target vehicle and the historical driving trajectory encoding features of the associated vehicle are input into the first interaction encoding module to obtain the interaction feature between the vehicles, and the interaction feature between the vehicles and the map encoding features are input into the second interaction encoding module to obtain the interaction feature between the vehicles and the map.
[0063] It should be understood that during the training of the neural network, each module included in the encoding module will be trained, and when the neural network training is completed, the training of each module is also completed.
[0064] The following describes a specific implementation of step S24 for predicting the target vehicle's driving trajectory based at least on the interaction characteristics between the vehicle and the map.
[0065] In one embodiment, step S24 predicts the target vehicle's driving trajectory based at least on the vehicle-map interaction features, which may include predicting the target vehicle's driving trajectory based on the vehicle-map interaction features, vehicle-to-vehicle interaction features, and encoded features of the target vehicle's historical driving trajectory. For example, the vehicle-map interaction features, vehicle-to-vehicle interaction features, and encoded features of the target vehicle's historical driving trajectory, encoded by the encoding module, are combined, and then the combined features are decoded to predict the target vehicle's driving trajectory.
[0066] For example, Figure 4 As shown, the decoding module can include a feature merging module and a prediction module. The feature merging module is used to combine the interaction features between the vehicle and the map, the interaction features between vehicles, and the encoded features of the target vehicle's historical driving trajectory. The combined features are then processed using a multi-layer perceptron (MLP) to generate merged features. The merged features are then input into the prediction module, which decodes the merged features to predict the target vehicle's driving trajectory.
[0067] In another embodiment, step S24 predicts the driving trajectory of the target vehicle based at least on the interaction characteristics between the vehicle and the map, and may include: using a decoding module to randomly generate a reference position point of the target vehicle, and using the decoding module to predict the driving trajectory of the target vehicle based on the reference position point and the interaction characteristics between the vehicle and the map.
[0068] In this embodiment, a reference location point refers to a location point to which the target vehicle may travel in the future. There may be one or more reference locations points, each of which may indicate a driving direction. Given that the target vehicle may travel in different directions, i.e., its driving trajectory is diverse, there may be multiple reference locations points, each of which may indicate different driving directions.
[0069] In one embodiment, there are multiple reference position points, and the multiple reference position points indicate different driving directions. The specific implementation method of using the decoding module to predict the driving trajectory of the target vehicle based on the reference position points and the interaction characteristics between the vehicle and the map can be: for each reference position point, using the decoding module to predict the driving trajectory of the target vehicle based on the reference position points and the interaction characteristics between the vehicle and the map, so as to predict multiple driving trajectories.
[0070] As Figure 4 shown, the decoding module can further include a reference point generation module for randomly generating reference position points, and an interaction feature input prediction module for the vehicle and the map, to obtain a driving trajectory of the target vehicle predicted by the prediction module.
[0071] The reference point generation module can be trained in the neural network training process. At the initial stage, the reference point generation module can randomly generate an initialized reference point. For example, an orthogonal initialization method can be used, so that although the initialization is random, the reference position points obtained by the initialization are orthogonal to each other, which can be approximately understood as the reference position points pointing to different directions in the physical space in the physical sense. This helps the neural network to learn a multi-modal predicted trajectory. Multi-modal refers to predicting trajectories of different intentions. If N reference position points are randomly generated, the encoding module can finally predict N driving trajectories of different driving intentions, such as a straight driving intention, a left turn driving intention, a right turn driving intention, an acceleration driving intention, a deceleration driving intention, and a U-turn driving intention.
[0072] In the test, it is found that orthogonal initialization can significantly improve the quality and diversity of multi-modal intention generation. In order to avoid the mode-callapse phenomenon (the mode-callapse phenomenon refers to that all predicted trajectories are biased towards an average intention), in the present disclosure, a mixed Gaussian distribution loss function GMMloss can be designed on the loss function of the neural network to ensure that multi-modal driving trajectories can be predicted. For example, assuming that N driving trajectories are subject to a two-dimensional Gaussian distribution, each driving trajectory has its own probability, and the superposition is the total trajectory probability distribution subject to by the target vehicle. In this way, the loss function of the neural network can allow the predicted trajectories to have different modes to the greatest extent, thereby reducing the mode-callapse phenomenon.
[0073] In this embodiment, when multiple driving trajectories are predicted, the possibility of the target vehicle driving according to each driving trajectory can also be predicted, so that the ego vehicle can know the possibility of the target vehicle for different driving intentions. Therefore, in one embodiment, the method can further include predicting the confidence of each driving trajectory by using the decoding module, and the confidence is used to represent the probability of the target vehicle driving according to the driving trajectory.
[0074] For example, as Figure 4 shown, the prediction module can predict the confidence of each driving trajectory in addition to predicting the driving trajectory of the target vehicle, to represent the probability of the target vehicle driving according to the driving trajectory.
[0075] It should be understood that in the embodiment in which the encoding module predicts the confidence of the driving trajectory, the confidence of the sample driving trajectory can also be included in the training sample when training the neural network, so that the neural network learns the ability to predict the confidence of the driving trajectory.
[0076] By using the above technical solution, the possibility of the target vehicle driving according to each driving trajectory is predicted, so that the ego vehicle can learn the possibility of the target vehicle for different driving intentions, and then the driving trajectory of the ego vehicle can be flexibly adjusted. In this way, the driving safety is improved.
[0077] It should be noted that in this embodiment, the driving trajectory of the target vehicle can also be predicted according to the reference position point, the interaction feature of the vehicle and the map, the interaction feature between vehicles, and the historical driving trajectory encoding feature of the target vehicle. The specific implementation can refer to the above solution, which will not be described here.
[0078] In the present disclosure, there can be multiple target vehicles, and the ego vehicle can predict the driving trajectory of each target vehicle in parallel.
[0079] Since the historical driving trajectory information of the target vehicle, the historical driving trajectory information of the associated vehicle, and the map information need to be input when predicting the driving trajectory of each target vehicle, the computational complexity of the neural network is strongly related to the number of target vehicles to be predicted. For example, the more the number of target vehicles, the more complex the calculation of the neural network. In the present disclosure, in order to reduce the computational complexity, a matrix calculation and parallel processing method can be used in engineering implementation. The Eigen library of c++ is used to make the calculation of predicting the driving trajectory of the target vehicle as matrix as possible, so as to greatly reduce the problem of linear growth of time consumption caused by the increase of the number of target vehicles. The driving trajectory of 64 target vehicles can be predicted simultaneously in real time per frame, and the time consumption at the vehicle end can be controlled within 30 ms.
[0080] In addition, the vehicle trajectory prediction method provided by the present disclosure is applicable to different scenes, such as high-speed and urban areas. The map information can be high-precision map information or perception map information. Therefore, by putting different scenes and different inputs into one neural network framework, one neural network can be used to process different scenes and different inputs, reducing the memory occupation and computational load of the vehicle end, etc.
[0081] Based on the same inventive concept, the present disclosure also provides a vehicle trajectory prediction device. Figure 5 is a block diagram of a vehicle trajectory prediction device according to an example embodiment. As shown in Figure 5 The vehicle trajectory prediction device 500 can include: The first obtaining module 501 is configured to obtain, in a driving process of the ego vehicle, historical driving trajectory information of a target vehicle in a surrounding of the ego vehicle, historical driving trajectory information of an associated vehicle of the target vehicle, and map information of a driving environment in which the target vehicle is located; The first determining module 502 is configured to determine an interaction feature between vehicles according to the historical driving trajectory information of the target vehicle and the historical driving trajectory information of the associated vehicle. The second determining module 503 is configured to determine an interaction feature between a vehicle and a map according to the interaction feature information between the vehicles and the map information. The first predicting module 504 is configured to predict a driving trajectory of the target vehicle according to at least the interaction feature between the vehicle and the map.
[0082] Optionally, the first determining module 502 can include: The first processing submodule is configured to process the historical driving trajectory information of the target vehicle and the historical driving trajectory information of the associated vehicle by using a first interaction encoding module to obtain the interaction feature between the vehicles. The second determining module 503 can include: The second processing submodule is configured to process the interaction feature information between the vehicles and the map information by using a second interaction encoding module to obtain the interaction feature between the vehicle and the map, the second interaction encoding module being different from the first interaction encoding module.
[0083] Optionally, the first determining module 502 can further include: The encoding submodule is configured to encode the historical driving trajectory information of the target vehicle and the historical driving trajectory information of the associated vehicle by using a time sequence interaction encoding module to obtain historical driving trajectory encoding features of the target vehicle and the associated vehicle. The first processing submodule is configured to process the historical driving trajectory encoding features of the target vehicle and the associated vehicle by using a first interaction encoding module to obtain the interaction feature between the vehicles.
[0084] Optionally, the first predicting module 504 can include: The generating module is configured to randomly generate a reference position point of the target vehicle by using a decoding module. The first predicting submodule is configured to predict the driving trajectory of the target vehicle according to the reference position point and the interaction feature between the vehicle and the map by using a decoding module.
[0085] Optionally, the reference position points are multiple, and the multiple reference position points indicate different driving directions; the first prediction submodule is configured to, for each reference position point, predict a driving trajectory of the target vehicle by using the decoding module to predict the driving trajectory of the target vehicle according to the reference position point and the interaction feature of the vehicle and the map, to obtain multiple driving trajectories.
[0086] Optionally, the vehicle trajectory prediction apparatus 500 can further include: The second prediction module is configured to predict a confidence of each driving trajectory by using the decoding module, the confidence being used to represent a probability of the target vehicle driving according to the driving trajectory.
[0087] Optionally, the first prediction module 504 can include: The second prediction submodule is configured to predict the driving trajectory of the target vehicle according to the interaction feature of the vehicle and the map, the interaction feature between the vehicles and the historical driving trajectory coding feature of the target vehicle.
[0088] Optionally, the target vehicles are multiple, and the driving trajectories of each target vehicle are predicted in parallel.
[0089] As to the apparatus in the above-mentioned embodiments, the specific manners in which various modules perform operations have been described in detail in the embodiments of the method, and thus will not be described in detail here.
[0090] The present disclosure also provides a computer-readable storage medium having stored thereon computer program instructions, which, when executed by a processor, implement the steps of the vehicle trajectory prediction method provided by the present disclosure.
[0091] The present disclosure also provides a vehicle, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the vehicle trajectory prediction method provided by the present disclosure.
[0092] Figure 6 is a block diagram of a vehicle according to an exemplary embodiment. For example, the vehicle 600 can be a hybrid vehicle, or a non-hybrid vehicle, an electric vehicle, a fuel cell vehicle, or other types of vehicles. The vehicle 600 can be an autonomous vehicle, a semi-autonomous vehicle, or a non-autonomous vehicle.
[0093] Referring to Figure 6In some embodiments, the vehicle 600 can include various subsystems, such as an infotainment system 610, a perception system 620, a decision control system 630, a drive system 640, and a computing platform 650. The vehicle 600 can include more or fewer subsystems, and each subsystem can include multiple components. In addition, each subsystem and each component of the vehicle 600 can be interconnected through wired or wireless means.
[0094] In some embodiments, the infotainment system 610 can include a communication system, an entertainment system, a navigation system, and the like.
[0095] The perception system 620 can include several sensors for sensing information of the environment surrounding the vehicle 600. For example, the perception system 620 can include a global positioning system (which can be a GPS system, a Beidou system, or other positioning system), an inertial measurement unit (IMU), a lidar, a millimeter wave radar, an ultrasonic radar, and a camera.
[0096] The decision control system 630 can include a computing system, a vehicle controller, a steering system, a throttle, and a braking system.
[0097] The drive system 640 can include components that provide power motion for the vehicle 600. In one embodiment, the drive system 640 can include an engine, an energy source, a transmission system, and wheels. The engine can be one or a combination of an internal combustion engine, an electric motor, an air compression engine, or the like. The engine can convert energy provided by the energy source into mechanical energy.
[0098] Some or all functions of the vehicle 600 are controlled by the computing platform 650. The computing platform 650 can include at least one processor 651 and a memory 652, and the processor 651 can execute instructions 653 stored in the memory 652.
[0099] The processor 651 can be any conventional processor, such as commercially available CPUs. The processor can also include a graphics processing unit (GPU), a field programmable gate array (FPGA), a system on chip (SOC), an application specific integrated circuit (ASIC), or a combination thereof.
[0100] Memory 652 can be implemented by any type of volatile or nonvolatile storage devices or a combination thereof such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic storage, flash memory, magnetic or optical disk.
[0101] In addition to instructions 653, memory 652 can store data such as road maps, route information, vehicle's position, direction, speed, etc. The data stored by memory 652 can be used by computing platform 650.
[0102] In embodiments of the present disclosure, processor 651 can execute instructions 653 to complete all or part of the steps of the vehicle trajectory prediction method described above.
[0103] In another exemplary embodiment, a computer program product is also provided, which contains a computer program capable of being executed by a programmable device, and the computer program has code portions for executing the vehicle trajectory prediction method described above when executed by the programmable device.
[0104] Further, the word "exemplary" is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as "exemplary" is not necessarily to be construed as advantageous over other aspects or designs. Rather, use of the word exemplary is intended to present concepts in a concrete manner. As used in this application, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or". That is, unless specified otherwise, or clear from context, "X employs A or B" is intended to mean any of the natural inclusive permutations. That is, if X employs A; X employs B; or X employs both A and B, then "X employs A or B" is satisfied under any of the foregoing instances. In addition, the articles "a" and "an" as used in this application and the appended claims should generally be construed to mean "one or more" unless specified otherwise or clear from context to be directed to a singular form. Thus, use of the articles in this application and the following claims is not limiting.
[0105] Likewise, although the present disclosure has been described and illustrated with respect to one or more implementations, equivalent alterations and modifications will become apparent to those skilled in the art that do not depart from the true spirit and scope of the disclosure. The present disclosure includes all such modifications and alterations and is limited only by the scope of the following claims. In particular regard to the various functions performed by the above described components (e.g., elements, resources, etc.), the terms (including a reference to a "means") used to describe such components are intended to correspond, unless otherwise indicated, to any component which performs the described function (e.g., a functional equivalent), even though not structurally equivalent to the disclosed structure. In addition, although a particular feature of the disclosure can have been disclosed with respect to only one of several implementations, other implementations can include the particular feature. For example, the disclosure can be implemented with respect to other implementations that incorporate the particular feature, and that implement other features as disclosed herein, and each of the various implementations have a reasonable expectation of support. Furthermore, to the extent that the terms "includes", "including", "has", "have", "having", or variants thereof are used in either the detailed description or the claims, such terms are intended to be inclusive in a manner similar to the term "comprising".
[0106] Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the features of the disclosure disclosed herein. It is intended that the present disclosure be considered as including any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such steps, compositions, components, and / or elements known in the art to be appropriate. The specification and examples given are considered exemplary only and the true scope and spirit of the disclosure is indicated by the appended claims.
[0107] It is to be understood that the present disclosure is not limited to the precise construction described and illustrated above and that various modifications and changes can be made therein without departing from the scope of the present disclosure. The scope of the present disclosure is limited only by the claims appended hereto.
Claims
1. A vehicle trajectory prediction method, characterized by, The method comprises the following steps: During driving of the ego vehicle, historical driving trajectory information of a target vehicle around the ego vehicle, historical driving trajectory information of a related vehicle of the target vehicle, and map information of a driving environment in which the target vehicle is located are acquired; Interaction features between vehicles are determined according to the historical driving trajectory information of the target vehicle and the historical driving trajectory information of the related vehicle; Interaction features between the vehicle and the map are determined according to the interaction feature information between the vehicles and the map information; The driving trajectory of the target vehicle is predicted according to at least the interaction features between the vehicle and the map.
2. The method of claim 1, wherein, The determination of the interaction features between the vehicles according to the historical driving trajectory information of the target vehicle and the historical driving trajectory information of the related vehicle comprises: The historical driving trajectory information of the target vehicle and the historical driving trajectory information of the related vehicle are processed by a first interaction encoding module to obtain the interaction features between the vehicles. The determination of the interaction features between the vehicle and the map according to the interaction feature information between the vehicles and the map information comprises: The interaction feature information between the vehicles and the map information are processed by a second interaction encoding module to obtain the interaction features between the vehicle and the map, and the second interaction encoding module is different from the first interaction encoding module.
3. The method of claim 2, wherein, The determination of the interaction feature information between the vehicles according to the historical driving trajectory information of the target vehicle and the historical driving trajectory information of the related vehicle further comprises: The historical driving trajectory information of the target vehicle and the historical driving trajectory information of the related vehicle are respectively encoded by a time sequence interaction encoding module to obtain the historical driving trajectory encoding features of the target vehicle and the historical driving trajectory encoding features of the related vehicle. The processing of the historical driving trajectory information of the target vehicle and the historical driving trajectory information of the related vehicle by the first interaction encoding module to obtain the interaction features between the vehicles comprises: The historical driving trajectory encoding features of the target vehicle and the historical driving trajectory encoding features of the related vehicle are processed by the first interaction encoding module to obtain the interaction features between the vehicles.
4. The method according to any one of claims 1 to 3, characterized in that, The prediction of the driving trajectory of the target vehicle according to at least the interaction features between the vehicle and the map comprises: A reference position point of the target vehicle is randomly generated by a decoding module; The driving trajectory of the target vehicle is predicted by the decoding module according to the reference position point and the interaction features between the vehicle and the map.
5. The method of claim 4, wherein, The reference position point is multiple, and the driving directions indicated by the multiple reference position points are different; the prediction of the driving trajectory of the target vehicle by the decoding module according to the reference position point and the interaction features between the vehicle and the map comprises: For each reference position point, the driving trajectory of the target vehicle is predicted by the decoding module according to the reference position point and the interaction features between the vehicle and the map, so as to obtain multiple driving trajectories.
6. The method of claim 5, wherein, The method further comprises: The confidence of each driving trajectory is predicted by the decoding module, and the confidence is used to represent the probability of driving of the target vehicle according to the driving trajectory.
7. The method of claim 3, wherein, The at least vehicle-map interaction feature is used to predict the driving trajectory of the target vehicle, comprising: The driving trajectory of the target vehicle is predicted according to the vehicle-map interaction feature, the inter-vehicle interaction feature and the historical driving trajectory coding feature of the target vehicle.
8. The method of claim 1, wherein, The target vehicle is multiple, and the driving trajectory of each target vehicle is predicted in parallel.
9. A vehicle trajectory prediction device characterized by comprising: Comprising: The first acquisition module is configured to acquire the historical driving trajectory information of the target vehicle and the associated vehicle of the target vehicle, and the map information of the driving environment where the target vehicle is located during the driving process of the ego vehicle; The first determination module is configured to determine the inter-vehicle interaction feature according to the historical driving trajectory information of the target vehicle and the historical driving trajectory information of the associated vehicle; The second determination module is configured to determine the vehicle-map interaction feature according to the inter-vehicle interaction feature information and the map information: The first prediction module is configured to predict the driving trajectory of the target vehicle at least according to the vehicle-map interaction feature.
10. A vehicle characterized by comprising: Comprising: A processor; A memory for storing processor-executable instructions; The processor is configured to execute the instructions to implement the vehicle trajectory prediction method according to any one of claims 1-8.
11. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the vehicle trajectory prediction method according to any one of claims 1-8.
12. A computer program product, characterised in that, The computer program is executed by the processor to implement the vehicle trajectory prediction method according to any one of claims 1-8.