A vehicle trajectory prediction method, device and computer storage medium

Abstract

The application relates to the technical field of vehicle control, in particular to a vehicle trajectory prediction method and device and a computer storage medium, the method comprising the following steps: obtaining historical driving trajectories corresponding to a target vehicle and surrounding vehicles respectively, and driving road information corresponding to the target vehicle; performing driving feature extraction on the historical driving trajectories respectively to obtain vehicle driving features; performing road feature extraction on the driving road information to obtain road features; performing feature fusion based on the vehicle driving features and the road features to obtain first fusion features; performing trajectory prediction based on the driving road information, the driving features, the road features and the first fusion features to obtain a trajectory prediction result of the target vehicle; and performing feature fusion on the vehicle driving features and the road features to obtain the correlation between the vehicle driving trajectory and the driving road information, so that the noise resistance of the vehicle trajectory prediction is improved, and the accuracy of the trajectory prediction is improved.

Description

Technical Field

[0001] This application relates to the field of vehicle control technology, and in particular to a vehicle trajectory prediction method, device, and computer storage medium. Background Technology

[0002] In autonomous driving, it is necessary to first predict the trajectory of the vehicle so that the vehicle can perform autonomous driving control based on the predicted trajectory. In existing technologies, common methods for predicting vehicle trajectory are: trajectory prediction based on motion constraint models or trajectory prediction based on deep learning.

[0003] Among them, the trajectory prediction method based on motion constraint model is highly dependent on the accuracy of vehicle data acquisition; the trajectory prediction method based on deep learning does not consider the impact of the environment on vehicle driving, resulting in poor noise resistance of vehicle trajectory prediction and insufficient accuracy of trajectory prediction results. Summary of the Invention

[0004] To address the aforementioned problems in the prior art, the purpose of this application is to fuse vehicle driving features and road features to obtain the correlation between vehicle driving trajectory and driving road information, thereby improving the prediction accuracy of vehicle trajectory and enhancing the noise resistance of vehicle trajectory prediction.

[0005] To address the aforementioned problems, this application provides a vehicle trajectory prediction method, comprising:

[0006] Obtain the historical driving trajectories of the target vehicle and surrounding vehicles, as well as the driving road information around the target vehicle;

[0007] Driving features are extracted from the historical driving trajectories of the target vehicle and the surrounding vehicles respectively to obtain the vehicle driving features of the target vehicle and the surrounding vehicles respectively.

[0008] The driving road information is used to extract road features to obtain road features;

[0009] Based on the vehicle driving characteristics of the target vehicle and the surrounding vehicles, as well as the road characteristics, feature fusion is performed to obtain a first fused feature; the first fused feature represents the correlation between the vehicle driving trajectory and the driving road information.

[0010] Based on the driving road information, the driving characteristics of the target vehicle and the surrounding vehicles, the road features, and the first fused features, trajectory prediction is performed to obtain the trajectory prediction result of the target vehicle.

[0011] In this embodiment of the application, the step of extracting road features from the driving road information to obtain road features includes:

[0012] Lane information is sampled from the driving road information at a first sampling interval to obtain first lane information;

[0013] The road features are obtained by extracting features from the first lane information.

[0014] In this embodiment of the application, the step of performing trajectory prediction based on the driving road information, the vehicle driving characteristics corresponding to the target vehicle and the surrounding vehicles, the road features, and the first fused features to obtain the trajectory prediction result of the target vehicle includes:

[0015] Lane information is sampled from the driving road information at a second sampling interval to obtain second lane information; the first sampling interval is greater than the second sampling interval.

[0016] Based on the second lane information, the vehicle driving characteristics corresponding to the target vehicle and the surrounding vehicles, the road features, and the first fused features, trajectory prediction is performed to obtain the trajectory prediction result of the target vehicle.

[0017] In this embodiment of the application, the second lane information includes multiple lane collection points. The trajectory prediction of the target vehicle, based on the second collected information, the vehicle driving characteristics corresponding to the target vehicle and the surrounding vehicles, the road features, and the first fused features, includes:

[0018] Based on the second collected information, the vehicle driving characteristics corresponding to the target vehicle and the surrounding vehicles, the road characteristics, and the first fused features, probability prediction is performed on the multiple lane collection points to obtain the regional center probability corresponding to each of the multiple lane collection points.

[0019] Based on the regional central probabilities corresponding to the multiple lane collection points, target collection points are determined; the target collection points are multiple lane collection points selected sequentially from front to back after sorting the multiple lane collection points in descending order of their respective regional central probabilities.

[0020] The area surrounding the target acquisition point is sampled to obtain multiple associated acquisition points;

[0021] Based on the multiple associated collection points, multiple target endpoints are determined; the distance between the multiple target endpoints and the nearest associated collection point is less than the second sampling interval.

[0022] Based on the multiple target endpoints, the vehicle driving characteristics corresponding to the target vehicle and the surrounding vehicles, and the road characteristics, trajectory prediction is performed to obtain multiple predicted trajectories corresponding to the target vehicle.

[0023] In this embodiment of the application, determining multiple target endpoints based on the multiple associated collection points includes:

[0024] Based on the multiple associated collection points, the vehicle driving features corresponding to the target vehicle and the surrounding vehicles, the road features, and the first fusion feature, the scores of the multiple associated collection points are predicted to obtain the regional boundary scores corresponding to each of the multiple associated collection points.

[0025] Based on the regional boundary scores corresponding to each of the multiple associated collection points, target associated points are determined; the target associated points are multiple associated collection points selected sequentially from front to back after sorting the multiple associated collection points in descending order based on their respective regional boundary scores.

[0026] Based on the target association points and the scores corresponding to the target association points, target endpoints are predicted, and multiple target endpoints are determined.

[0027] In this embodiment of the application, the step of predicting the target endpoint based on the target association point and the score corresponding to the target association point, and determining multiple target endpoints, includes:

[0028] Based on the target association points and the scores corresponding to the target association points, a high-dimensional mapping is performed to obtain multiple association feature matrices;

[0029] Elements at the same position in the plurality of associated feature matrices are filtered to obtain a first feature matrix; the element at each position in the first feature matrix is ​​the maximum value of the corresponding element in the plurality of associated feature matrices.

[0030] The elements at the same position in the multiple correlation feature matrices are weighted and averaged to obtain the second feature matrix;

[0031] The first feature matrix and the second feature matrix are merged to obtain the third feature matrix;

[0032] The third feature matrix is ​​input into the endpoint prediction model to predict the endpoint, thereby obtaining the multiple target endpoints.

[0033] In this embodiment of the application, the step of predicting trajectories based on the multiple target endpoints, the vehicle driving characteristics corresponding to the target vehicle and the surrounding vehicles, and the road characteristics to obtain multiple predicted trajectories corresponding to the target vehicle includes:

[0034] Perform high-dimensional mapping on the multiple target endpoints to obtain endpoint mapping information corresponding to each of the multiple target endpoints;

[0035] Feature extraction is performed on the endpoint mapping information corresponding to each of the multiple target endpoints to obtain multiple endpoint features;

[0036] Based on the road features and the multiple endpoint features, feature fusion is performed to obtain the second fused feature;

[0037] The endpoint feature, the second fusion feature, and the vehicle driving feature corresponding to the target vehicle are input into the trajectory prediction model to perform trajectory prediction, thereby obtaining multiple predicted trajectories corresponding to the target vehicle.

[0038] On the other hand, this application also provides a vehicle trajectory prediction device, comprising:

[0039] The acquisition module is used to acquire the historical driving trajectories of the target vehicle and surrounding vehicles, as well as the driving road information around the target vehicle.

[0040] The first feature extraction module is used to extract driving features from the historical driving trajectories of the target vehicle and the surrounding vehicles respectively, so as to obtain the vehicle driving features corresponding to the target vehicle and the surrounding vehicles respectively.

[0041] The second feature extraction module is used to extract road features based on the driving road information to obtain road features;

[0042] The first fusion module is used to perform feature fusion based on the vehicle driving characteristics corresponding to the target vehicle and the surrounding vehicles, as well as the road features, to obtain a first fused feature; the first fused feature represents the correlation between the vehicle driving trajectory and the driving road information.

[0043] The trajectory prediction module is used to perform trajectory prediction based on the driving road information, the vehicle driving characteristics corresponding to the target vehicle and the surrounding vehicles, the road characteristics, and the first fused features, to obtain the trajectory prediction result of the target vehicle.

[0044] On the other hand, this application also provides an electronic device, the device including a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the vehicle trajectory prediction method as described above.

[0045] On the other hand, this application also provides a computer storage medium storing at least one instruction or at least one program, wherein the at least one instruction or the at least one program is loaded and executed by a processor to implement the vehicle trajectory prediction method described above.

[0046] Due to the above technical solution, the vehicle trajectory prediction method described in this application has the following beneficial effects:

[0047] By extracting the driving features of the target vehicle and surrounding vehicles, and extracting road features, and then fusing the driving features of the target vehicle and surrounding vehicles with the road features to obtain the first fused feature, trajectory prediction is performed based on the driving features of the target vehicle and surrounding vehicles, the road features, and the first fused feature. This fully considers the vehicle's own driving factors, road driving factors, and the influence of the road on vehicle driving, thereby improving the noise resistance of vehicle trajectory prediction and thus improving the accuracy of trajectory prediction. Attached Figure Description

[0048] To more clearly illustrate the technical solutions of this application, the accompanying drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0049] Figure 1 This is a schematic flowchart of a vehicle trajectory prediction method provided in an embodiment of this application;

[0050] Figure 2 This is a schematic diagram of the road feature extraction process in a vehicle trajectory prediction method provided in this application embodiment;

[0051] Figure 3 This is a schematic diagram of the trajectory prediction process in a vehicle trajectory prediction method provided in this application embodiment. Figure 1 ;

[0052] Figure 4 This is a schematic diagram of the trajectory prediction process in a vehicle trajectory prediction method provided in this application embodiment. Figure 2 ;

[0053] Figure 5 This is a schematic diagram of the target endpoint determination process in a vehicle trajectory prediction method provided in this application embodiment;

[0054] Figure 6 This is a schematic diagram of the target endpoint prediction process in a vehicle trajectory prediction method provided in this application embodiment;

[0055] Figure 7 This is a schematic diagram of the trajectory prediction process in a vehicle trajectory prediction method provided in an embodiment of this application;

[0056] Figure 8 This is a schematic diagram of a vehicle trajectory prediction device provided in an embodiment of this application;

[0057] Figure 9 This is a hardware structure block diagram of a vehicle trajectory prediction method provided in an embodiment of this application. Detailed Implementation

[0058] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0059] The term "an embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of this application. In the description of this application, it should be understood that the terms "upper," "lower," "left," "right," "top," "bottom," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. Moreover, the terms "first," "second," etc., are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein.

[0060] Combination Figure 1 This application introduces a vehicle trajectory prediction method provided by an embodiment, the method comprising:

[0061] S1001. Obtain the historical driving trajectory of the target vehicle and surrounding vehicles, as well as the driving road information around the target vehicle.

[0062] The target vehicle refers to the vehicle whose trajectory needs to be predicted. Surrounding vehicles refer to vehicles that continuously appear around the target vehicle within a preset time period. Historical driving trajectory refers to the driving trajectory before the current moment. Preferably, the historical driving trajectory consists of multiple driving trajectory points. Driving road information is the road information within a preset range centered on the current location of the target vehicle. Driving road information can refer to high-precision map data. Specifically, driving road information includes information such as road center lines and traffic lights.

[0063] In a specific embodiment of this application, the driving positions of the target vehicle and neighboring vehicles are collected based on a preset time interval and a preset time period to obtain the historical driving trajectories of the target vehicle and neighboring vehicles respectively; the preset time interval is less than 0.3s and the preset time period is less than 6s; specifically, the preset time interval can be 0.1s and the preset time period can be 2s, that is, the historical trajectory of the target vehicle is collected to obtain 20 historical trajectory points. Simultaneous data collection is performed on neighboring vehicles. If the historical trajectory points corresponding to a neighboring vehicle are greater than or equal to five, the neighboring vehicle is identified as a surrounding vehicle. In other embodiments of this application, the preset time interval can be 0.15s, the preset time period can be 3s, and the historical trajectory points of surrounding vehicles can be greater than or equal to six. By collecting data in a shorter time, a longer predicted trajectory can be predicted from a shorter historical trajectory, thus improving the practicality of trajectory prediction.

[0064] In a specific embodiment of this application, a planar coordinate system is established with the current position of the target vehicle as the origin, thereby obtaining the planar coordinates corresponding to each historical trajectory point.

[0065] In this embodiment of the application, the driving road information can be based on high-precision map data stored locally on the vehicle, or it can be high-precision map data obtained from the cloud.

[0066] S1002. Extract driving features from the historical driving trajectories of the target vehicle and surrounding vehicles respectively to obtain the driving features of the target vehicle and surrounding vehicles respectively; the driving features represent the regularity of the historical driving trajectory; specifically, the driving feature extraction can be performed using a cross-attention mechanism or Euclidean distance.

[0067] In a specific embodiment of this application, the formula for the attention mechanism is as follows:

[0068]

[0069] Where Q is the first matrix, K is the second matrix, and K T Let V be the transpose of the second matrix, and let d be the third matrix. kThe dimension of the representation vector, Attention is the result of cross-attention, and softmax is the normalization function. Specifically:

[0070] Q = x1 × W1 (2)

[0071] K = x² × W² (3)

[0072] V=x3×W3 (4)

[0073] Where x1 is the first input value, x2 is the second input value, x3 is the third input value, and W1, W2, and W3 are matrix parameters obtained through intelligent learning.

[0074] In a specific embodiment of this application, when extracting features from historical trajectories, any two trajectory points in the historical trajectory are used as inputs to the cross-attention mechanism, specifically x1, x2 = x3, thereby obtaining the node features corresponding to the two adjacent trajectory points;

[0075] Preferably, the feature value of the largest node feature in each historical trajectory is obtained as the vehicle driving feature corresponding to the target vehicle and surrounding vehicles respectively:

[0076]

[0077] Among them, F sg This refers to the vehicle's driving characteristics. This refers to the node features corresponding to the i-th target at time a. It refers to the node features corresponding to the i-th target at time b, N is the total number of target vehicles and surrounding vehicles, and T is the number of trajectory points corresponding to the maximum time of historical trajectory collection.

[0078] In this embodiment, a cross-attention mechanism is adopted to fully consider the mutual influence between each historical trajectory point of the vehicle, thereby improving the accuracy of vehicle driving feature extraction and thus improving the accuracy of vehicle trajectory prediction.

[0079] In another specific embodiment of this application, the reciprocal of the Euclidean distance can also be used as the influence between two historical trajectory points, so as to select the smaller value as the vehicle driving characteristic; specifically, the smaller the reciprocal of the Euclidean distance between two historical trajectory points, the more distant the relationship between them; the closer the two historical trajectory points are, the closer the relationship between them.

[0080] S1003. Extract road features from the driving road information to obtain road features; road features represent the regular characteristics of driving road information; specifically, driving feature extraction can be performed using a cross-attention mechanism.

[0081] S1004. Based on the vehicle driving characteristics of the target vehicle and surrounding vehicles, as well as the road characteristics, feature fusion is performed to obtain the first fused feature; the first fused feature represents the correlation between the vehicle driving trajectory and the driving road information; preferably, the feature fusion process can be performed using a cross-attention mechanism; the input of the cross-attention mechanism is any two feature values.

[0082] S1005. Based on the driving road information, the driving characteristics of the target vehicle and the surrounding vehicles, the road characteristics, and the first fusion features, trajectory prediction is performed to obtain the trajectory prediction result of the target vehicle.

[0083] In this embodiment, by extracting the vehicle driving features corresponding to the target vehicle and surrounding vehicles, and extracting road features, and then fusing the vehicle driving features corresponding to the target vehicle and surrounding vehicles, as well as the road features, to obtain a first fused feature, trajectory prediction is performed based on the vehicle driving features corresponding to the target vehicle and surrounding vehicles, the road features, and the first fused feature. This fully considers the vehicle's own driving factors, road driving factors, and the influence of the road on vehicle driving, thereby improving the noise resistance of vehicle trajectory prediction and thus improving the accuracy of trajectory prediction.

[0084] refer to Figure 2 In this embodiment of the application, S1003 includes:

[0085] S2001. Sampling lane information of the driving road information at a first sampling interval to obtain first lane information; the first sampling interval refers to the distance between adjacent collection points; preferably, the first sampling interval can be 5m or 10m; for example, collecting lane information of the driving road information at 5m interval, wherein the driving road information includes data such as the road centerline and traffic lights, and the map accuracy corresponding to the driving road information is greater than the map accuracy corresponding to the first lane information; the first lane information refers to the information corresponding to the target vehicle's driving lane, preferably, it can be multiple collection points on the centerline of the target vehicle's driving lane, or multiple collection points on the boundary of the target vehicle's driving lane.

[0086] S2002. Extract features from the information of the first lane to obtain road features.

[0087] In a specific embodiment of this application, the first lane information includes multiple lane collection points. The multiple lane collection points are divided into regions to obtain multiple sets of lane collection points. Cross-attention calculation is performed on the multiple sets of lane collection points to obtain the regional features corresponding to the multiple sets of lane collection points. The road features are the regional features corresponding to the multiple sets of lane collection points.

[0088] In this embodiment of the application, road features are obtained by collecting lane information of driving road information at a first sampling interval and extracting features from the collected first lane information, thereby reducing the amount of data processing for feature extraction and improving the feature extraction rate.

[0089] refer to Figure 3 In this embodiment of the application, S1005 includes:

[0090] S3001. Sampling lane information of the driving road information at a second sampling interval to obtain second lane information; the first sampling interval is greater than the second sampling interval; the second sampling interval refers to the distance between adjacent sampling points; preferably, the second sampling interval can be 2m or 3m; for example, lane information is collected from the driving road information at a distance of 2m; the second lane information refers to the information corresponding to the target vehicle's driving lane, preferably, it can be multiple sampling points on the center line of the target vehicle's driving lane or multiple sampling points on the boundary of the target vehicle's driving lane; when collecting lane information from the same driving road information, the larger the sampling interval, the less data is collected; when the first sampling interval is greater than the second sampling interval, the collected data corresponding to the second lane information is greater than the collected data corresponding to the first lane information.

[0091] S3002. Based on the second lane information, the vehicle driving characteristics, road characteristics, and first fusion features corresponding to the target vehicle and surrounding vehicles, trajectory prediction is performed to obtain the trajectory prediction result of the target vehicle.

[0092] In this embodiment, by collecting lane information of the driving road at a second sampling interval and performing trajectory prediction based on the collected second lane information, the amount of data processing for trajectory prediction is reduced and the trajectory prediction rate is improved compared with directly using driving road information for trajectory prediction; compared with using first lane information for trajectory prediction, the amount of data processed for trajectory prediction is increased, thereby improving the reliability and accuracy of trajectory prediction.

[0093] refer to Figure 4 In this embodiment of the application, the second lane information includes multiple lane collection points, and S3002 includes:

[0094] S4001. Based on the second collected information, the vehicle driving characteristics and road characteristics corresponding to the target vehicle and surrounding vehicles, and the first fused features, the probability prediction of multiple lane collection points is performed to obtain the regional center probability corresponding to each of the multiple lane collection points; the regional center probability refers to the probability that multiple collection points are on the central axis of the region where the target endpoint is located.

[0095] In a specific embodiment of this application, the second collected information, the vehicle driving characteristics and road characteristics corresponding to the target vehicle and surrounding vehicles, and the first fusion characteristics are input into the probability prediction model for probability prediction to obtain the regional center probability corresponding to each of the multiple lane collection points. The probability prediction model is obtained by intelligent recognition training based on sample sampling information, multiple sample vehicle driving characteristics, sample road characteristics, sample fusion characteristics, and sample probabilities. The probability prediction model can be a multilayer perceptron (MLP). By using a multilayer perceptron, the data processing time is reduced and the efficiency of trajectory prediction is improved.

[0096] S4002. Based on the regional center probability corresponding to each of the multiple lane collection points, determine the target collection point; the target collection point is the multiple lane collection points selected sequentially from front to back after sorting the multiple lane collection points in descending order of their respective regional center probabilities.

[0097] S4003. Sample the area surrounding the target collection point to obtain multiple associated collection points;

[0098] Specifically, sampling the area surrounding the target sampling point can be performed by sampling multiple points at a preset sampling interval centered on the target sampling point. Specifically, the preset sampling interval is less than the second sampling interval. Preferably, the preset sampling interval can be 0.8m or 1m. Each target sampling point corresponds to a preset number of associated sampling points. Preferably, the preset number can be 8 or 10. For example, within a 0.8m range around each target sampling point, 8 points are evenly collected as associated sampling points.

[0099] In a specific embodiment of this application, it is necessary to perform deduplication on the collected multiple associated points to reduce the amount of data processing and thereby improve the trajectory prediction rate.

[0100] S4004. Based on multiple associated acquisition points, determine multiple target endpoints; the distance between the multiple target endpoints and the nearest associated acquisition point is less than the second sampling interval; that is, the target endpoints are located within the area bounded by the second sampling interval area of ​​the multiple associated acquisition points.

[0101] S4005. Based on multiple target endpoints, the driving characteristics of the target vehicle and surrounding vehicles, and road features, trajectory prediction is performed to obtain multiple predicted trajectories corresponding to the target vehicle.

[0102] In this embodiment of the application, the target endpoint is first solved, and trajectory prediction is performed based on the target endpoint. By predicting the driving intention of the target vehicle in advance, the accuracy and reliability of trajectory prediction are improved.

[0103] In this embodiment, by predicting multiple predicted trajectories, it is possible to have multiple routes to choose from in subsequent autonomous driving planning, thereby improving the intelligence of autonomous driving planning.

[0104] refer to Figure 5 In this embodiment of the application, S4004 includes:

[0105] S5001. Based on multiple associated collection points, the vehicle driving features, road features, and first fusion features corresponding to the target vehicle and surrounding vehicles, score prediction is performed on the multiple associated collection points to obtain the regional boundary scores corresponding to each of the multiple associated collection points; the regional boundary score refers to the score of the target endpoint located within the region when the second sampling interval region of this associated point is taken as the boundary.

[0106] In a specific embodiment of this application, multiple associated collection points, the vehicle driving features, road features, and first fusion features corresponding to the target vehicle and surrounding vehicles are input into the scoring prediction model to obtain the area boundary scores corresponding to each of the multiple associated collection points. The scoring prediction model is obtained by intelligent recognition training based on multiple sample associated collection points, multiple sample vehicle driving features, sample road features, sample fusion features, and sample scores. The scoring prediction model can be a multilayer perceptron (MLP). By using a multilayer perceptron, the data processing time is reduced and the efficiency of trajectory prediction is improved.

[0107] S5002. Based on the regional boundary scores corresponding to each of the multiple associated collection points, determine the target associated point; the target associated point is a number of associated collection points selected sequentially from front to back after sorting the multiple associated collection points in descending order of their respective regional boundary scores.

[0108] S5003. Based on the target association points and the scores corresponding to the target association points, predict the target endpoint and determine multiple target endpoints; the target endpoint is the endpoint of the trajectory prediction.

[0109] In this embodiment of the application, by predicting the scores of multiple associated collection points, filtering based on the regional boundary scores corresponding to each of the multiple associated collection points, and then predicting the target endpoint based on the filtered target associated points, the amount of data processing for target endpoint prediction can be reduced, thereby improving the efficiency of target endpoint prediction.

[0110] refer to Figure 6 In this embodiment of the application, S5003 includes:

[0111] S6001. Based on the target association points and the scores corresponding to the target association points, perform high-dimensional mapping to obtain multiple association feature matrices; multiple association matrices refer to the association feature matrices of multiple association collection points, which are used to characterize the features of the corresponding association collection points.

[0112] In a specific embodiment of this application, the target associated point is located on the coordinate plane of the component centered on the target vehicle. Based on the coordinate axes of the target associated point and the score corresponding to the target associated point, the three-dimensional vector (x, y, z) corresponding to the target associated point can be obtained, where x and y represent the position of the target associated point in the coordinate plane, and z represents the score corresponding to the target associated point. In this application, the target associated point can be mapped to a 12-dimensional space or a 9-dimensional space, which is not limited here.

[0113] S6002. Filter the elements at the same position in the multiple associated feature matrices to obtain the first feature matrix; the element at each position in the first feature matrix is ​​the maximum value of the corresponding element in the multiple associated feature matrices.

[0114] S6003. Perform a weighted average on the elements at the same position in the multiple correlation feature matrices to obtain the second feature matrix; the elements at each position in the second feature matrix are the average of the elements at the corresponding positions in the multiple correlation matrices.

[0115] S6004. Merge the first and second characteristic matrices to obtain the third characteristic matrix. Matrix merging refers to combining the dimensions of the first and second characteristic matrices to obtain a higher-dimensional matrix. For example, if the first characteristic matrix is ​​A and the second characteristic matrix is ​​B, then the third characteristic matrix is...

[0116] S6005. Input the third feature matrix into the endpoint prediction model to predict the endpoint and obtain multiple target endpoints. The endpoint prediction model is obtained by intelligent recognition training based on the sample feature matrix and the sample endpoints.

[0117] In a specific embodiment of this application, the output of the endpoint prediction model includes the location information of multiple target endpoints and the score information corresponding to the multiple target endpoints. Specifically, the endpoint prediction model outputs multiple sets of target endpoints and the score information corresponding to each set of target endpoints based on the expected number of trajectories. Each set of target endpoints includes multiple target endpoints. For example, if the expected number of trajectories is three, then the endpoint prediction model outputs the score information corresponding to the three sets of target endpoints. The set with the highest score among the three sets of target endpoints is selected, and multiple target endpoints in it are determined as the multiple target endpoints output.

[0118] In this embodiment, multiple association feature matrices are obtained by performing high-dimensional mapping on the target association points and the scores corresponding to the target association points. A third feature matrix is ​​obtained based on the multiple association feature matrices. The third feature matrix is ​​used as the input of the endpoint prediction model to perform endpoint prediction. This not only expands the feature capacity of the input data for endpoint prediction, but also adopts the endpoint prediction model, thereby improving the prediction accuracy of the target endpoint.

[0119] refer to Figure 7 In this embodiment of the application, S4005 includes:

[0120] S7001. Perform high-dimensional mapping on multiple target endpoints to obtain endpoint mapping information corresponding to each of the multiple target endpoints; the endpoint mapping information is a high-dimensional expression of the target endpoint, specifically calculated by a multilayer perceptron.

[0121] In a specific embodiment of this application, the scores of multiple target endpoints can be determined based on the positions of multiple target endpoints and the scores of neighboring target associated points; specifically, the scores of neighboring target associated points are the scores of the corresponding target endpoint positions; therefore, a three-dimensional vector can be constructed using the positions of multiple target endpoints and the scores corresponding to each target endpoint, and then the constructed three-dimensional vector can be mapped in a high dimension to improve the feature capabilities of the target endpoints.

[0122] S7002. Extract features from the endpoint mapping information corresponding to each of the multiple target endpoints to obtain multiple endpoint features; the endpoint features represent the feature values ​​of the connotation of the target endpoint; preferably, a self-attention mechanism can be used for feature extraction, that is, the input is any endpoint mapping information; that is, input x1=x2=x3 to formula (1)-(4).

[0123] S7003. Based on road features and multiple endpoint features, feature fusion is performed to obtain the second fused feature; preferably, the feature fusion process can be performed using a cross-attention mechanism; the input of the cross-attention mechanism is any two feature values, that is, input one feature value x1 and the other feature value x2 = x3 to formula (1)-(4).

[0124] S7004. Input the endpoint features, the second fusion features, and the vehicle driving features corresponding to the target vehicle into the trajectory prediction model to predict the trajectory and obtain multiple predicted trajectories corresponding to the target vehicle. The trajectory prediction model is obtained by intelligent recognition training based on the sample endpoint features, sample fusion features, sample vehicle driving trajectories, and sample predicted trajectories.

[0125] In this embodiment, by performing high-dimensional mapping on multiple target endpoints, the feature attributes of the target endpoints are expanded, thereby extracting the endpoint features corresponding to each of the multiple target endpoints, thus improving the feature representation capability. In addition, trajectory prediction is performed based on multiple endpoint features, road features, and vehicle trajectory features of the target vehicle, and a trajectory prediction model is used in the prediction process, which can improve the accuracy of the input data of the trajectory prediction model, thereby improving the prediction accuracy of the trajectory prediction model.

[0126] In this embodiment, the vehicle trajectory prediction method has the following beneficial effects:

[0127] By extracting the driving features of the target vehicle and surrounding vehicles, and extracting road features, and then fusing the driving features of the target vehicle and surrounding vehicles with the road features to obtain the first fused feature, trajectory prediction is performed based on the driving features of the target vehicle and surrounding vehicles, the road features, and the first fused feature. This fully considers the vehicle's own driving factors, road driving factors, and the influence of the road on vehicle driving, thereby improving the noise resistance of vehicle trajectory prediction and thus improving the accuracy of trajectory prediction.

[0128] refer to Figure 8 This application also provides a vehicle trajectory prediction device, which includes:

[0129] The acquisition module 101 is used to acquire the historical driving trajectories of the target vehicle and surrounding vehicles, as well as the driving road information of the target vehicle; the driving road information is the road information within a preset range centered on the current position of the target vehicle.

[0130] The first feature extraction module 102 is used to extract driving features from the historical driving trajectories of the target vehicle and the surrounding vehicles respectively, so as to obtain the vehicle driving features of the target vehicle and the surrounding vehicles respectively.

[0131] The second feature extraction module 103 is used to extract road features based on driving road information to obtain road features;

[0132] The first fusion module 104 is used to perform feature fusion based on the vehicle driving characteristics of the target vehicle and the surrounding vehicles, as well as the road characteristics, to obtain a first fused feature; the first fused feature represents the correlation between the vehicle driving trajectory and the driving road information.

[0133] The trajectory prediction module 105 is used to predict the trajectory of the target vehicle based on the driving road information, the driving characteristics of the target vehicle and the surrounding vehicles, the road characteristics, and the first fusion features.

[0134] The second feature extraction module includes:

[0135] The first sampling unit is used to sample lane information from the driving road information at a first sampling interval to obtain the first lane information;

[0136] The feature extraction unit is used to extract features from the first lane information to obtain road features.

[0137] The trajectory prediction module includes:

[0138] The second sampling unit is used to sample lane information from the driving road information at a second sampling interval to obtain second lane information; the first sampling interval is greater than the second sampling interval.

[0139] The first trajectory prediction unit is used to predict the trajectory of the target vehicle based on the second lane information, the vehicle driving characteristics of the target vehicle and the surrounding vehicles, the road characteristics, and the first fusion features.

[0140] The first trajectory prediction unit includes:

[0141] The probability prediction unit is used to perform probability prediction on multiple lane collection points based on the second collected information, the vehicle driving characteristics of the target vehicle and the surrounding vehicles, the road characteristics, and the first fused features, to obtain the regional center probability of each of the multiple lane collection points.

[0142] The target acquisition point determination unit is used to determine the target acquisition point based on the regional center probability corresponding to each of the multiple lane acquisition points. The target acquisition point is a number of lane acquisition points selected sequentially from front to back after sorting the multiple lane acquisition points in descending order of their respective regional center probabilities.

[0143] The associated acquisition point determination unit is used to sample the area surrounding the target acquisition point to obtain multiple associated acquisition points;

[0144] The target endpoint determination unit is used to determine multiple target endpoints based on multiple associated acquisition points; the distance between the multiple target endpoints and the nearest associated acquisition point is less than the second sampling interval;

[0145] The second trajectory prediction unit is used to predict the trajectory based on multiple target endpoints, the driving characteristics of the target vehicle and surrounding vehicles, and road features, so as to obtain multiple predicted trajectories corresponding to the target vehicle.

[0146] The target endpoint determination unit includes:

[0147] The scoring prediction unit is used to predict the scores of multiple associated collection points based on the vehicle driving features, road features, and first fusion features of the target vehicle and surrounding vehicles, so as to obtain the regional boundary scores corresponding to each of the multiple associated collection points.

[0148] The target association point determination unit is used to determine the target association point based on the regional boundary scores corresponding to each of the multiple association collection points. The target association point is a number of association collection points selected sequentially from front to back after sorting the multiple association collection points in descending order of their respective regional boundary scores.

[0149] The target endpoint prediction unit is used to predict target endpoints based on target association points and the scores corresponding to those points, thereby determining multiple target endpoints.

[0150] The target endpoint prediction unit includes:

[0151] The first high-dimensional mapping module is used to perform high-dimensional mapping based on the target association points and the scores corresponding to the target association points to obtain multiple association feature matrices;

[0152] The filtering module is used to filter elements at the same position in multiple associated feature matrices to obtain a first feature matrix; the element at each position in the first feature matrix is ​​the maximum value of the corresponding element in the multiple associated feature matrices.

[0153] The weighting module is used to perform weighted averaging on the elements at the same position in multiple correlation feature matrices to obtain the second feature matrix;

[0154] The merging module is used to merge the first feature matrix and the second feature matrix to obtain the third feature matrix;

[0155] The first model prediction module is used to input the third feature matrix into the endpoint prediction model to predict the endpoint and obtain multiple target endpoints.

[0156] The second trajectory prediction unit includes:

[0157] The second high-dimensional mapping module is used to perform high-dimensional mapping on multiple target endpoints to obtain the endpoint mapping information corresponding to each of the multiple target endpoints.

[0158] The endpoint feature extraction module is used to extract features from the endpoint mapping information corresponding to multiple target endpoints to obtain multiple endpoint features;

[0159] The second fusion module is used to perform feature fusion based on road features and multiple endpoint features to obtain the second fused features;

[0160] The second model prediction module is used to input the endpoint features, the second fusion features, and the vehicle driving features corresponding to the target vehicle into the trajectory prediction model to predict the trajectory and obtain multiple predicted trajectories corresponding to the target vehicle.

[0161] This application also provides an electronic device, which includes a processor and a memory. The memory stores at least one instruction or at least one program. The processor loads and executes the at least one instruction or at least one program to implement the vehicle trajectory prediction method described above.

[0162] Memory is used to store software programs and modules. The processor executes these stored software programs and modules to perform various functional applications and data processing. Memory can primarily consist of a program storage area and a data storage area. The program storage area stores the operating system, application programs required for functionality, etc.; the data storage area stores data created based on device usage, etc. Furthermore, memory can include high-speed random access memory (RAM) and non-volatile memory, such as at least one hard disk drive, flash memory, or other volatile solid-state storage devices. Correspondingly, memory can also include a memory controller to provide the processor with access to the memory.

[0163] The methods and embodiments provided in this application can be executed in electronic devices such as mobile terminals, computer terminals, servers, or similar computing devices. Figure 9 This is the electronic device provided in the embodiments of this application. For example... Figure 9As shown, the electronic device 900 can vary significantly due to differences in configuration or performance. It may include one or more central processing units (CPUs) 910 (CPUs 910 may include, but are not limited to, microprocessors such as MCUs or programmable logic devices such as FPGAs), a memory 930 for storing data, and one or more storage media 920 (e.g., one or more mass storage devices) for storing application programs 923 or data 922. The memory 930 and storage media 920 may be temporary or persistent storage. The program stored in the storage media 920 may include one or more modules, each module may include a series of instruction operations on the electronic device. Furthermore, the CPU 910 may be configured to communicate with the storage media 920 and execute the series of instruction operations in the storage media 920 on the electronic device 900. Electronic device 900 may also include one or more power supplies 960, one or more wired or wireless network interfaces 950, one or more input / output interfaces 940, and / or one or more operating systems 921, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0164] The input / output interface 940 can be used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the electronic device 900. In one example, the input / output interface 940 includes a network interface controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the input / output interface 940 may be a radio frequency (RF) module for wireless communication with the Internet.

[0165] Those skilled in the art will understand that Figure 9 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, the electronic device 900 may also include... Figure 9 The more or fewer components shown, or having the same Figure 9 The different configurations shown.

[0166] Embodiments of this application also provide a storage medium storing at least one instruction or at least one program, wherein the at least one instruction or at least one program is loaded and executed by a processor to implement the vehicle trajectory prediction method as described above.

[0167] The foregoing description has fully disclosed the specific embodiments of this application. It should be noted that any modifications made by those skilled in the art to the specific embodiments of this application do not depart from the scope of the claims. Accordingly, the scope of the claims of this application is not limited to the foregoing specific embodiments.

Claims

1. A vehicle trajectory prediction method, characterized by, include: Obtain the historical driving trajectories of the target vehicle and surrounding vehicles, as well as the driving road information around the target vehicle; Driving features are extracted from the historical driving trajectories of the target vehicle and the surrounding vehicles respectively to obtain the vehicle driving features of the target vehicle and the surrounding vehicles respectively. The driving road information is used to extract road features to obtain road features; Based on the vehicle driving characteristics of the target vehicle and the surrounding vehicles, as well as the road characteristics, feature fusion is performed to obtain a first fused feature; the first fused feature represents the correlation between the vehicle driving trajectory and the driving road information. Lane information is sampled from the driving road information at a second sampling interval to obtain second lane information; the second lane information includes multiple lane collection points. Based on the second lane information, the vehicle driving characteristics corresponding to the target vehicle and the surrounding vehicles, the road features, and the first fusion features, probability prediction is performed on the multiple lane collection points to obtain the regional center probability corresponding to each of the multiple lane collection points. Based on the regional central probabilities corresponding to the multiple lane collection points, target collection points are determined; the target collection points are multiple lane collection points selected sequentially from front to back after sorting the multiple lane collection points in descending order of their respective regional central probabilities. The area surrounding the target acquisition point is sampled to obtain multiple associated acquisition points; Based on the aforementioned multiple associated data collection points, multiple target endpoints are determined; The distance between the multiple target endpoints and the nearest associated collection point is less than the second sampling interval; Based on the multiple target endpoints, the vehicle driving characteristics of the target vehicle and the surrounding vehicles, and the road characteristics, trajectory prediction is performed to obtain multiple predicted trajectories corresponding to the target vehicle; the multiple predicted trajectories correspond to multiple selectable routes in autonomous driving planning.

2. The vehicle trajectory prediction method of claim 1, wherein, The road feature extraction from the driving road information yields the following road features: Lane information is sampled from the driving road information at a first sampling interval to obtain first lane information; The road features are obtained by extracting features from the first lane information.

3. The vehicle trajectory prediction method of claim 1, wherein, The determination of multiple target endpoints based on the multiple associated collection points includes: Based on the multiple associated collection points, the vehicle driving features corresponding to the target vehicle and the surrounding vehicles, the road features, and the first fusion feature, the scores of the multiple associated collection points are predicted to obtain the regional boundary scores corresponding to each of the multiple associated collection points. Based on the regional boundary scores corresponding to each of the multiple associated collection points, target associated points are determined; the target associated points are multiple associated collection points selected sequentially from front to back after sorting the multiple associated collection points in descending order based on their respective regional boundary scores. Based on the target association points and the scores corresponding to the target association points, target endpoints are predicted, and multiple target endpoints are determined.

4. The vehicle trajectory prediction method of claim 3, wherein, The step of predicting the target endpoint based on the target association points and the scores corresponding to the target association points, and determining multiple target endpoints, includes: Based on the target association points and the scores corresponding to the target association points, a high-dimensional mapping is performed to obtain multiple association feature matrices; Elements at the same position in the plurality of associated feature matrices are filtered to obtain a first feature matrix; the element at each position in the first feature matrix is ​​the maximum value of the corresponding element in the plurality of associated feature matrices. The elements at the same position in the multiple correlation feature matrices are weighted and averaged to obtain the second feature matrix; The first feature matrix and the second feature matrix are merged to obtain the third feature matrix; The third feature matrix is ​​input into the endpoint prediction model to predict the endpoint, thereby obtaining the multiple target endpoints.

5. The vehicle trajectory prediction method of claim 3, wherein, The trajectory prediction based on the multiple target endpoints, the vehicle driving characteristics corresponding to the target vehicle and the surrounding vehicles, and the road characteristics, to obtain multiple predicted trajectories corresponding to the target vehicle, include: Perform high-dimensional mapping on the multiple target endpoints to obtain endpoint mapping information corresponding to each of the multiple target endpoints; Feature extraction is performed on the endpoint mapping information corresponding to each of the multiple target endpoints to obtain multiple endpoint features; Based on the road features and the multiple endpoint features, feature fusion is performed to obtain the second fused feature; The endpoint feature, the second fusion feature, and the vehicle driving feature corresponding to the target vehicle are input into the trajectory prediction model to perform trajectory prediction, thereby obtaining multiple predicted trajectories corresponding to the target vehicle.

6. A vehicle trajectory prediction device characterized by comprising: include: The acquisition module is used to acquire the historical driving trajectories of the target vehicle and surrounding vehicles, as well as the driving road information around the target vehicle. The first feature extraction module is used to extract driving features from the historical driving trajectories of the target vehicle and the surrounding vehicles respectively, so as to obtain the vehicle driving features corresponding to the target vehicle and the surrounding vehicles respectively. The second feature extraction module is used to extract road features based on the driving road information to obtain road features; The first fusion module is used to perform feature fusion based on the vehicle driving characteristics corresponding to the target vehicle and the surrounding vehicles, as well as the road features, to obtain a first fused feature; the first fused feature represents the correlation between the vehicle driving trajectory and the driving road information. The trajectory prediction module is used to perform trajectory prediction based on the driving road information, the vehicle driving characteristics corresponding to the target vehicle and the surrounding vehicles, the road features, and the first fusion features, to obtain the trajectory prediction result of the target vehicle. The trajectory prediction module includes: The second sampling unit is used to sample lane information from the driving road information at a second sampling interval to obtain second lane information; the first sampling interval is greater than the second sampling interval. The probability prediction unit is used to perform probability prediction on multiple lane collection points based on the second lane information, the vehicle driving characteristics of the target vehicle and the surrounding vehicles, the road characteristics, and the first fusion features, to obtain the regional center probability of each of the multiple lane collection points. The target acquisition point determination unit is used to determine the target acquisition point based on the regional center probability corresponding to each of the multiple lane acquisition points. The target acquisition point is a number of lane acquisition points selected sequentially from front to back after sorting the multiple lane acquisition points in descending order of their respective regional center probabilities. The associated acquisition point determination unit is used to sample the area surrounding the target acquisition point to obtain multiple associated acquisition points; The target endpoint determination unit is used to determine multiple target endpoints based on multiple associated acquisition points; the distance between the multiple target endpoints and the nearest associated acquisition point is less than the second sampling interval; The second trajectory prediction unit is used to predict the trajectory based on multiple target endpoints, the driving characteristics of the target vehicle and surrounding vehicles, and road characteristics, to obtain multiple predicted trajectories corresponding to the target vehicle; the multiple predicted trajectories correspond to multiple selectable routes in autonomous driving planning.

7. A computer storage medium, characterized in that The storage medium stores at least one instruction, at least one program, code set, or instruction set, which is loaded and executed by a processor to implement the vehicle trajectory prediction method as described in any one of claims 1-5.

8. An electronic device, comprising: The device includes a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the vehicle trajectory prediction method as described in any one of claims 1-5.

Images